Delete hidden files starting with ~$?

I am trying to run a script on user folders to cleanup .tmp files and office temp files that start with ~$ that are hidden.  Here is my script:

$SourceDir = "\\Server\Students\"

foreach ($file in Get-ChildItem -Force $SourceDir -Recurse)
{
if (($file.Extension -match ".tmp") -or ($file.Name -like "~$*"))
	{
	Remove-Item $file.FullName -Force | Out-Null
	}
}

It seems to work fine for the .tmp files but doesn't seem to work at all on the ~$ files? I know they are hidden so I added -Force on Get-ChildItem but still no luck. I am sure it is something simple, but any hints?