Can you see the file in a file manager such as Nautilus or mc, if you enable "show hidden files" (Ctrl+H in Nautilus)? Can you delete it then? A file management program should make the unlink system calls directly, bypassing any shell interpretation etc.

You type rm .#filename.txt using file completion and get an error meassage of "File not found"? Is this correct? You might try temporary move all other .txt file in that directory to another directory and the try again with rm *.txt.
But you haven't told us if you can display the contents, also if you don't what it is, google the name and see what come up. I did that a few years ago and it turned out to be the conficker worm.

Not quite sure, but ":" requires to have an \ previous in order to use it for filenames.
I assume the same works for #...
Asuming that the dot in the file name indicates a hidden file, and should not mark the 'current dir'?

I think my suggestion using "find" could work with these cases. Human intervention is needed to identify the file, though, but I guess this is inevitable. It is inevitable because you don't know the machine-readable form of the filename in the first place. We have to assume the filename is completely garbage.

I'm still a bit worried about where the files came from - there were four, in different directories, both were the names of other innocuous files just with a .# in front of them. one was a .tex and there were .txt files.

They were all sim-links to non-existent files with names of the format user@localhost.13571:1318356082 (so a bit like a maildir message format).

They all had lrwxrwxrwx permissions.

What programme might produce files like that? I'm guessing emacs as that is what I might have used to edit .tex and .txt files.

I'm still a bit worried about where the files came from - there were four, in different directories, both were the names of other innocuous files just with a .# in front of them. one was a .tex and there were .txt files.