On line 1, you are guilty of stringificiation. You don't need to put quotes around a variable like this, when the variable is all by itself; the habit can become dangerous when you start using hard references. (Resource: "NO, THAT'S WRONG!" -- Common Perl Pitfalls (TPJ, Vol. 4, No. 2))

The rest of the code brings me to the dual nature of readdir(). You can use it to get filenames one at a time, or all at once. If you're going to get them all at once, it's easiest to do whatever filtering you need to do then, rather than store unwanted elements in an array that you need to weed through later: