Getting 404 when trying to access images

I'm trying to access files stored in a media library, and get a 404.
When accessing it through the perma link (/getmedia/e674ea37-d73b-44ba-a31f-530819119402/DSC00973_120),
I get to it.
But when accessing it through /BlankSite/media/Authors/DSC00973_120.gif, I get a 404.

The weird thing, is that if I make a configuration change, like changing the web.config, I can access it for a minute or two.
Also, the request physical path (C:\inetpub\wwwroot\BlankSite\media\Authors\DSC00973_120.gif), exists.