Description

When using an ImageField and uploading an image via admin, say the filename was Test.jpg, uploading it in Linux will save the file as Test.jpg on the disk, and the reference in the database is also Test.jpg, things work normally as expected here.

However, when uploading the same file in Windows, the filename is saved on the disk as test.jpg (all lower case), but in the database is still referenced as Test.jpg. Since the filesystem in Windows is not case sensitive, this is not really much of an issue immediately. If you however, then copy your Django based site back to a Linux based machine, you will get a missing image, as it is expected to find the file "Test.jpg" in the filesystem, however the file is "test.jpg". In this case, this can become a bit of a problem.

I am not sure if the problem is related to Django or Python on Windows.

If you want to move the process along, the next step would be posting a patch that fixes the issue, passes all existing tests, and contains a test that fails on the current code (it's OK if it fails only on Windows -- but it can't rely on a particular configuration like a networked drive) and succeeds with the fix.

I think that, contrarily to what SmileyChris suggests, it is worth going for the real and correct solution to this problem as the reporter of #9442 suggests rather than adding a workaround.

By correct solution I mean making sure Django uses the same path (case-wise) both for:

the file written to disk

and the path stored in the database

when storing a FileField/ImageField in all supported platforms.

I've attached a patch (8593-regression-tests.diff) that add tests to the test suite that show how currently, when running on Windows (whose file systems are case-insensitive but case-preserving), Django is lowercasing the path used to create the file in the file system.

These new tests shouldn't fail on other platfoms (i've tested also on Linux).

8593-r9832.diff implements a fix for this issue by modifying Django's django.utils._os.safe_join to not call os.path.normcase() under Windows, the modification has been implemented in a way that doesn't break the security safeguards that are the raison d'être of such function.

As safe_join() is also used by the template loaders, the case-insensitive-filesystems-specific template regression tests have been also modified (AFAICS these tests as implemented aren't really useful because they aren't using a template file path as written to the file system when comparing paths but rather two Django maintained in-memory representations of such path.)

Net effect of the path is that now Django doesn't use a path in the file system that differs from the path stored in the DB for uploaded files, irrespective of platform. This should help when moving projects and deployments form platform to platform.

The case is more complex. On OS X the IO system performs unicode normalization for the filename after the file was stored. This is fine until you mount the volume on a linux machine or access it over the network. I'm not sure what the fix is, OS X normalizes in the IO system somehow, it also affects IO access on NFS mounts and similar volumes. You can't really find out how the normalization works from inside Python.