Description

As discussed in the ​django-developers mailing list, it appears that SimpleCookies in HttpResponses are not being correctly serialized when one uses either the file or the database cache backends.

The following steps are enough to trigger the incorrect behaviour:

Enable UpdateCacheMiddleware and FetchFromCacheMiddleware in settings.py, and set CACHE_BACKEND accordingly

Enable SessionMiddleware and CsrfViewMiddleware

Have a view with a simple form and no specific cache decorators. Since the session application is being used, the Vary: Cookie header will be added anyway.

In the template used by the view, include the csrf_token tag, as usual.

Access the view, either via curl or a web browser.

The first time the view is accessed, the csrf token is both set in the header as a cookie and displayed as a hidden form element, as expected. The header has the format Set-Cookie: csrftoken=XX; Max-Age: YY; Path=/.

The next times the view is accessed, the cookie header has the format Set-Cookie: csrftoken="Set-Cookie: csrftoken=XX Max-Age: YY; Path=/", and so has the csrf form element, which causes the submitted form to be invalid when the csrf checks are made.

It turns out that UpdateCacheMiddleware serializes the returned HttpResponse in process_response, and both the file and the database cache backends use pickle.dumps with protocol=pickle.HIGHEST_PROTOCOL. It is known that ​SimpleCookies are incompatible with pickle.HIGHEST_PROTOCOL. FetchFromCacheMiddleware later retrieves this same HttpResponse and returns it, however the cookies have invalid values.

Hey there. I've finally found some time to go back to this problem. The patch I've just attached is suits better in Django's unit tests -- it uses middlewares in the base test class, but by being in the base class it is easier to test all existing backends.

As for changing the locmem cache backend to use the highest pickle protocol, would you like me to create a new ticket with a patch for that?

From the Python bug report, it looks like they are going to WONTFIX it. Ideally, our SimpleCookie is exactly the same as the stdlib SimpleCookie (currently it isn't with any released version of Python, but as fixes get in to Pythno stdlib it should become that). Since we don't to diverge from stdlib if we can avoid it, we'd prefer to put the fix elsewhere.

Regarding Django 1.3.1, our current policy is that we do not backport bugfixes like this to the stable branch.