Django: Ticket #19362: Recursion error when deleting model object through adminhttps://code.djangoproject.com/ticket/19362
<p>
When I try to delete a specific object using the admin interface I get an error:
</p>
<pre class="wiki">RuntimeError at /admin/gtd/node/1470/delete/
maximum recursion depth exceeded while calling a Python object
</pre><p>
It appears to be trying to convert something to unicode but I don't know what. From the Django error message I can see that the object appears as <tt>&lt;Node: [&lt;span style="color: rgba(230, 138, 0, 1.0)"&gt;&lt;strong&gt;DFRD&lt;/strong&gt;&lt;/span&gt;] Test todo item&gt;</tt>, which doesn't seem to have any weird characters in it. The title is what I expect to be return from <tt>__str__(self)</tt> as described below.
</p>
<p>
If I delete a different object (which has similar HTML markup in it) from the same model it works with no errors. The model in question defines <tt>__str__(self)</tt> which returns the value from a models.TextField() attribute wrapped in some HTML (&lt;span style="..."&gt;&lt;/span&gt;) which is passed through mark_safe() before being returned. The model is decorated with <tt>@python_2_unicode_compatible</tt>. Following the python3 migration guide, I added <tt>from __future__ import unicode_literals</tt> at the top of my models.py and removed <tt>__unicode__(self)</tt> but no difference.
</p>
<p>
The stacktrace doesn't seem to go through my code anywhere.
</p>
<p>
I git-pulled the latest changes to stable/1.5.x from github and reinstalled Django after removed the previous installation from dist-packages but the problem persists.
</p>
<p>
I have not tried deleting this object pythonically; I figured I'd keep that object for now in case anyone needs me to reproduce the problem.
</p>
<p>
Stacktrace: <a class="ext-link" href="https://gist.github.com/4148524"><span class="icon">​</span>https://gist.github.com/4148524</a>
</p>
en-usDjangohttps://www.djangoproject.com/s/img/site/hdr_logo.gifhttps://code.djangoproject.com/ticket/19362
Trac 1.0.4akaariaiMon, 26 Nov 2012 16:49:38 GMTstage changed; needs_better_patch, needs_docs, needs_tests sethttps://code.djangoproject.com/ticket/19362#comment:1
https://code.djangoproject.com/ticket/19362#comment:1
<ul>
<li><strong>needs_better_patch</strong>
unset
</li>
<li><strong>needs_docs</strong>
unset
</li>
<li><strong>needs_tests</strong>
unset
</li>
<li><strong>stage</strong>
changed from <em>Unreviewed</em> to <em>Accepted</em>
</li>
</ul>
<p>
I was able to reproduce similar stack trace with:
</p>
<pre class="wiki">from django.db import models
from django.utils.encoding import python_2_unicode_compatible
@python_2_unicode_compatible
class Foof(models.Model):
def __unicode__(self):
return unicode(self.id)
repr(Foof())
</pre><p>
I don't get a similar stack trace if I define <tt>__str__</tt> instead of <tt>__unicode__</tt> or if I don't have python_2_unicode_compatible.
</p>
<p>
I have also seen the same stack trace somewhere else, don't remember where and why.
</p>
<p>
There seems to be something strange going on. In the following code
</p>
<pre class="wiki"> def __str__(self):
if not six.PY3 and hasattr(self, '__unicode__'):
return force_text(self).encode('utf-8')
</pre><p>
if I replace the force_text(self) with <tt>self.__unicode__()</tt> then the recursion goes away. But to me it seems calling the <tt>__unicode__</tt> method is the only thing that force_text is actually doing (apart of a couple of isinstance checks).
</p>
<p>
In any case it seems the <tt>__unicode__</tt> + force_text() form an endless loop in some cases. I am marking this as accepted as it seems likely there is something strange going on in here. Although it is possible this is some kind of user error after all.
</p>
TicketaaugustinMon, 26 Nov 2012 20:27:47 GMThttps://code.djangoproject.com/ticket/19362#comment:2
https://code.djangoproject.com/ticket/19362#comment:2
<p>
Anssi first encountered the problem by checking out stable/1.4.x, running the pagination tests, checking out master, and running the pagination tests again.
</p>
<p>
(The pagination tests were moved from <tt>modeltests</tt> to <tt>regressiontests</tt>; the sequence above leaves stale <tt>.pyc</tt> files in <tt>modeltests/pagination</tt>.)
</p>
<p>
At least, that gives us a reproducible way to trigger this infinite recursion.
</p>
TicketaaugustinMon, 26 Nov 2012 21:05:53 GMThttps://code.djangoproject.com/ticket/19362#comment:3
https://code.djangoproject.com/ticket/19362#comment:3
<p>
Indeed, replacing <tt>force_text(self)</tt> with <tt>self.__unicode__()</tt> makes the recursion go away. But replacing it with <tt>unicode(self)</tt> (or<tt>type(self).__unicode__(self)</tt>, which is how the interpreter computes <tt>unicode(self)</tt>) brings it back.
</p>
<p>
In the situation explained just above, it seems that Django will run the tests from the stale <tt>.pyc</tt> in <tt>modeltests</tt>, but the <tt>Article</tt> model from <tt>regressiontests</tt> is also loaded in the app cache, and that causes problems.
</p>
TicketaaugustinMon, 26 Nov 2012 21:47:37 GMThttps://code.djangoproject.com/ticket/19362#comment:4
https://code.djangoproject.com/ticket/19362#comment:4
<p>
Here's what happens when a subclass of <tt>models.Model</tt> that doesn't define <tt>__str__</tt> is decorated with <tt>python_2_unicode_compatible</tt>:
</p>
<ul><li>when the decorator is executed:
<ul><li>its <tt>__unicode__</tt> method points to the <tt>__str__</tt> method from <tt>models.Model</tt> — which returns a <tt>str</tt> and not a <tt>unicode</tt> as expected by <tt>python_2_unicode_compatible</tt>
</li><li>its <tt>__str__</tt> method is a new method that calls <tt>__unicode__</tt> and returns the result encoded as utf-8
</li></ul></li><li>when attempting to create the unicode representation of an instance, say, <tt>obj</tt>:
<ul><li>the interpreter treats <tt>unicode(obj)</tt> as <tt>type(obj).__unicode__(obj)</tt>
</li><li>in this case, that's actually <tt>models.Model.__str__(obj)</tt>
</li><li>this function calls <tt>force_text(obj)</tt>, which calls<tt>obj.__unicode__()</tt> — and that's also <tt>models.Model.__str__(obj)</tt>, hence the infinite recursion.
</li></ul></li></ul><hr />
<p>
I wish I could just remove the definition of <tt>models.Model.__str__</tt>, but after having spent years educating developers to write a <tt>__unicode__</tt> method for their models, this isn't a viable option.
</p>
<p>
It's possible to catch this programming mistake and raise an explicit exception at runtime like this:
</p>
<pre class="wiki">--- a/django/db/models/base.py
+++ b/django/db/models/base.py
@@ -416,6 +416,11 @@ class Model(six.with_metaclass(ModelBase, object)):
def __str__(self):
if not six.PY3 and hasattr(self, '__unicode__'):
+ if type(self).__unicode__ == Model.__str__:
+ klass_name = type(self).__name__
+ raise RuntimeError("%s.__unicode__ is aliased to __str__. Did"
+ " you apply @python_2_unicode_compatible"
+ " without defining __str__?" % klass_name)
return force_text(self).encode('utf-8')
return '%s object' % self.__class__.__name__
</pre><p>
That ugly, but it works... It's more difficult to catch the exception at compile time because <tt>python_2_unicode_compatible</tt> is defined in <tt>django.utils.encoding</tt>, and I don't want to import <tt>django.db.models.Model</tt> there.
</p>
<hr />
<p>
m3wolf, I have an explanation for why you can delete one instance and not another of the same model.
</p>
<p>
You probably have a foreign key pointing from an instance of another model to the instance you can't delete, and that other model has <tt>@python_2_unicode_compatible</tt> but no <tt>__str__</tt> method. When Django tries to display that instance to warn you about the cascade deletion, it enters the infinite loop.
</p>
<p>
If you apply the patch above, Django should tell you which model has this issue. Could you test that and report your results here?
</p>
Ticketm3wolfTue, 27 Nov 2012 00:28:04 GMThttps://code.djangoproject.com/ticket/19362#comment:5
https://code.djangoproject.com/ticket/19362#comment:5
<p>
Thanks, aaugustine. Applied your patch and it correctly identified the offending model. Defined <tt>__str__</tt> for that model and I can now delete the instance.
</p>
TicketaaugustinTue, 27 Nov 2012 08:56:00 GMThas_patch, component, severity, needs_tests changedhttps://code.djangoproject.com/ticket/19362#comment:6
https://code.djangoproject.com/ticket/19362#comment:6
<ul>
<li><strong>has_patch</strong>
set
</li>
<li><strong>component</strong>
changed from <em>Uncategorized</em> to <em>Python 2</em>
</li>
<li><strong>severity</strong>
changed from <em>Normal</em> to <em>Release blocker</em>
</li>
<li><strong>needs_tests</strong>
set
</li>
</ul>
TicketAymeric Augustin <aymeric.augustin@…>Tue, 27 Nov 2012 08:57:07 GMTstatus changed; resolution sethttps://code.djangoproject.com/ticket/19362#comment:7
https://code.djangoproject.com/ticket/19362#comment:7
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
</ul>
<p>
In <a class="changeset" href="https://code.djangoproject.com/changeset/2ea80b94d7f6e0d25207d6b716c52ca4c57a02fb" title="Fixed #19362 -- Detected invalid use of @python_2_unicode_compatible.
...">2ea80b94d7f6e0d25207d6b716c52ca4c57a02fb</a>:
</p>
<div class="message"><p>
Fixed <a class="closed ticket" href="https://code.djangoproject.com/ticket/19362" title="Bug: Recursion error when deleting model object through admin (closed: fixed)">#19362</a> -- Detected invalid use of @python_2_unicode_compatible.<br />
</p>
<p>
Thanks m3wolf for the report and akaariai for reproducing the problem.<br />
</p>
</div>
TicketAymeric Augustin <aymeric.augustin@…>Tue, 27 Nov 2012 08:57:09 GMThttps://code.djangoproject.com/ticket/19362#comment:8
https://code.djangoproject.com/ticket/19362#comment:8
<p>
In <a class="changeset" href="https://code.djangoproject.com/changeset/71e5ad248e104f68a0d755cd5f76e86631607be5" title="[1.5.x] Fixed #19362 -- Detected invalid use of ...">71e5ad248e104f68a0d755cd5f76e86631607be5</a>:
</p>
<div class="message"><p>
[1.5.x] Fixed <a class="closed ticket" href="https://code.djangoproject.com/ticket/19362" title="Bug: Recursion error when deleting model object through admin (closed: fixed)">#19362</a> -- Detected invalid use of @python_2_unicode_compatible.<br />
</p>
<p>
Thanks m3wolf for the report and akaariai for reproducing the problem.<br />
</p>
<p>
Backport of 2ea80b9.<br />
</p>
</div>
TicketaaugustinSun, 13 Oct 2013 15:58:57 GMThttps://code.djangoproject.com/ticket/19362#comment:9
https://code.djangoproject.com/ticket/19362#comment:9
<p>
<a class="closed ticket" href="https://code.djangoproject.com/ticket/21198" title="Bug: @python_2_unicode_compatible, abstract models, working on 1.4 but ... (closed: fixed)">#21198</a> suggests a better approach: raising an exception at compile time rather than at runtime.
</p>
TicketAymeric Augustin <aymeric.augustin@…>Sun, 13 Oct 2013 16:25:46 GMThttps://code.djangoproject.com/ticket/19362#comment:10
https://code.djangoproject.com/ticket/19362#comment:10
<p>
In <a class="changeset" href="https://code.djangoproject.com/changeset/f0c7649b1692f8441eb9b9b923b2bed8e95f9185" title="Fixed #21198 -- Prevented invalid use of @python_2_unicode_compatible. ...">f0c7649b1692f8441eb9b9b923b2bed8e95f9185</a>:
</p>
<div class="message"><p>
Fixed <a class="closed ticket" href="https://code.djangoproject.com/ticket/21198" title="Bug: @python_2_unicode_compatible, abstract models, working on 1.4 but ... (closed: fixed)">#21198</a> -- Prevented invalid use of @python_2_unicode_compatible.<br />
</p>
<p>
Thanks jpic for the report and chmodas for working on a patch.<br />
</p>
<p>
Reverts 2ea80b94. Refs <a class="closed ticket" href="https://code.djangoproject.com/ticket/19362" title="Bug: Recursion error when deleting model object through admin (closed: fixed)">#19362</a>.<br />
</p>
<p>
Conflicts:<br />
</p>
<blockquote>
<p>
tests/utils_tests/test_encoding.py<br />
</p>
</blockquote>
</div>
TicketAymeric Augustin <aymeric.augustin@…>Sun, 13 Oct 2013 16:25:47 GMThttps://code.djangoproject.com/ticket/19362#comment:11
https://code.djangoproject.com/ticket/19362#comment:11
<p>
In <a class="changeset" href="https://code.djangoproject.com/changeset/589dc49e129f63801c54c15e408c944a345b3dfe" title="Fixed #21198 -- Prevented invalid use of @python_2_unicode_compatible. ...">589dc49e129f63801c54c15e408c944a345b3dfe</a>:
</p>
<div class="message"><p>
Fixed <a class="closed ticket" href="https://code.djangoproject.com/ticket/21198" title="Bug: @python_2_unicode_compatible, abstract models, working on 1.4 but ... (closed: fixed)">#21198</a> -- Prevented invalid use of @python_2_unicode_compatible.<br />
</p>
<p>
Thanks jpic for the report and chmodas for working on a patch.<br />
</p>
<p>
Reverts 2ea80b94. Refs <a class="closed ticket" href="https://code.djangoproject.com/ticket/19362" title="Bug: Recursion error when deleting model object through admin (closed: fixed)">#19362</a>.<br />
</p>
</div>
Ticket