Django: Ticket #16609: Admin searches produce duplicate results if search_fields contains M2M traversalshttps://code.djangoproject.com/ticket/16609
<p>
I believe this has already been touched upon in <a class="new ticket" href="https://code.djangoproject.com/ticket/15819" title="#15819: Bug: Admin searches should use distinct, if query involves joins (new)">#15819</a>, however the consensus (comment <a class="new ticket" href="https://code.djangoproject.com/ticket/15819#comment:13" title="#15819: Bug: Admin searches should use distinct, if query involves joins (new)">ticket:15819#comment:13</a>) was
</p>
<p>
"For people in situations where <code>.distinct()</code> is broken for whatever reason, the workaround here is simple: don't use <code>search_fields</code> with relations in a way that triggers it."
</p>
<p>
I couldn't find a ticket for this, so here it is.
</p>
<p>
In my case, I am searching for a keyword which is found in a <code>search_fields</code> item that isn't traversing M2M. When <code>search_fields</code> also contains a M2M traversal, I get duplicate results.
</p>
<p>
In my case - this might be relevant or not - I am traversing a ForeignKey first and then an M2M field in that problematic search_fields entry, e.g:
</p>
<pre class="wiki">search_fields = ['=id', '=fkmodel__m2mmodel__name']
</pre><p>
This happens even though the string is not matched by <code>fkmodel__m2mmodel__name</code>, only by <code>id</code>.
</p>
<p>
This might not be clear, this is what the model relations look like:
</p>
<pre class="wiki">(Model SearchedModel) --ForeignKey--&gt; (Model FKModel) --ManyToMany--&gt; (Model M2MModel)
</pre><p>
M2MModel contains the name field that I have specified in <code>search_fields</code>.
</p>
<p>
This might or might not be related to <a class="new ticket" href="https://code.djangoproject.com/ticket/15559" title="#15559: Bug: Distinct queries will cause errors with some custom model fields (new)">#15559</a>.
</p>
<p>
I have tested this against 1.3, yesterday's 1.3.X, and yesterday's dev and it is broken in all of them. (today is 10 Aug 2011)
</p>
en-usDjangohttps://www.djangoproject.com/s/img/site/hdr_logo.gifhttps://code.djangoproject.com/ticket/16609
Trac 1.2.2cheater <cheater00@…>Wed, 10 Aug 2011 09:55:34 GMTcc sethttps://code.djangoproject.com/ticket/16609#comment:1
https://code.djangoproject.com/ticket/16609#comment:1
<ul>
<li><strong>cc</strong>
<em>cheater00@…</em> added
</li>
</ul>
Ticketcheater <cheater00@…>Wed, 10 Aug 2011 09:58:00 GMThttps://code.djangoproject.com/ticket/16609#comment:2
https://code.djangoproject.com/ticket/16609#comment:2
<p>
Sorry about not using code blocks above - only noticed the error after submitting the ticket. The search_fields should look like this:
</p>
<pre class="wiki">search_fields = ['=id', '=fkmodel__m2mmodel__name']
</pre>
TicketAymeric AugustinThu, 11 Aug 2011 22:14:22 GMTdescription changedhttps://code.djangoproject.com/ticket/16609#comment:3
https://code.djangoproject.com/ticket/16609#comment:3
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/16609?action=diff&amp;version=3">diff</a>)
</li>
</ul>
<p>
Fixed formatting.
</p>
TicketAymeric AugustinThu, 11 Aug 2011 22:15:17 GMTdescription changedhttps://code.djangoproject.com/ticket/16609#comment:4
https://code.djangoproject.com/ticket/16609#comment:4
<ul>
<li><strong>description</strong>
modified (<a href="/ticket/16609?action=diff&amp;version=4">diff</a>)
</li>
</ul>
TicketAymeric AugustinThu, 11 Aug 2011 22:16:57 GMTstage changedhttps://code.djangoproject.com/ticket/16609#comment:5
https://code.djangoproject.com/ticket/16609#comment:5
<ul>
<li><strong>stage</strong>
changed from <em>Unreviewed</em> to <em>Accepted</em>
</li>
</ul>
<p>
It's likely that this is an ORM bug, but since the symptoms show in the admin, I'll leave it like this until we have more information.
</p>
Ticketcheater <cheater00@…>Fri, 12 Aug 2011 09:24:54 GMThttps://code.djangoproject.com/ticket/16609#comment:6
https://code.djangoproject.com/ticket/16609#comment:6
<p>
For one thing, the admin actions also get doubled items. In fact, it is so that this could lead to fairly serious data loss and other errors. Imagine an admin action which bills users for something - they get found twice - and they get billed twice! I think this warrants bumping up the importance. I have also stumbled upon this when I was writing a money-related summary which counted the amount of items in stock, and summarized them, also calculating the prices. The prices were way off, of course. I have done a workaround wich skips objects that have already been seen by tracking the ID, but some other people might not be so lucky as to notice the problem before it's too late.
</p>
Ticketandreas@…Mon, 22 Apr 2013 07:25:30 GMThttps://code.djangoproject.com/ticket/16609#comment:7
https://code.djangoproject.com/ticket/16609#comment:7
<p>
Has this bug been fixed in later versions of django? Still broken in 1.3.
</p>
TicketTimothy SchillingSat, 21 Sep 2013 11:06:21 GMTcc changedhttps://code.djangoproject.com/ticket/16609#comment:8
https://code.djangoproject.com/ticket/16609#comment:8
<ul>
<li><strong>cc</strong>
<em>Timothy Schilling</em> added
</li>
</ul>
<p>
This bug has been fixed by at least 1.3.7 and it's fixed in master.
</p>
TicketTim GrahamSat, 21 Sep 2013 11:16:25 GMTstatus changed; resolution sethttps://code.djangoproject.com/ticket/16609#comment:9
https://code.djangoproject.com/ticket/16609#comment:9
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
</ul>
TicketSamuel PaccoudTue, 14 Apr 2015 11:31:01 GMThttps://code.djangoproject.com/ticket/16609#comment:10
https://code.djangoproject.com/ticket/16609#comment:10
<p>
I came across this bug which does not seem to be totally fixed yet? I tried to propose a fix and submitted a PR: <a class="ext-link" href="https://github.com/django/django/pull/4504"><span class="icon">​</span>https://github.com/django/django/pull/4504</a>
</p>
TicketSamuel PaccoudTue, 14 Apr 2015 11:31:40 GMTstatus changed; resolution deletedhttps://code.djangoproject.com/ticket/16609#comment:11
https://code.djangoproject.com/ticket/16609#comment:11
<ul>
<li><strong>status</strong>
changed from <em>closed</em> to <em>new</em>
</li>
<li><strong>resolution</strong>
<em>fixed</em> deleted
</li>
</ul>
TicketTim GrahamTue, 14 Apr 2015 12:52:24 GMThas_patch changedhttps://code.djangoproject.com/ticket/16609#comment:12
https://code.djangoproject.com/ticket/16609#comment:12
<ul>
<li><strong>has_patch</strong>
set
</li>
</ul>
<p>
I wonder if we might mark this as a duplicate of <a class="new ticket" href="https://code.djangoproject.com/ticket/15819" title="#15819: Bug: Admin searches should use distinct, if query involves joins (new)">#15819</a>. The proposed patch there removes <code>lookup_needs_distinct()</code> instead of making it more complicated and looks like it would fix other issues.
</p>
TicketTim GrahamTue, 21 Apr 2015 17:26:02 GMTneeds_better_patch changedhttps://code.djangoproject.com/ticket/16609#comment:13
https://code.djangoproject.com/ticket/16609#comment:13
<ul>
<li><strong>needs_better_patch</strong>
set
</li>
</ul>
TicketTim Graham <timograham@…>Wed, 06 May 2015 13:55:13 GMTstatus changed; resolution sethttps://code.djangoproject.com/ticket/16609#comment:14
https://code.djangoproject.com/ticket/16609#comment:14
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
</ul>
<p>
In <a class="changeset" href="https://code.djangoproject.com/changeset/d084176cc1273d5faf6f88eedb4c490e961f3a68" title="Fixed #16609 -- Fixed duplicate admin results when searching nested ...">d084176c</a>:
</p>
<div class="message"><p>
Fixed <a class="closed ticket" href="https://code.djangoproject.com/ticket/16609" title="#16609: Bug: Admin searches produce duplicate results if search_fields contains M2M ... (closed: fixed)">#16609</a> -- Fixed duplicate admin results when searching nested M2M relations.<br />
</p>
<p>
This was fixed earlier but only when the M2M relation was at the first<br />
level on the object. This commit fixes the issue even when the M2M is<br />
at deeper levels, such as behind a foreign key.<br />
</p>
</div>
Ticket