Traceback (most recent call last):
Module lazr.jobrunner.jobrunner, line 194, in runJobHandleError
self.runJob(job, fallback)
Module lp.services.job.runner, line 289, in runJob
super(BaseJobRunner, self).runJob(IRunnableJob(job), fallback)
Module lazr.jobrunner.jobrunner, line 162, in runJob
job.run()
Module lp.soyuz.model.packagecopyjob, line 578, in run
self.attemptCopy()
Module lp.soyuz.model.packagecopyjob, line 675, in attemptCopy
to_spr.requestDiffTo(self.requester, from_spr)
Module lp.soyuz.model.sourcepackagerelease, line 392, in requestDiffTo
candidate = self.getDiffTo(to_sourcepackagerelease)
Module lp.soyuz.model.sourcepackagerelease, line 388, in getDiffTo
from_source=self, to_source=to_sourcepackagerelease)
Module storm.sqlobject, line 356, in selectOneBy
return SQLObjectResultSet(cls, by=kwargs)._one()
Module storm.sqlobject, line 516, in _one
return detuplelize(self._result_set.one())
Module storm.store, line 1164, in one
raise NotOneError("one() used with more than one result available")
NotOneError: one() used with more than one result available

This is not the only case where that assumption has been violated. On dogfood:

The timestamps of each group from "SELECT * FROM PackageDiff NATURAL JOIN (SELECT from_source, to_source FROM PackageDiff GROUP BY from_source, to_source HAVING count(*) > 1) AS dups ORDER BY from_source, to_source, date_requested;" are all pretty close together: the largest gap I saw was under 0.4 seconds. I think therefore that this is probably a race in SourcePackageRelease.requestDiffTo: it checks whether a candidate has already been requested, but it's perfectly possible for multiple simultaneous transactions to pass this test and create the same diff before committing the transaction. Indeed, for instance, it's not entirely improbable in the case of the same package being copied to multiple series at the same time.

I think we should at least have a unique constraint on from_source/to_source; perhaps the simplest code fix would then be to try store.flush() right after creating the PackageDiff object and check for IntegrityError, which is a pattern we use in a couple of other places. Regardless, once we have a fix in place we should clean up the existing duplicate rows as well.

The duplicates have been deleted from production, and a new unique index added to prevent more from showing up. The racing copies will now crash without corrupting the DB, and a simple retry should work.

Changed in launchpad:

assignee:

William Grant (wgrant) → nobody

status:

In Progress → Triaged

summary:

- Package diffs should be unique by from_source/to_source but aren't+ Racing package copies crash when trying to create duplicate PackageDiffs