Description

To reproduce:
Create a new project. Create some registration templates (ie: templates/password_reset_form.html). Use the {% url %} template tag. For example, {% url index %}. Make sure you have a valid 'index' named url!

This bug is still present. It's not a duplicate of either of the above mentioned cases (#10747, #10976).

The same exact problem is happening for me using auth and django-registration. It doesn't matter whether I reference the URL using the view or it's name. The problem lies with the fact that the tests for both django.contrib.auth and django-registration set their ​TestCase.urls setting to override the default URLConf. This problem can be resolved by concatenating the test URL patterns and your root URL patterns. This is an unfeasible solution because it requires modifying the source code. As far as I know, this is the only real solution.

After googling for hours, I've only found two references to this problem besides this ticket:

Although this workaround does not require a source code modification to auth or django-registration, it sucks bad because all reversing errors are suppressed whether or not it's during a test. I don't really see the point of testing if things like that need to be done.

Apparently not many people use the test suite because this bug will affect anyone testing with a reverse() or 'url' tag in their template.

Possible solution:
If a ​TestCase.urls attribute is set, combine it with the ROOT_URLCONF instead of overwriting it. This could be an optional or default feature.

I've been thinking a little more about this, and I propose we could use the following options for "./manage.py test"

--combine-urlconfs: tells the testing framework to combine the overridden URLConf with the ROOT_URLCONF. If --combine-urlconfs-with is specified, combine it with that instead of ROOT_URLCONF
--combine-urlconfs-with: the URLConf to use for combining instead of ROOT_URLCONF. This is useful if you need to provide a minimal URL configuration for only the URLs you need to avoid conflicts.

Most of the errors described here are because of auth and django-registration templates including a base.html that reverses to links such as the main index page, a login page, or an about us page (most of them do). --combine-urlconfs-with would be a useful option in that case because it would allow minimalistic URL schemes to be included for just those pages.

I added the POC for the feature I described above. I made a few changes to my design, though. Instead of using command line switches, I opted to use variables in settings.py. Using command line switches would have required big changes to the testing framework. These changes are 100% backwards compatible. As long as TEST_COMBINE_URLCONFS is kept False, it will essentially be running the exact same code as before. Please keep in mind that the patch currently has no documentation changes. As far as I know, I need to change these documents:

There is a broader issue to be decided here - namely, how to set up tests that are isolated from the project settings. The patch on this ticket solves one version of the problem by adding "combine_urls" and "combine_urls_with", but other version are still lurking. Although the symptoms are different, #10976, #10747 and #7611 are all different presentations of the same root problem. We need a consolidated approach that differentiates between integration tests and application tests. Simon started a ​discussion on django-dev a while back, but it never came to resolution; we need to restart that discussion.

I agree that the testing framework needs to be more seamless. It works great for Django itself but for actual projects it's sort of clunky. Anyways, the patches I uploaded above are somewhat of a temporary fix if links to other apps are preventing you from testing parts of your project. Since this is for the testing framework, it only needs to be patched on developer copies of Django.

We are implementing a new web service with Django and spent a lot of time trying to figure out why 'manage.py test' were producing so many errors. Luckily we have found this bug report which describes the problem. We think it is important to be fixed, otherwise 'manage.py test' is useless for any project using the url tag in templates.

I sat down to write some tests on a new application and there's nothing more discouraging than to start with failure. The suggestion by Adam V in the django-dev thread was super quick and easy to avoid the problem ... only test your own apps [1]!

I've peared down the suggestion down to creating a MY_APPS tuple containing only my apps and running the tests with:

I think that some work has been done recently to isolate more closely the settings for contrib apps (e.g. TEMPLATE_DIRS is isolated in contrib.auth). This ticket doesn't seem very useful to me as it doesn't help to track specific issues.

Note that I proposed a different approach that completely isolates the TestCase settings from current project settings in #17194 (see the patch attachment:17194-3.diff:ticket:17194​). But finally an other solution was adopted for that ticket.

My take on this issue (and on the many similar ones) is that contrib apps tests aren't designed to be run with arbitrary settings, and cannot be made to without:

large amounts of work,

large override_settings clauses.

Even then, the result will be very fragile, since the CI cannot test all possible settings combinations. It will also be less readable than the current tests. In my opinion, every commit that adds a little piece of test isolation in contrib apps is a loss.

I was willing to do the work of moving the auth tests to tests/ so they won't be run by default, which would solve this problem. But Russell vetoed that. I'll let him propose another solution.

I believe this ticket can simply be closed when we merge #17365, since that will change the default behavior of manage.py test so it doesn't run contrib tests, making it feasible for us to simply say "contrib app tests aren't intended to pass with arbitrary settings."

Thanks to Preston Timmons for the bulk of the work on the patch, especially
updating Django's own test suite to comply with the requirements of the new
runner. Thanks also to Jannis Leidel and Mahdi Yusuf for earlier work on the
patch and the discovery runner.

Closing wontfix. Since the new test runner does not run contrib app tests by default in user projects, we will no longer be attempting to make contrib app tests pass with arbitrary settings, signals, templates, etc.