Description

Once the hot club of france opens and we have more third party apps, the probability of clashing app names will increase. We can fix this by allowing users to define a custom app_label in their settings file. In addition, we can add a verbose_name for apps at the same time. The patch here also correctly assigns and app_label to models in a package as long as you import all of those models in the packages __init__.py This possible breaks quite a few internals if people are using them, and needs to wait until after 0.96 to be considered.

In management.py, why is the import change on line 366 (of the original) necessary? I think there's some subtlety escaping me there (if it's not necessary, it's probably worth removing so that we can tell the real changes from the stylistic ones).

(this question left blank, because it was stupid and I worked out I was missing something obvious.)

Do you really mean you have to import all models in an app's __init__. py (a big change and not ideal, but maybe inavoidable; I realise this is a bit of a tarpit) or just in app_name/models/__init__.py? The stuff in loading.py make me suspect you might mean the latter, but I might be reading it incorrectly.

First of all, thanks for taking a look at this, and second, none of this is meant to be final. This is the "get it working" version that I more or less only put up as a reference for another ticket that this fixes.

In management.py, why is the import change on line 366 (of the original) necessary? I think there's some subtlety escaping me

there (if it's not necessary, it's probably worth removing so that we can tell the real changes from the stylistic ones).

It has an effect. Changing the current import from from django.db.models import get_modelsfrom django.db.models import get_app, get_models to
would get the right effect.

Do you really mean you have to import all models in an app's __init__. py (a big change and not ideal, but maybe inavoidable; I realise this is a bit of a tarpit) or just in app_name/models/__init__.py? The stuff in loading.py make me suspect you might mean the latter, but I might be reading it incorrectly.

It's the latter. That could probably be worked around as well, but I haven't spent much time thinking about that part of it yet.

It looks like we have a little bit of end-goal overlap with #4144. The patch over there is just a simple move of a single line, but it relates to how Django determines app_label, so it looks like that might be best handled in this ticket.

Essentially, this patch moves the model_module.__name__.split('.')[-2] into get_app_label, and in fact seems to remove any need for sys.modules at all (unless I'm reading something wrong). If the sys.modules line referenced in #4144 were to be removed as part of this ticket, #4144 would be unnecessary, and I'd be very happy.

I've added an updated patch. This covers changes to make-messages.py and also documentation changes, and is against trunk revision r5146. All tests in runtests.py pass, plus browsing data from apps with a custom app label in admin and databrowse works fine. Here are the relevant changes to my settings.py:

Marty: Although my latest patch still leaves the line where it is, the model_module is not being used at all. I have removed the line entirely from my working copy - once I get other feedback on the patch, I will post another patch which incorporates the feedback.

That ticket would have fixed the problem of app_label being required when using the Django ORM outside of an actual Django application. I often write programs at my job which use the Django ORM for non-website programming, but I always have to add the Meta class to the model definition, which is ugly and looks like black magic to my co-workers who aren't familiar with Django internals:

Eli, app_label is required to cleanly split models up into multiple files inside a models module. And I'm doing my best to keep all of the app_label-related stuff here in this ticket so we can get a clean solution that solves all of the associated issues.

The latest patch doesn't work against the current version of the Django trunk in svn. Nor does it work against r5171, which is the revision that patch 4 is said to work against, nor does it work against r5343, which was the latest revision on the date that the most recent patch was uploaded.

Someone needs to update the patch to work with the current version of Django. I won't have time to do this for at least the next few weeks, and I'm probably the wrong person to do it anyway, since I only use the Django ORM and don't write Django webapps. However, if no one else gets to this by the time I have some more free time, I'll take a stab at it.

I'm surprised that the last patch doesn't version with the earlier SVN version of Django - before uploading the patch, I had checked that all tests passed. Never mind - I'll try to look at this over the next week or two.

I tested this patch and updated to apply to current trunk (r6635). All works for me.

However, I do have a question why get_installed_app_paths exists? Why can't it be done in __init__ of Settings when it expands out .* app paths? That way there isn't a need to change all instances of settings.INSTALLED_APPS. Making this completely backward compatible.

Good call - I'd done it the other way because it leaves INSTALLED_APPS untouched, and I wanted the impact of the patch to be clearly visible at least until the overall idea was approved. However, though it's been quite a long time - 8 months - since my first patch, and though there have been no adverse comments and a few comments with the gist "it just works", the devs have not seen fit to pronounce on it - so I'm treading water until I get a pronouncement about it. I just occasionally check to see if trunk changes break the patch, and update the patch accordingly. I'll merge your changes into mine so that my next patch has your changes, too.

However, I do have a question why get_installed_app_paths exists? Why can't it be done in __init__ of Settings when it expands out .* app paths? That way there isn't a need to change all instances of settings.INSTALLED_APPS. Making this completely backward compatible.

I had another look at the get_installed_app_paths vs. INSTALLED_APPS issue. With my patch, INSTALLED_APPS can contain either package names (explicit or wild-card) or app instances. However, get_installed_app_paths always returns a set of strings - the package paths of the applications. This is used, as you've seen, in a lot of places. If a user puts an app instance into INSTALLED_APPS, I'm not sure they'd take kindly to having it automatically replaced with the corresponding path string. So, get_installed_app_paths insulates the rest of the code from having to know whether the INSTALLED_APPS entries are path strings or app instances. It seems to me that some kind of encapsulation will be needed - and get_installed_app_paths performs this function. I would like to be able to use INSTALLED_APPS and do away with get_installed_app_paths - but I'm not quite sure how, yet. If you provide a patch which sorts out this issue, I'll happy incorporate it, as I mentioned.

In your testing, did you have any app instances in your INSTALLED_APPS? I'd be interested in seeing what your INSTALLED_APPS looks like. From a quick inspection of your patch, I would expect some tests to fail if INSTALLED_APPS contained any app instances, because in some places where framework code expects a string, it would get an app instance.

No, I don't believe #6080 overlaps with this ticket. That ticket is to do with loading apps from eggs; this ticket allows for easy specification of an app_label to disambiguate apps which end in the same name (e.g. 'django.contrib.admin' clashing with 'myapp.mypackage.admin'), and allows verbose names with i18n support for use in the admin (e.g. 'Authentication/Authorization' rather than 'auth').

I believe another feature that would improve admin index page usability a lot belongs conceptually here.

Suppose I have a project that contains 10 applications, each containing several models. Some of the apps are of primay importance, some are less important. Currently, there is no way to impose either ordering or hide app contents. Thus it's hard for users to discern important bits from non-important ones and confusion is guaranteed.

AppAdmin classes could be implicitly created for apps that don't provide one.

I'm not familar with nfa yet (as I'm currently using trunk without admin), so this might be a painfull change.
But it should be doable with a little refactoring (app level permission code then should probably move to AppAdmin as well).

I should also throw out there that while Django itself might only use these options in the admin, other apps may make good use of them as well. My dbsettings app, for instance, shows application names on its configuration screen, and I'd rather not reach into the guts of the admin just to get the app's verbose_name. After all, verbose_name isn't being moved out into an admin directive in newforms-admin, is it? I don't see why an app's verbose_name should be treated any differently.

I think we should think of this matter not as a problem with the admin as a general inflexibility of the app loading mechanism.

This results, for example, in the weird behaviour that only apps with a models.py are loaded correctly. I think it's worthwhile to make the app loading happen in a more generic representation, like an App class, just like suggested in InstalledAppsRevision. Such an App could very well include a verbose_name (and other meta data) that would be found by Django and 3rd party apps.

Admittedly I was wrong about app()'s usefulness only in admin context. path, name and db_prefix parameters listed in InstalledAppsRevision should be generally available. style and weight (order is perhaps a better name) are evidently admin-specific. description lies somewhere in the middle -- it may be useful in other contexts besides admin.

Looks like both app() and something in the lines of AppAdmin are necessesary. Once the presentation layer has been decoupled with AppAdmin, other clever ways for building even more flexible custom admin interfaces utilizing it are free to emerge. Coupling presentation to app() on the other hand is a dead end. So I'm -1 to my own proposal to extend app() with additional keywords, but still +1 for a flexible mechanism for app presentation in the admin index page.

It should go far beyond what's outlined in InstalledAppsRevision. There's a lot to gain, e.g. no more magic in models autodiscovery, app dependencies etc. Apps should be packaged as eggs and the setuptools config structure setuptools.setup should be extended in Django-specific way. I'll eventually write a spec draft and bring it up in django-developers after 1.0 is out.

This latest patch (against r8965) passes all tests (tested with PostgreSQL backend). Please try it out and post your feedback on the mailing lists! Hopefully now that Django 1.0 is out (yay!), the core devs will have more time to review this patch and give some feedback on what's lacking...

Updated Vinay's patch so that the admin views expose the app's verbose_name as app_label in the templates; also fixed a bug in django/contrib/auth/management/__init__.py that prevents create_superuser to be called in syncdb.

Applies cleanly to r9498, the latest trunk revision before the the 1.0.2 release. Hope some core-dev takes a look at it and gives feedback before 1.1.

I don't know what is the current suggested syntax for this, but either way it should be defined inside the app, and not during installation of app like in the example of description. E.g. if we allow the app_label be variable that can be defined in INSTALLED_APPS, what happens to all permission checks? Someone installs the auth app as myauth and suddenly all permission checks stops working.

Probably it should be in the __init__.py of app, with something like:

APP_LABEL='myauth'

This way all application developers should define APP_LABEL in the __init__.py and they could use it like namespace, with values like ciantic_myauth. It would also allow backwards compatible way to do this.

Does it make sense to allow the overriding of app_label? It seems app_label should only be used internally by django to differentiate between apps. I'm not sure where the need is for overriding app_label in the first place. Wouldn't something like this be enough?

INSTALLED_APPS =('some.foo.app',
app('another.app', name='My App'),)

That way models can be grouped together by name in the admin. One reason I have a need for this is we have a blog app called "zinnia" installed, which has models called "Entries". It uses the django-comments app, which puts all of its models under "comments". Ideally, I should have in the admin an app area called "Blog", with models for "Entries" and "Comments", even though those models live in different apps.

Okay I think I found something from the branch. There will be App class for each app. This is a good thing, the verbose_name and app_label will be defined inside the app, which is the right thing to do. See the ​apps.py in GSoC 2010 branch which allowes one to define verbose_name etc. Basically like defining new App class for each app in __init__.py or anywhere for that matter. After which the INSTALLED_APPS is just list of App classes.

Btw it would be helpful if there were a README or something considering the branch, so one would know what and how the branch proposes to implement the feautre. Now I just "luckily" guessed the place for this apps.py.

@bendavis78: I am pretty sure your help would be greatly appreciated - considering the amount of attention/CC's for this ticket.

Maybe get on IRC and discuss with one or two of the core devs as to how and whether your work might be merged into trunk as soon as possible - and as to what the bottlenecks and design goals are for this feature. -- ​irc://irc.freenode.net/django

Other properties that can be set on Meta are db_prefix and models_path. This will allow you to override existing app definitions in order to prevent potential conflicts.

When the app loader loops through INSTALLED_APPS, it converts any normal modules an App class and loads everything into the apps cache (which was moved from django.models.loading to django.apps.cache). This cache is, among other things, used by the admin for introspection of apps and models.

For further customization, one may also specify how an App class is initialized by providing kwargs in the INSTALLED_APPS setting:

My only question at this point is regarding the design decision to use an options class (ie, MyApp.Meta). The code suggests that other properties may be set on your custom App class, but how might one use custom properties on an App class? I'm sure there's a reason for not putting verbose_name, db_prefix, and models_path in the class's main properties, I'm just curious what the thinking is on that.

On another note, I was thinking it would also be useful to allow app authors to define how apps should be displayed to the end user. Specifically, I think it would be nice to be able to group and order models in a way that would be more meaningful or helpful to admin users. Perhaps that's one thing that could be done on within a custom App class. Although, that might be a discussion for another time.

My only question at this point is regarding the design decision to use an options class (ie, MyApp.Meta). The code suggests that other properties may be set on your custom App class, but how might one use custom properties on an App class? I'm sure there's a reason for not putting verbose_name, db_prefix, and models_path in the class's main properties, I'm just curious what the thinking is on that.

It's simple, Meta is used internally by Django during startup, so it needs to be handled with care, reducing the chance to break a rather fundamental part. In other words, it's internal API when it comes to app loading and is configured depending on the (Python) meta class AppBase. Using it in app-specific implementation details like looking for a specific module in all apps (say for implementing the discovery pattern) is fine though. If you want to override/set a specific option of an app class we need to differ between those internal (via _meta) and the "usual" class or instance attributes.

On another note, I was thinking it would also be useful to allow app authors to define how apps should be displayed to the end user. Specifically, I think it would be nice to be able to group and order models in a way that would be more meaningful or helpful to admin users. Perhaps that's one thing that could be done on within a custom App class. Although, that might be a discussion for another time.

That's certainly a good idea but not really part of this ticket. In other words, this can be added at a later time on top of the new app classes.

It looks like my assumption that an App subclass can implement its own attributes is incorrect with the current patch. Currently the python metaclass AppBase does not propagate the class's attrs to the new_class It only propagates {'__module__': module}:

No, you misunderstood me, the class attributes of the app classes shouldn't be put into _meta since they are not part of the internal API. This follows the same pattern as the ModelForm and Model classes .

Any parameter which is passed during initialization to the app class (e.g. from the settings) ought to be set as an instance attribute, unless it's one of the few internal options. It's a simple matter of easy customization to be able to do that.

Sure, I understand that they shouldn't be passed into _meta. But one would expect any custom attributes on their app class to stay there, and not magically disappear, correct? For example, if I create this:

classMyApp(App):
foo ="bar"

the foo attribute of the class will magially disappear. This happens because in the python metaclass, new_class is not being given the attributes of the original class.

I noticed that in the admin, app titles are not run through .title(), as they were before the patch.

Actually, the application label is, in current admin code, inconsistently run through either the capfirst template filter (or its underlying function in view Python code) or tranformed by the .title() method.

This would be a good chance to get that fixed and IMHO allowing the app author to define the capitalization of the label would be the best approach (now the label can be something more complex that the app module/package name e.g. "TLD Registries Contacts") instead of forcing such a basic treatment like .title() to be always applied (same thing for translators of that label to other languages).

2b9fb2e6443c04e4415b17083d727bd80047b6e5 fixed a deadlock in app loading. It does not contain tests as it seemed too hard to write one into Django's test suite. So, a heads up: the app-loading refactor should take care not to reintroduce the deadlock condition again. Refs #18251.

Wanted to let you know I've started a discussion on the django developers group that is related to this ticket. I would appreciate your opinions regarding the topic and advice on how to make it go forward.