Q on naming nested packages/modules

I'm having a hard time getting the hang of Python's package/module
scheme. I'd like to find out what's considered best practice when
dealing with the scenario illustrated below.

The quick description of the problem is: how can I have two nested
modules, spam.ham and spam.ham.eggs?

Suppose I have a module (I'm not even sure this is the right word)
called spam.ham, so I start out with the following file structure:

spam/
|-- ham.py
`-- __init__.py

With this arrangement, the line

import spam.ham

....in client code works as expected.

But now suppose that I want to factor out some code in spam/ham.py
to a helper module. (The reason behind factoring out this new
module is to "declutter" spam/ham.py, and improve its readibility.)
My instinct (from my Perl past) is to put this factored-out code
in a file spam/ham/eggs.py, i.e. to create the "nested" module
spam.ham.eggs, which requires expanding the tree as follows

This doesn't work so well, because now spam/ham.py is not read.
It seems that adding the spam/ham directory, or maybe adding the
file spam/ham/__init__.py, causes spam/ham.py to be overlooked.

Clearly, I'm not playing this game right...

What is considered "best practice" for the use case sketched above?
Should I, e.g. rename the directory spam/ham something like spam/ham_
and refer to the helper module as spam.ham_.eggs? Or is some other
convention preferred?

I consulted PEP 8, but besides recommending "short, all-lowercase
names" for modules, it gives little guidance on the situation
described above.

Advertisements

On Tue, Sep 1, 2009 at 11:58 AM, kj<> wrote:
>
>
>
> I'm having a hard time getting the hang of Python's package/module
> scheme. I'd like to find out what's considered best practice when
> dealing with the scenario illustrated below.
>
> The quick description of the problem is: how can I have two nested
> modules, spam.ham and spam.ham.eggs?
>
> Suppose I have a module (I'm not even sure this is the right word)
> called spam.ham, so I start out with the following file structure:
>
> spam/
> |-- ham.py
> `-- __init__.py
>
> With this arrangement, the line
>
> import spam.ham
>
> ...in client code works as expected.
>
> But now suppose that I want to factor out some code in spam/ham.py
> to a helper module. (The reason behind factoring out this new
> module is to "declutter" spam/ham.py, and improve its readibility.)
> My instinct (from my Perl past) is to put this factored-out code
> in a file spam/ham/eggs.py, i.e. to create the "nested" module
> spam.ham.eggs, which requires expanding the tree as follows
>
> spam/
> |-- ham/
> | |-- eggs.py
> | `-- __init__.py
> |-- ham.py
> `-- __init__.py
>
> ...and adding the following spam/ham.py to
>
> # spam/ham.py
> from . import eggs
>
> This doesn't work so well, because now spam/ham.py is not read.
> It seems that adding the spam/ham directory, or maybe adding the
> file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
>
>
> Clearly, I'm not playing this game right...
>
> What is considered "best practice" for the use case sketched above?
> Should I, e.g. rename the directory spam/ham something like spam/ham_
> and refer to the helper module as spam.ham_.eggs? Or is some other
> convention preferred?
>
> I consulted PEP 8, but besides recommending "short, all-lowercase
> names" for modules, it gives little guidance on the situation
> described above.
>

Take everything in ham.py and stick it in ham/__init__.py instead.
This will give you the behavior you're looking for (don't import
spam.ham.__init__. Everything in __init__.py is loaded into spam.ham)
> TIA!
>
> kynn
> --
> http://mail.python.org/mailman/listinfo/python-list
>

Advertisements

In <h7jga8$ijj$> kj <> writes:
>I'm having a hard time getting the hang of Python's package/module
>scheme. I'd like to find out what's considered best practice when
>dealing with the scenario illustrated below.
>The quick description of the problem is: how can I have two nested
>modules, spam.ham and spam.ham.eggs?

Following up my own post...

From inspecting the directory structure of some of the standard
Python modules I infer the following rules:

1. the source for "leaf" modules lives in files named after them
(e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)

2. the source for "non-leaf" modules lives in files named __init__.py
(e.g. if x.y is a "non-leaf" module, its source code lives in
the file x/y/__init__.py)

In the examples above, the module x.y is a "non-leaf" module because
there is a module x.y.z.

I.e. the "leaf"-ness of a module depends solely on whether other
modules deeper in the hierarchy are present.

An implication of all this is that if now I wanted to create a new
module x.y.z.w, this means that the previously "leaf"-module x.y.z
would become "non-leaf". In other words, I'd have to:

1. create the new directory x/y/z
2. *rename* the file x/y/z.py to x/y/z/__init__.py
3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
module

Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
is the one that causes most distress... But I think I can cope.)

On Sep 1, 8:58 am, kj <> wrote:
> I'm having a hard time getting the hang of Python's package/module
> scheme. I'd like to find out what's considered best practice when
> dealing with the scenario illustrated below.
>
> The quick description of the problem is: how can I have two nested
> modules, spam.ham and spam.ham.eggs?
>
> Suppose I have a module (I'm not even sure this is the right word)
> called spam.ham, so I start out with the following file structure:
>
> spam/
> |-- ham.py
> `-- __init__.py
>
> With this arrangement, the line
>
> import spam.ham
>
> ...in client code works as expected.
>
> But now suppose that I want to factor out some code in spam/ham.py
> to a helper module. (The reason behind factoring out this new
> module is to "declutter" spam/ham.py, and improve its readibility.)
> My instinct (from my Perl past) is to put this factored-out code
> in a file spam/ham/eggs.py, i.e. to create the "nested" module
> spam.ham.eggs, which requires expanding the tree as follows
>
> spam/
> |-- ham/
> | |-- eggs.py
> | `-- __init__.py
> |-- ham.py
> `-- __init__.py
>
> ...and adding the following spam/ham.py to
>
> # spam/ham.py
> from . import eggs
>
> This doesn't work so well, because now spam/ham.py is not read.
> It seems that adding the spam/ham directory, or maybe adding the
> file spam/ham/__init__.py, causes spam/ham.py to be overlooked.

There's a hint here.

You can move the contents of ham.py into ham/__init__.py, and it'll
work the way you want, maybe with only a minor change or two.

> Clearly, I'm not playing this game right...
>
> What is considered "best practice" for the use case sketched above?
> Should I, e.g. rename the directory spam/ham something like spam/ham_
> and refer to the helper module as spam.ham_.eggs? Or is some other
> convention preferred?

The way you were intending is often the approach people use. I doubt
there are any detailed consensus recommendations about this in the
Python community.

kj wrote:
> In <h7jga8$ijj$> kj <> writes:
>
>
>>I'm having a hard time getting the hang of Python's package/module
>>scheme. I'd like to find out what's considered best practice when
>>dealing with the scenario illustrated below.
>
>
>>The quick description of the problem is: how can I have two nested
>>modules, spam.ham and spam.ham.eggs?
>
>
> Following up my own post...
>
>>From inspecting the directory structure of some of the standard
> Python modules I infer the following rules:
>
> 1. the source for "leaf" modules lives in files named after them
> (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
>
> 2. the source for "non-leaf" modules lives in files named __init__.py
> (e.g. if x.y is a "non-leaf" module, its source code lives in
> the file x/y/__init__.py)
>
> In the examples above, the module x.y is a "non-leaf" module because
> there is a module x.y.z.
>
> I.e. the "leaf"-ness of a module depends solely on whether other
> modules deeper in the hierarchy are present.
>
> An implication of all this is that if now I wanted to create a new
> module x.y.z.w, this means that the previously "leaf"-module x.y.z
> would become "non-leaf". In other words, I'd have to:
>
> 1. create the new directory x/y/z
> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
> module
>
> Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
> is the one that causes most distress... But I think I can cope.)
>
> kynn

Looking at the layout of the (most excellent!-) xlrd package, the bulk
of the code is in the __init__.py file, and other supporting code is the
same directory, accessed in __init__ with normal imports.

I also am unclear on when it's best to have supporting files in the same
directory versus a subdirectory... perhaps it is that flat is better
than nested?

kj wrote:
>
>
> But now suppose that I want to factor out some code in spam/ham.py
> to a helper module. (The reason behind factoring out this new
> module is to "declutter" spam/ham.py, and improve its readibility.)
> My instinct (from my Perl past) is to put this factored-out code
> in a file spam/ham/eggs.py, i.e. to create the "nested" module
> spam.ham.eggs, which requires expanding the tree as follows
>
> spam/
> |-- ham/
> | |-- eggs.py
> | `-- __init__.py
> |-- ham.py
> `-- __init__.py
>
> ...and adding the following spam/ham.py to
>
> # spam/ham.py
> from . import eggs
>
> This doesn't work so well, because now spam/ham.py is not read.
> It seems that adding the spam/ham directory, or maybe adding the
> file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
>
>
> Clearly, I'm not playing this game right...

One way is the __init__.py answer you have already. Another is to put
eggs.py (or _eggs.py to indicate that it is private) in spam and skip
the ham directory.

> An implication of all this is that if now I wanted to create a new
> module x.y.z.w, this means that the previously "leaf"-module x.y.z
> would become "non-leaf". In other words, I'd have to:
>
> 1. create the new directory x/y/z
> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
> module

With regard to point 2 -- would it be possible to just move z.py into
x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same
effect? Or is that not a good idea?

On Tue, 01 Sep 2009 09:53:08 -0700, kj <> wrote:
> In <h7jga8$ijj$> kj <> writes:
>
>> I'm having a hard time getting the hang of Python's package/module
>> scheme. I'd like to find out what's considered best practice when
>> dealing with the scenario illustrated below.
>
>> The quick description of the problem is: how can I have two nested
>> modules, spam.ham and spam.ham.eggs?
>
> Following up my own post...
>
>> From inspecting the directory structure of some of the standard
> Python modules I infer the following rules:
>
> 1. the source for "leaf" modules lives in files named after them
> (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
>
> 2. the source for "non-leaf" modules lives in files named __init__.py
> (e.g. if x.y is a "non-leaf" module, its source code lives in
> the file x/y/__init__.py)
>
> In the examples above, the module x.y is a "non-leaf" module because
> there is a module x.y.z.
>
> I.e. the "leaf"-ness of a module depends solely on whether other
> modules deeper in the hierarchy are present.
>
> An implication of all this is that if now I wanted to create a new
> module x.y.z.w, this means that the previously "leaf"-module x.y.z
> would become "non-leaf". In other words, I'd have to:
>
> 1. create the new directory x/y/z
> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
> module
>
> Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
> is the one that causes most distress... But I think I can cope.)
>
> kynn

In <> "Rami Chowdhury" <> writes:
>> An implication of all this is that if now I wanted to create a new
>> module x.y.z.w, this means that the previously "leaf"-module x.y.z
>> would become "non-leaf". In other words, I'd have to:
>>
>> 1. create the new directory x/y/z
>> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
>> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
>> module
>With regard to point 2 -- would it be possible to just move z.py into
>x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same
>effect? Or is that not a good idea?

I think in this case what you would need is something like 'from
...z import *', but this does not work either; one gets the error
"SyntaxError: 'import *' not allowed with 'from .'".

A third solution would be to change step 2 above to this:

2. create the *symlink* x/y/z/__init__.py --> ../z.py

This certainly would make things clearer-looking to me. Having
all these files called the same thing, __init__.py, but having
completely different meaning, makes my head hurt... In contrast,
the __init__.py *symlinks* would be there just to keep python happy;
the real content is where I'd expect it.

I ran a simple test of this idea and it works, but there may be
some subtle reasons for not doing it... Or maybe it's not a good
idea simply because it is not the Python way, and I'd agree that
this would be reason enough to avoid it.

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!