Strange problem when using imp.load_module

Discussion in 'Python' started by pythoncurious@gmail.com, Apr 23, 2009.

Guest

Hi,

I'm having problem when I'm trying to import modules using the
imp.load_module function.
At the end of this post there's some code I wrote to illustrate the
problem.
The code istself doesn't make much sense, but what I'm trying to do in
reality is allow people to customize an application by writing a small
python module. If I find a file in a known location with a known name,
I'll import it and use some data or function in it.
I ran into problems when writing unit tests for it.

So it seems that test_2 fails. It does pretty much the same thing as
test_1, but with different data.
Changing test_2 to check for 'x' instead of 'y' makes it pass, but
that's not the result I should get.
So it seems that I still have the module that I imported in test_1.

So, I'm thinking that I might be dealing with some sort of gc issue.
I add the "time.sleep(1)", (commented in the code) and the tests pass.
'y' is found in test_2.
Replacing the sleep with a gc.collect() makes the test fail again, so
garbage collection doesn't seem to help.

I've tried it on python 2.6.1 on windows, 2.5.2 in cygwin and 2.6.1 on
solaris with the same results.
Could anyone explain what I am missing?

Advertisements

Hi, I have a guess at explaining the behaviour you describe - see below.
> I'm having problem when I'm trying to import modules using the
> imp.load_module function.
> At the end of this post there's some code I wrote to illustrate the
> problem.
> The code istself doesn't make much sense, but what I'm trying to do in
> reality is allow people to customize an application by writing a small
> python module. If I find a file in a known location with a known name,
> I'll import it and use some data or function in it.
> I ran into problems when writing unit tests for it.
>
> What happens when run my code is this:
>
> .E
> ======================================================================
> ERROR: test_2 (__main__.Test)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "test_import.py", line 30, in test_2
> getattr(imported, 'y')
> AttributeError: 'module' object has no attribute 'y'
> ----------------------------------------------------------------------
> Ran 2 tests in 0.015s
>
>
> So it seems that test_2 fails. It does pretty much the same thing as
> test_1, but with different data.
> Changing test_2 to check for 'x' instead of 'y' makes it pass, but
> that's not the result I should get.
> So it seems that I still have the module that I imported in test_1.
>
> So, I'm thinking that I might be dealing with some sort of gc issue.
> I add the "time.sleep(1)", (commented in the code) and the tests pass.
> 'y' is found in test_2.
> Replacing the sleep with a gc.collect() makes the test fail again, so
> garbage collection doesn't seem to help.
>
> I've tried it on python 2.6.1 on windows, 2.5.2 in cygwin and 2.6.1 on
> solaris with the same results.
> Could anyone explain what I am missing?
>
> Thanks
> /Mattias
>
> Here's the code:
>
> import unittest
> import tempfile, os, imp, time, gc
>
> module_file_name=os.path.join(tempfile.gettempdir(), 'my_module.py')
>
> def write_module(data):
> f = open(module_file_name, 'w')
> f.write(data)
> f.close()
>
> def import_from_file(path):
> imported_module = imp.load_source(module_file_name, path)
> return imported_module
>
> class Test(unittest.TestCase):
> def tearDown(self):
> os.unlink(module_file_name)
>
> def test_1(self):
> module_data='''x=1'''
> write_module(module_data)
> imported=import_from_file(module_file_name)

My guess is that without the sleep(1), the imp.load_source function will
use the compiled file 'my_module.pyc' created in test_1 instead of
compiling the new 'my_module.py' file.

This would be because the new 'my_module.py' file is created so soon
after the last compilation of the module that they have the same
timestamp, thus fooling imp.load_source into thinking that
'my_module.py' was not modified since its last compilation.
> getattr(imported, 'y')
>
> if __name__ == "__main__":
> unittest.main()

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!