The cdef Widget field keeps the PyWidget alive until after PySprocket.__dealloc__ destroys the Sprocket. However, as soon as the Python garbage collected gets involved, the tp_clear function Cython constructs for PySprocket messes this up:

Since there's a reference cycle, the garbage collector invokes the tp_clear to try to break the cycle. Cython's tp_clear drops all references to Python objects. Only after this happens does PySprocket.__dealloc__ get to run.

Cython documentation warns about __dealloc__ (although it took me a while to learn what conditions it was talking about, since it doesn't go into any detail). So perhaps this approach is entirely invalid.

Can Cython support this use case?

As (what I hope is) a temporary work-around, I've moved to an approach that looks something like this:

string.h and stdio.h are not valid C++ headers; the C++ equivalents are <cstring> and <cstdio>, and you really, really shouldn't be using them anyway. Why would you ever consider denying yourself the power of std::string, especially when performance is obviously not a concern (given that you're doing half the work in Python anyway)?
–
Karl KnechtelDec 21 '10 at 18:18

2

@Karl: they are valid in C++: D.5 of the standard.
–
Steve JessopDec 21 '10 at 19:20

The name field was a way to demonstrate there's a problem. I was actually going for a segfault, but only managed to get some garbage data (on my system, at least). I think the char* could be replaced with a std::string without substantively affecting the question. Thanks for the reminder about header names, though.
–
Jean-Paul CalderoneDec 21 '10 at 21:17

I did not test it, but I have used both Python and C++ a lot, and checked the Cython documentation.
–
ch0keeDec 30 '10 at 5:18

1

Hi, thanks for the suggestion. I tweaked your Cython a little to get it to compile (needed a forward PySprocket declaration, some self parameters were forgotten, and del spr isn't legal) - wish I could share the exact code but it won't fit in this comment box - and re-tried my simple test from the question, with this result: Exception AttributeError: "'NoneType' object has no attribute 'remove'" in <proposed.PySprocket object at 0xb756810c> ignored. There is a similar error for 'copy' as well.
–
Jean-Paul CalderoneJan 2 '11 at 0:03

1

self.sprockets is an object of python set type, it is supported in cython. What self parameters were forgotten ? just mention 1-2 so I can edit my post, I didn't find them. Both of your codes run as expected, you see ? In your second code you inserted a del widget before del sprocket, that makes the difference.
–
ch0keeJan 2 '11 at 8:43

1

The PyWidget attach and detach methods are missing the self parameter. A step-by-step explanation would be great; I think I already understand what's happening, but perhaps I overlooked something. As regards the "del widget" preceding the "del sprocket" - the objects are in a cycle, so the order of the del should not matter.
–
Jean-Paul CalderoneJan 2 '11 at 16:08

1

Unfortunately this still doesn't work. The problem is that in PySprocket.__dealloc__, self.widget is not guaranteed to be valid and in PyWidget.__dealloc__, neither is self.sprockets. Cython's tp_clear may have already set both attributes to None. This will only happen when there is a reference cycle, since Python only calls tp_clear in that case.
–
Jean-Paul CalderoneJan 10 '11 at 14:57