Search Discussions

17 responses

Tim Peters[Lenard Lindstrom] Nothing implemented in C is subclassable unless somebody volunteers the work to make it subclassable; nobody volunteered the work to make the function type subclassable. It sure wasn't at the top of my list .

Nothing implemented in C is subclassable unless somebody volunteers the workto make it subclassable; nobody volunteered the work to make the functiontype subclassable. It sure wasn't at the top of my list <wink>.

* To provide methods on the object allowing OO-style API's forintrospection?o retrieving arguments and/or variable names, defaults,presence of * or ** argument-setso retrieving text of the function from source fileso retrieving formatted "parameter" help text, or more detailedrun-time help-texto retrieving "raised exception" information and/orcaught-exception information* Experimenting with meta-programming mechanismso automatically generating pre and/or post-condition checks+ argument and/or return value types/interfaces/values+ system-status checks (there is an open databaseconnection, there is a TCP/IP stack, or whatever)o providing a "restart" API for extremely long-runningfunctions which have encountered exceptionso providing enhanced "cleanup" functionality for an entireclass of functions which a framework may call if thefunction fails* To provide, for instance, a "script" function object defined insuch a way that the function object stores the source code,provides pickling support, and is generally a first-class contentobject, but is a simple function object as far as the system isconcerned (with no overhead for its call method, standard supportin IDE's and documentation systems etceteras)

You can certainly do that stuff using wrapping, and it's certainlyeasier to do it that way today (given that the subclassing isimpossible). The question then becomes, is it appropriate to allow thesub-classing approach?

Recent changes in Python (i.e. 2.2+) have made the type-class splitconsiderably less visible, with a trend toward using sub-classing ofcore objects as an acceptable and useful idiom. Consistency and theprinciple of least surprise would suggest that having all of theprimitive types sub-classable, where it doesn't negatively impactperformance, is a good thing. (It's also nice to allow systems such asthe generic introspection systems to deal with the objects withoutneeding explicit modification to support them (i.e., your functionsstill show up as functions in your IDE, you don't have them showing upas data attributes)).

Which comes back to Tim's post, suggesting that a volunteer creating asub-classable version of the function object would be sufficient tointroduce such a thing to the core.

Martin v. LÃ¶wisThat won't work. If you create a function through the def statement, it will be of type function, not of a subclass thereof. So on all function objects, your introspection methods won't be available. Same counter-argument: How do you create such specialized function objects? Not unless accompanied with some other change to conveniently create specialized function objects. One proposal (originally proposed to create class methods) might work: def foo(args)[enhanced_cleanup]: code This would

This would create a function object first, then invoke thecallable enhanced_cleanup, which returns a modified (orwrapped) object. However, if this extension were available,specializing the function type is still not necessary, sinceyou could just as well create wrapper objects in allusage scenarios you have given.

Which comes back to Tim's post, suggesting that a volunteer creating asub-classable version of the function object would be sufficient tointroduce such a thing to the core.

I suspect any volunteer would notice quickly that this is not sufficient.

Mike C. FletcherIt would certainly be required to have a base-class constructor. You'll note, however, that this wasn't really an insurmountable problem for the type meta-class ;) . Like that change, you need the system to support the operation (i.e. call it) if you want syntactic support, but it's not an impossible change. BTW Even today, without support built into the system to call the function-constructor when reading a source-file, you could still use it within systems where run-time definition of classes

That won't work. If you create a function through the def statement,it will be of type function, not of a subclass thereof. So on all functionobjects, your introspection methods won't be available.

It would certainly be required to have a base-class constructor. You'llnote, however, that this wasn't really an insurmountable problem for thetype meta-class ;) . Like that change, you need the system to supportthe operation (i.e. call it) if you want syntactic support, but it's notan impossible change.

BTW Even today, without support built into the system to call thefunction-constructor when reading a source-file, you could still use itwithin systems where run-time definition of classes is the norm, such asembedded scripting systems (you call the function-type's constructorwhenever you want to create a new function).

...

The question then becomes, is it appropriate to allow thesub-classing approach?

Sure, there's lots of problems with it (about equal to those with"property" IMO), but the question of the *hook* to invoke the mechanismis somewhat distinct from the mechanism itself, no? After all, we hadthe "new" module for years before the functionality eventually migratedto the base-type constructors.

This would create a function object first, then invoke thecallable enhanced_cleanup, which returns a modified (orwrapped) object. However, if this extension were available,specializing the function type is still not necessary, sinceyou could just as well create wrapper objects in allusage scenarios you have given.

See notes regarding wrapping versus inheriting in the original post. My"arguments" aren't necessarily compelling, but the trend does appear tobe toward sub-classable base-types.

Which comes back to Tim's post, suggesting that a volunteer creating asub-classable version of the function object would be sufficient tointroduce such a thing to the core.

I suspect any volunteer would notice quickly that this is not sufficient.

Regards,Martin

I'd guess it would depend on the volunteer's purposes, and whetherthey're trying to get the syntactic operation you want ;) working :) .I find the meta-type constructor's syntactic sugar gets in the way aboutas much as it helps (you wind up having to muck about in the class'namespace to define named arguments which you then need to decidewhether to include in the final class or not).

The simple call approach works. It lets you define arguments (how do youdo that with the def x()[a,b,c] syntactic construct, for instance), itis familiar and readily understood (no new syntax required, you canimmediately see that you're getting something back from aconstructor/function, there's no magic invocation going on). It's notconvenient for replacing every function/method in a file, but neitherwould the [] syntax.

Aside:

It might be neat to see def and class (maybe even if and else ;)(just kidding :) )) decommisioned as keywords and a syntacticconstruct created something like this:

callable name [ ( args, named ) ]:suite

creating pseudo-code like this:

name = callable( name, suite-code-string, argumentDescriptions )

with def and class being objects (a new function meta-type, and thetype meta-type (or rather, a sub-class or factory to generateappropriately new/old style classes by executing suite-code in adictionary then calling the current constructor)) which obey theprotocol.

Even cooler, you might rework the parser so that the suites arepassed to their constructors w/out going through the parser'ssyntactic check. You then have a hook for such things as Pyrex toembed their functions directly in the Python code. Of course,that's probably a little *too* flexible if we're going to maintainsome level of "Python-1.5-ness" in the core :) .

Greg Ewing (using news.cis.dfn.de)Maybe something like the __metaclass__ syntax is called for: def myfunction(args): __functionclass__ = myspecialfunc ... (I know that looks like a local variable assignment, but it wouldn't be. As Guido once said about something else, "It's a double-underscore name -- I can give it whatever semantics I want!") -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg

"Greg Ewing (using news.cis.dfn.de)" <me at privacy.net> wrote in messagenews:b5o1c0$2af4lj$1 at ID-169208.news.dfncis.de...

...Maybe something like the __metaclass__ syntax iscalled for:

def myfunction(args):__functionclass__ = myspecialfunc...

(I know that looks like a local variable assignment,but it wouldn't be. As Guido once said about something else,"It's a double-underscore name -- I can give it whateversemantics I want!")...

In a class definition '__metaclass__ = something' is an actual assignmentstatement. Also the class suite is executed at definition time, unlike afunction body suite, which is not. Perhaps __functionclass__ could beexternal to the definition, then:

Greg Ewing (using news.cis.dfn.de)Guido is on record as saying that's just a detail of the current implementation. But I've had a better idea: def f(arg, ..., __functionclass__ = myfunkyfunc): ... Now it's a real assignment (of a default argument value), and it's outside the function's local scope, and it's restricted to that function! Happy now? :-) -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg

Lenard Lindstrom"Greg Ewing (using news.cis.dfn.de)" wrote in message news:b5qnia$2863q6$1 at ID-169208.news.dfncis.de... assignment Actually I was happy with your first suggestion. However in a previous posting someone proposed having C like static variable declarations within a function body. Beni Cherniavsky said this would complicate things since all function bodies would then have to be checked for the special code ( messages 11 and 12 of 'PEP 309 - Built-in closure type (with tentative syntax proposal)'

"Greg Ewing (using news.cis.dfn.de)" <me at privacy.net> wrote in messagenews:b5qnia$2863q6$1 at ID-169208.news.dfncis.de...

Lenard Lindstrom wrote:

In a class definition '__metaclass__ = something' is an actual

assignment

statement.

Guido is on record as saying that's just a detail ofthe current implementation.

But I've had a better idea:

def f(arg, ..., __functionclass__ = myfunkyfunc):...

Now it's a real assignment (of a default argument value),and it's outside the function's local scope, and it'srestricted to that function!

Happy now? :-)

--...

Actually I was happy with your first suggestion. However in a previousposting someone proposed having C like static variable declarations within afunction body. Beni Cherniavsky said this would complicate things since allfunction bodies would then have to be checked for the special code (messages 11 and 12 of 'PEP 309 - Built-in closure type (with tentativesyntax proposal)' ). It seems to me __functionclass__ in the function bodywould be the same way. Having __functionclass__ in the argumet list is finewith me as well. And it doesn't require a syntax change.

Lenard Lindstrom"Lenard Lindstrom" wrote I forgot about doc strings. I guess I finally got used to them. :-) They get stripped from the function body. So putting '__functionclass__ =' right after a doc string should not cause too much difficulty. And why not permit a module level __functionclass__, like with __metaclass__, to keep everything consistant? Lenard Lindstrom "<%s@%s.%s>" % ("len-l.", "telus", "net")

...Actually I was happy with your first suggestion. However in a previousposting someone proposed having C like static variable declarations within afunction body. Beni Cherniavsky said this would complicate things since allfunction bodies would then have to be checked for the special code (messages 11 and 12 of 'PEP 309 - Built-in closure type (with tentativesyntax proposal)' ). It seems to me __functionclass__ in the function bodywould be the same way. Having __functionclass__ in the argumet list is finewith me as well. And it doesn't require a syntax change....

I forgot about doc strings. I guess I finally got used to them. :-) Theyget stripped from the function body. So putting '__functionclass__ =' rightafter a doc string should not cause too much difficulty. And why not permita module level __functionclass__, like with __metaclass__, to keepeverything consistant?

Lenard Lindstrom"Erik Max Francis" wrote in message news:3E7E81EF.1992BFD4 at alcyone.com... I was just curious. While playing around with function currying I tried to subtype function to allow operator overloading. Given PEP 253 I wondered if the language development team had run into some complications with type function. As for uses, should not staticmethod and classmethod be subtypes of function? If one wants a function wrapper that looks like a function, e.g. has func_doc, func_code, and func_defaults

That would probably be best done by asking a counter question: Whywould you want to?...

I was just curious. While playing around with function currying I tried tosubtype function to allow operator overloading. Given PEP 253 I wondered ifthe language development team had run into some complications with typefunction. As for uses, should not staticmethod and classmethod be subtypesof function? If one wants a function wrapper that looks like a function,e.g. has func_doc, func_code, and func_defaults members, should not thatwrapper class be derived from function? Maybe the wrapper is a proxy forremote function calls. This is just conjecture. I am not yet familiar enoughwith Python to know if it makes practical sense.

Greg Ewing (using news.cis.dfn.de)Staticmethods and classmethods don't have all those attributes, so they shouldn't be subclasses of function. Yes, but it will only have those attributes if it's a subtype of "function consisting of interpreted Python bytecode", not "callable object in general". The use cases of *that* are somewhat more restricted. That's not to say they don't exist, and I agree that there's no reason for function objects *not* to be subclassable. It's just that nobody's had a pressing enough need to subclass

Lenard Lindstrom wrote:As for uses, should not staticmethod and classmethod be subtypesof function? If one wants a function wrapper that looks like a function,e.g. has func_doc, func_code, and func_defaults members,

Staticmethods and classmethods don't have all those attributes,so they shouldn't be subclasses of function.

should not that wrapper class be derived from function?

Yes, but it will only have those attributes if it's asubtype of "function consisting of interpreted Pythonbytecode", not "callable object in general". The usecases of *that* are somewhat more restricted.

That's not to say they don't exist, and I agree thatthere's no reason for function objects *not* to besubclassable. It's just that nobody's had a pressingenough need to subclass one yet to make it so.

Lenard Lindstrom"Greg Ewing wrote Bad choices. But certainly instancemethod could stand to look more function-like? You cannot 'inspect' it directly. You must inspect meth.im_func. So inspect.getargspec(meth) would probably show parameter 'self'. This is such a defacto keyword that its usage is obvious. There are ways to hide it in func_code.co_varnames, but would require a certain amount of overhead. I am not neccessarily suggesting function subtyping. That was just an observation. What I see is an

...Staticmethods and classmethods don't have all those attributes,so they shouldn't be subclasses of function....

Bad choices. But certainly instancemethod could stand to look morefunction-like? You cannot 'inspect' it directly. You must inspectmeth.im_func. So inspect.getargspec(meth) would probably show parameter'self'. This is such a defacto keyword that its usage is obvious. There areways to hide it in func_code.co_varnames, but would require a certain amountof overhead.

...That's not to say they don't exist, and I agree thatthere's no reason for function objects *not* to besubclassable. It's just that nobody's had a pressingenough need to subclass one yet to make it so.

I am not neccessarily suggesting function subtyping. That was just anobservation. What I see is an inconsistancy between real functions and allother non-builin callables. Having some way to get 'inspect' to work withnon-function callables would probably be enough. So maybe only inspect needschanging.

Lenard Lindstrom"Tim Peters" wrote in message news:mailman.1048478018.32024.python-list at python.org... Okay. I think I see. I notice that functions are not provided for the tp_new, tp_init, tp_alloc, and tp_free slots of PyFunction_Type. Just adding Py_TPFLAGS_BASETYPE to tp_flags will not enable subclassing for type function. More work than I first thought. Given PEP 253 I assumed most classes would already be converted, leaving just thoughs which would break something if subtyping were permitted on them.

Nothing implemented in C is subclassable unless somebody volunteers the workto make it subclassable; nobody volunteered the work to make the functiontype subclassable. It sure wasn't at the top of my list <wink>.

Okay. I think I see. I notice that functions are not provided for thetp_new, tp_init, tp_alloc, and tp_free slots of PyFunction_Type. Justadding Py_TPFLAGS_BASETYPE to tp_flags will not enable subclassing for typefunction. More work than I first thought. Given PEP 253 I assumed mostclasses would already be converted, leaving just thoughs which would breaksomething if subtyping were permitted on them. Instead function was simplyneglected due to time constaints. ;-)

Tim Peters[Tim] Yup, it always is. See the other replies in this thread too (e.g., if it were subclassable, how would you create an instance? the natural "def" notation doesn't have a way to specify a base class). Or things that were low on the perceived bang-for-the-buck scale. I made the file type subclassable very late in the release cycle, and that may well have been the last builtin type to become subclassable for 2.2. Being able to subclass file was worth something to me. For 2.3, someone else

Nothing implemented in C is subclassable unless somebody volunteers thework to make it subclassable; nobody volunteered the work to makethe function type subclassable. It sure wasn't at the top of mylist <wink>.[Lenard Lindstrom]Okay. I think I see. I notice that functions are not provided for thetp_new, tp_init, tp_alloc, and tp_free slots of PyFunction_Type. Justadding Py_TPFLAGS_BASETYPE to tp_flags will not enablesubclassing for type function. More work than I first thought.

Yup, it always is. See the other replies in this thread too (e.g., if itwere subclassable, how would you create an instance? the natural "def"notation doesn't have a way to specify a base class).

Given PEP 253 I assumed most classes would already be converted,leaving just thoughs which would break something if subtyping werepermitted on them.

Or things that were low on the perceived bang-for-the-buck scale. I madethe file type subclassable very late in the release cycle, and that may wellhave been the last builtin type to become subclassable for 2.2. Being ableto subclass file was worth something to me. For 2.3, someone else (IIRC)made the array type (array.array) subclassable. The universe of volunteersfor this stuff is small.

Instead function was simply neglected due to time constaints. ;-)

That's so even without the smiley. There's never enough time to do all thatwe'd like to do. BTW, that's why Python is more usable than Scheme <0.8wink>.

The example _wrap method is rudimentary. A truly powerfully wrapper wouldreproduce the arguments of the function being wrapped. But no, permittingsubtyping of function is not absolutely necessary. The above example workswith a function standin class. So I suppose all I am really suggesting is anabstract function class, a contract saying a callable object will stand upto introspection. Then the inspect module functions can check for this aswell as type function.

Python 2.3a2 provides a tp_new slot method for function. I used type tupleas a model for filling in the rest. Preliminary profiling shows a call to afunction subtype is 13 percent slower that a function call, but 16 percentfaster that calling an object with a static __call__ method.

Most of the above session can be done without subtyping function. Onlyisinstance() and inspect.getargspec() would fail. If anything comes fromthese postings it should be a more general purpose form ofinspect.getargspec() which handles all callable objects.

Thanks to everyone who answered my questions and participated in thediscussion that followed.