Greg Stein wrote: > > +1 on breaking it now, rather than deferring it Yet Again. > > IMO, there has been plenty of warning, and there is plenty of time to > correct the software. > > I'm +0 on adding a warning architecture to Python to support issuing a > warning/error when .append is called with multiple arguments.

Well, the (bad) effect of this patch is that you cannot run PythonWin any longer unless Mark either supplies an updated distribution, or one corrects the two barfing Scintilla support scripts by hand. Bad for me, since I'm building Stackless Python against 1.5.2+, and that means the users will see PythonWin barf when installing SLP.

Adding a warning instead of raising an exception would be nice IMHO, since the warning could probably contain the file name and line number to change, and I would leave my users with this easy task.

Christian> Well, the (bad) effect of this patch is that you cannot run Christian> PythonWin any longer unless Mark either supplies an updated Christian> distribution, or one corrects the two barfing Scintilla Christian> support scripts by hand.

Zope has the multi-arg append problem as well.

It wasn't clear to me where all the problems resided. I've been thinking mostly about the problem in Python code. Thankfully Tim released checkappend last night! Even after getting rid of all the cases it found I was still getting some flaky tracebacks which suggest to me that some C code makes the same sort of assumptions.

I've never labored under the assumption that append would take multiple arguments. I was mildly surprised to find an instance of this crop up in my own code...

On Tue, 29 Feb 2000, Christian Tismer wrote: > Greg Stein wrote: > > +1 on breaking it now, rather than deferring it Yet Again. > > > > IMO, there has been plenty of warning, and there is plenty of time to > > correct the software. > > > > I'm +0 on adding a warning architecture to Python to support issuing a > > warning/error when .append is called with multiple arguments. > > Well, the (bad) effect of this patch is that you cannot run > PythonWin any longer unless Mark either supplies an updated > distribution, or one corrects the two barfing Scintilla > support scripts by hand.

Yes, but there is no reason to assume this won't happen.

Why don't we simply move forward with the assumption that PythonWin and Scintilla will be updated? If we stand around pointing at all the uses of append that are incorrect and claim that is why we can't move forward, then we won't get anywhere. Instead, let's just *MOVE* and see that software authors update accordingly. It isn't like it is a difficult change to make. Heck, PythonWin and Scintilla could be updated within the week and re-released. *WAY* ahead of the 1.6 release.

> Bad for me, since I'm building Stackless Python against 1.5.2+, > and that means the users will see PythonWin barf when installing SLP.

If you're building a system using an interim release of Python, then I think you need to take responsibility for that. If you don't want those people to have problems, then you can back out the list.append change. Or you can release patches to PythonWin. I don't think the Python world at large should be hampered because somebody is using an unstable/interim version of Python. Again: we couldn't move forward.

> Adding a warning instead of raising an exception would be nice IMHO, > since the warning could probably contain the file name and line > number to change, and I would leave my users with this easy task.

Yes, this would be nice. But somebody has to take the time to code it up. The warning won't appear out of nowhere...

> Why don't we simply move forward with the assumption that PythonWin and > Scintilla will be updated?

Done :-)

However, I think dropping it now _is_ a little heavy handed. I decided to do a wider search and found a few in, eg, Sam Rushings calldll based ODBC package.

Personally, I would much prefer a warning now, and drop it later. _Then_ we can say we have made enough noise about it. It would only be 2 years ago that I became aware that this "feature" of append was not a feature at all - up until then I used it purposely, and habits are sometimes hard to change :-)

> However, I think dropping it now _is_ a little heavy handed. I decided to > do a wider search and found a few in, eg, Sam Rushings calldll based ODBC > package. > > Personally, I would much prefer a warning now, and drop it later. _Then_ we > can say we have made enough noise about it. It would only be 2 years ago > that I became aware that this "feature" of append was not a feature at all - > up until then I used it purposely, and habits are sometimes hard to change > :-) What's the difference between a warning and an error? If you're running a program and it suddenly spits out a warning about a misuse of list.append, I'd certainly see that as "the program did something unexpected; that is an error."

But this is all moot. Guido has already said that we would be amenable to a warning/error infrastructure which list.append could use. His description used some awkward sentences, so I'm not sure (without spending some brain cycles to parse the email) exactly what his desired defaults and behavior are. But hey... the possibility is there, and is just waiting for somebody to code it.

IMO, Guido has left an out for people that are upset with the current hard-line approach. One of those people just needs to spend a bit of time coming up with a patch :-)

And yes, Guido is also the Benevolent Dictator and can certainly have his mind changed, so people can definitely continue pestering him to back away from the hard-line approach...

On Tue, 29 Feb 2000, Greg Stein wrote: > > What's the difference between a warning and an error? If you're running a > program and it suddenly spits out a warning about a misuse of list.append, > I'd certainly see that as "the program did something unexpected; that is > an error."

A big, big difference. Perhaps to one of us, it's the minor inconvenience of reading the error message and inserting a couple of parentheses in the appropriate file -- but to the end user, it's the difference between the program working (albeit noisily) and *not* working. When the program throws an exception and stops, it is safe to say most users will declare it broken and give up.

We can't assume that they're going to be able to figure out what to edit (or be brave enough to try) just by reading the error message... or even what interpreter flag to give, if errors (rather than warnings) are the default behaviour.

> > Why don't we simply move forward with the assumption that PythonWin and > > Scintilla will be updated? > > Done :-) > > However, I think dropping it now _is_ a little heavy handed. I decided to > do a wider search and found a few in, eg, Sam Rushings calldll based ODBC > package. > > Personally, I would much prefer a warning now, and drop it later. _Then_ we > can say we have made enough noise about it. It would only be 2 years ago > that I became aware that this "feature" of append was not a feature at all - > up until then I used it purposely, and habits are sometimes hard to change > :-) I agree with mark. Why the sudden rush?? It seems to me to be unfair to make such a change - one that will break peoples code - without advanced warning, which typically is handled by a deprecation period. There *are* going to be people who won't be informed of the change in the short span of less than a single release. Just because it won't cause you pain isn't a good reason to disregard the pain of those that will suffer, particularly when you can do something relatively low-cost to avoid it.

On Tue, 29 Feb 2000, Ken Manheimer wrote: >... > I agree with mark. Why the sudden rush?? It seems to me to be unfair to > make such a change - one that will break peoples code - without advanced > warning, which typically is handled by a deprecation period. There *are* > going to be people who won't be informed of the change in the short span > of less than a single release. Just because it won't cause you pain isn't > a good reason to disregard the pain of those that will suffer, > particularly when you can do something relatively low-cost to avoid it.

Sudden rush?!?

Mark said he knew about it for a couple years. Same here. It was a long while ago that .append()'s semantics were specified to "no longer" accept multiple arguments.

I see in the HISTORY file, that changes were made to Python 1.4 (October, 1996) to avoid calling append() with multiple arguments.

So, that is over three years that append() has had multiple-args deprecated. There was probably discussion even before that, but I can't seem to find something to quote. Seems like plenty of time -- far from rushed.

> On Tue, 29 Feb 2000, Ken Manheimer wrote: > >... > > I agree with mark. Why the sudden rush?? It seems to me to be unfair to > > make such a change - one that will break peoples code - without advanced > > warning, which typically is handled by a deprecation period. There *are* > > going to be people who won't be informed of the change in the short span > > of less than a single release. Just because it won't cause you pain isn't > > a good reason to disregard the pain of those that will suffer, > > particularly when you can do something relatively low-cost to avoid it. > > Sudden rush?!? > > Mark said he knew about it for a couple years. Same here. It was a long > while ago that .append()'s semantics were specified to "no longer" accept > multiple arguments. > > I see in the HISTORY file, that changes were made to Python 1.4 (October, > 1996) to avoid calling append() with multiple arguments. > > So, that is over three years that append() has had multiple-args > deprecated. There was probably discussion even before that, but I can't > seem to find something to quote. Seems like plenty of time -- far from > rushed.

None the less, for those practicing it, the incorrectness of it will be fresh news. I would be less sympathetic with them if there was recent warning, eg, the schedule for changing it in the next release was part of the current release. But if you tell somebody you're going to change something, and then don't for a few years, you probably need to renew the warning before you make the change. Don't you think so? Why not?

Software configuration management is HARD. Every sudden backwards incompatible change (warranted or not) makes it harder. Mutli-arg append is not hurting anyone as much as a sudden change to it would. It would be better to leave append() alone and publicize its near-term removal rather than cause random, part-time supported modules to stop working because their programmers may be too busy to update them right now.

So no, I'm not stepping up to do it. But I'm also saying that the better "lazy" option is to put something in a prominent place in the documentation and otherwise leave it alone.

<aside> As far as I am concerned, a formal warning-based deprecation mechanism is necessary for Python's continued evolution. Perhaps we can even expose the deprecation flag to the programmer so we can say:

if deprecation: print "This module isn't supported anymore."

if deprecation: print "Use method FooEx instead."

If we had a deprecation mechanism, maybe introducing new keywords would not be quite so painful. Version x deprecates, version y adds the keyword. Mayhap we should also deprecate implicit truncating integral division while we are at it... </aside>

-- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml

> Software configuration management is HARD. Every sudden backwards > incompatible change (warranted or not) makes it harder. Mutli-arg append > is not hurting anyone as much as a sudden change to it would. It would > be better to leave append() alone and publicize its near-term removal > rather than cause random, part-time supported modules to stop working > because their programmers may be too busy to update them right now.

I'm tired of this rhetoric. It's not like I'm changing existing Python installations retroactively. I'm planning to release a new version of Python which no longer supports certain long-obsolete and undocumented behavior. If you maintain a non-core Python module, you should test it against the new release and fix anything that comes up. This is why we have an alpha and beta test cycle and even before that the CVS version. If you are a Python user who depends on a 3rd party module, you need to find out whether the new version is compatible with the 3rd party code you are using, or whether there's a newer version available that solves the incompatibility.

There are people who still run Python 1.4 (really!) because they haven't upgraded. I don't have a problem with that -- they don't get much support, but it's their choice, and they may not need the new features introduced since then. I expect that lots of people won't upgrade their Python 1.5.2 to 1.6 right away -- they'll wait until the other modules/packages they need are compatible with 1.6. Multi-arg append probably won't be the only reason why e.g. Digital Creations may need to release an update to Zope for Python 1.6. Zope comes with its own version of Python anyway, so they have control over when they make the switch.

On Tue, 29 Feb 2000, Ken Manheimer wrote: >... > None the less, for those practicing it, the incorrectness of it will be > fresh news. I would be less sympathetic with them if there was recent > warning, eg, the schedule for changing it in the next release was part of > the current release. But if you tell somebody you're going to change > something, and then don't for a few years, you probably need to renew the > warning before you make the change. Don't you think so? Why not?

I agree.

Note that Guido posted a note to c.l.py on Monday. I believe that meets your notification criteria.

On Tue, 29 Feb 2000, Guido van Rossum wrote: > I'm tired of this rhetoric. It's not like I'm changing existing > Python installations retroactively. I'm planning to release a new > version of Python which no longer supports certain long-obsolete and > undocumented behavior. If you maintain a non-core Python module, you > should test it against the new release and fix anything that comes up. > This is why we have an alpha and beta test cycle and even before that > the CVS version. If you are a Python user who depends on a 3rd party > module, you need to find out whether the new version is compatible > with the 3rd party code you are using, or whether there's a newer > version available that solves the incompatibility. > > There are people who still run Python 1.4 (really!) because they > haven't upgraded. I don't have a problem with that -- they don't get > much support, but it's their choice, and they may not need the new > features introduced since then. I expect that lots of people won't > upgrade their Python 1.5.2 to 1.6 right away -- they'll wait until the > other modules/packages they need are compatible with 1.6. Multi-arg > append probably won't be the only reason why e.g. Digital Creations > may need to release an update to Zope for Python 1.6. Zope comes with > its own version of Python anyway, so they have control over when they > make the switch.

I wholeheartedly support his approach. Just ask Mark Hammond :-) how many times I've said "let's change the code to make it Right; people aren't required to upgrade [and break their code]."

Of course, his counter is that people need to upgrade to fix other, unrelated problems. So I relax and try again later :-). But I still maintain that they can independently grab the specific fixes and leave the other changes we make.

Maybe it is grey, but I think this change is quite fine. Especially given Tim's tool.

[Greg Stein] > ... > Maybe it is grey, but I think this change is quite fine. Especially given > Tim's tool.

What the heck does Tim's one-eyed trouser snake have to do with this? I know *it* likes to think it's the measure of all things, but, frankly, my tool barely affects the world at all a mere two feet beyond its base <wink>.

ahem. do you seriously believe that everyone in the Python universe reads comp.lang.python?

afaik, most Python programmers don't.

...

so as far as I'm concerned, this was officially deprecated with Guido's post. afaik, no official python documentation has explicitly mentioned this (and the fact that it doesn't explicitly allow it doesn't really matter, since the docs don't explicitly allow the x[a, b, c] syntax either. both work in 1.5.2).

has anyone checked the recent crop of Python books, btw? the eff-bot guide uses old syntax in two examples out of 320. how about the others?

...

sigh. running checkappend over a 50k LOC application, I just realized that it doesn't catch a very common append pydiom.

how fun. even though 99% of all append calls are "legal", this "minor" change will break every single application and library we have :-(

my point was that most Python programmers won't see that notification. when these people download 1.6 final and find that all theirs apps just broke, they probably won't be happy with a pointer to dejanews.

> And which is that? Care to help out? Maybe just a little bit?

this rather common pydiom:

append = list.append for x in something: append(...)

it's used a lot where performance matters.

> Or do you just want to talk about how bad this change is? :-(

yes, I think it's bad. I've been using Python since 1.2, and no other change has had the same consequences (wrt. time/money required to fix it)

call me a crappy programmer if you want, but I'm sure there are others out there who are nearly as bad. and lots of them won't be aware of this change until some- one upgrades the python interpreter on their server.

[/F] > ... > so as far as I'm concerned, this was officially deprecated > with Guido's post. afaik, no official python documentation > has explicitly mentioned this (and the fact that it doesn't > explicitly allow it doesn't really matter, since the docs don't > explicitly allow the x[a, b, c] syntax either. both work in > 1.5.2).

The "Subscriptions" section of the Reference Manual explicitly allows for

dict[a, b, c]

and explicitly does not allow for

sequence[a, b, c]

The "Mapping Types" section of the Library Ref does not explicitly allow for it, though, and if you read it as implicitly allowing for it (based on the Reference Manual's clarification of "key" syntax), you would also have to read the Library Ref as allowing for

dict.has_key(a, b, c)

Which 1.5.2 does allow, but which Guido very recently patched to treat as a syntax error.

> ... > sigh. running checkappend over a 50k LOC application, I > just realized that it doesn't catch a very common append > pydiom.

This limitation was pointed out in checkappend's module docstring. Doesn't make it any easier for you to swallow, but I needed to point out that you didn't *have* to stumble into this the hard way <wink>.

> how fun. even though 99% of all append calls are "legal", > this "minor" change will break every single application and > library we have :-( > > oh, wait. xmlrpclib isn't affected. always something!

What would you like to do, then? The code will be at least as broken a year from now, and probably more so -- unless you fix it. So this sounds like an indirect argument for never changing Python's behavior here. Frankly, I expect you could fix the 50K LOC in less time than it took me to write this naggy response <0.50K wink>.

Dito. Anyone remember the str(2L) == '2' change, BTW ? That one will cost lots of money in case someone implemented an eShop using the common str(2L)[:-1] idiom...

There will need to be a big warning sign somewhere that people see *before* finding the download link. (IMHO, anyways.)

> > And which is that? Care to help out? Maybe just a little bit? > > this rather common pydiom: > > append = list.append > for x in something: > append(...) > > it's used a lot where performance matters.

Same here. checkappend.py doesn't find these (a great tool BTW, thanks Tim; I noticed that it leaks memory badly though).

> > Or do you just want to talk about how bad this change is? :-( > > yes, I think it's bad. I've been using Python since 1.2, > and no other change has had the same consequences > (wrt. time/money required to fix it) > > call me a crappy programmer if you want, but I'm sure > there are others out there who are nearly as bad. and > lots of them won't be aware of this change until some- > one upgrades the python interpreter on their server.

more time to fix it, perhaps? it's surely a minor code change, but fixing it can be harder than you think (just witness Gerrit's bogus patches)

after all, python might be free, but more and more people are investing lots of money in using it [1].

> The code will be at least as broken a year > from now, and probably more so -- unless you fix it.

sure. we've already started. but it's a lot of work, and it's quite likely that it will take a while until we can be 100% confident that all the changes are pro- perly done.

(not all software have a 100% complete test suite that simply says "yes, this works" or "no, it doesn't")

</F>

1) fwiw, some poor soul over here posted a short note to the pythonworks mailing, mentioning that we've now fixed the price. a major flamewar erupted, and my mail- box is now full of mail from unknowns telling me that I must be a complete moron that doesn't understand that Python is just a toy system, which everyone uses just be- cause they cannot afford anything better...

Fredrik Lundh writes: >yes, I think it's bad. I've been using Python since 1.2, >and no other change has had the same consequences >(wrt. time/money required to fix it)

There are more things in 1.6 that might require fixing existing code: str(2L) returning '2', the int/long changes, the Unicode changes, and if it gets added, garbage collection -- and bugs caused by those changes might not be catchable by a nanny. IMHO it's too early to point at the .append() change as breaking too much existing code; there may be changes that break a lot more. I'd wait and see what happens once the 1.6 alphas become available; if c.l.p is filled with shrieks and groans, GvR might decide to back the offending change out. (Or he might not...)

-- A.M. Kuchling http://starship.python.net/crew/amk/I have no skills with machines. I fear them, and because I cannot help attributing human qualities to them, I suspect that they hate me and will kill me if they can. -- Robertson Davies, "Reading"

> On Tue, 29 Feb 2000, Ken Manheimer wrote: > >... > > None the less, for those practicing it, the incorrectness of it will be > > fresh news. I would be less sympathetic with them if there was recent > > warning, eg, the schedule for changing it in the next release was part of > > the current release. But if you tell somebody you're going to change > > something, and then don't for a few years, you probably need to renew the > > warning before you make the change. Don't you think so? Why not? > > I agree. > > Note that Guido posted a note to c.l.py on Monday. I believe that meets > your notification criteria.

Actually, by "part of the current release", i meant having the deprecation/impending-deletion warning in the release notes for the release before the one where the deletion happens - saying it's being deprecated now, will be deleted next time around.

Ken klm [at] digicool

I mean, you tell one guy it's blue. He tells his guy it's brown, and it lands on the page sorta purple. Wavy Gravy/Hugh Romney

Andrew M. Kuchling wrote: > There are more things in 1.6 that might require fixing existing code: > str(2L) returning '2', the int/long changes, the Unicode changes, and > if it gets added, garbage collection -- and bugs caused by those > changes might not be catchable by a nanny.