Posted
by
CmdrTaco
on Tuesday November 30, 2004 @11:31AM
from the watch-out-for-the-teeth dept.

Eventh writes "The final release of Python 2.4 was just released. Python 2.4 is the result of almost 18 month's worth of work on top of Python 2.3. New features are, but not limited to, function decorators, generator expressions, a number of new module and more. Check out Andrew Kuchling's What's New In Python for a detailed view of some of the new features of Python 2.4. "

It took them a while, but it's worth it. Here's some of the new features:

* multi-line imports - when using imports in the form from foo import bar, baz, bing, bang, you can surround the imported names with brackets, and they can be split across lines. This is part of PEP 328.
* Farewell to OverflowWarning - as documented in PEP 237, Python no longer generates OverflowWarnings.
* function/method decorators - function and method decorators, first described in PEP 318, have been added to the language, using 'pie-decorator' syntax. Decorators are on the line before the 'def', and prefixed with an '@' sign. (PEP 318)
* Assigning to None - the compiler now treats assigning to None as a SyntaxError.
* Failed import cleanup - when a module import failed, versions of Python prior to 2.4a2 would leave a broken module in sys.modules - subsequent attempts to import the failing module would silently succeed, but use the broken module object. The import machinery now removes the failing module from sys.modules if the import fails.
* The -m command line option - python -m modulename will find a module in the standard library, and invoke it. For example, python -m pdb is equivalent to python -m/usr/lib/python2.4/pdb.py

But it's important to remember that generator expressions are slower than list comprehensions. This is mainly because functions calls are expensive in Python, and when dealing with generators each iteration means one more function call.

Early timings showed that generators had a significant performance advantage over list comprehensions. However, the latter were highly optimized for Py2.4 and now the performance is roughly comparable for small to mid-sized data sets. As the data volumes grow larger, generator expressions tend to perform better because they do not exhaust cache memory and they allow Python to re-use objects between iterations.

Wow, generators, decorators.... how much more linguistic cruft is it gonna take before they just give up and implement Ruby style blocks? It seems like 90% of these are created because lamdba sucks and def isn't anonymous.

It seems like 90% of these are created because lamdba sucks and def isn't anonymous.

Uh, you do realize that generators have nothing to do with anonymous functions, and that it's rather a simplified, understandable approach for providing coroutines? Generators are significantly more powerful than Ruby code blocks.

At a language level, it's true that generators don't have anything to do with lambda or code blocks. But when you consider what these features are used for, there is some overlap.

Consider the example you gave:

print sum(obj.count for obj in list_all_objects())

In Ruby, you can accomplish the same thing by writing:

print (objGenerator.map { |obj| obj.count } ).sum

where "objGenerator" is an object that mixes in the "Enumerable" module. An even better way (that really avoids building an intermediate list, even of integers) is:

a = 0objGenerator.each { |obj| a += obj.count }print a

There is one thing that generators give you that blocks can't, that is very, very cool. With a generator, you can create programs with a pipeline architecture, where different steps of the pipeline all can be written as if they have the main loop.

Imagine you are writing a compiler. You could write (be easy on my syntax, I haven't written Python in a while)

The beauty is that both your lexer and your parser can be written as if they have the main loop, and all they have to do is yield a token/expression when they find one. In reality, it's pipelined and you get the efficiency of not having to build up the entire list of tokens before you start parsing.

Linus' comments in the kernel's CodingStyle document are relevant too -- try to keep your functions so that you can read most of them on one screen. Python allows you to do this more often (I find) without resorting to strange brace positions.

Linus' comments in the kernel's CodingStyle document are relevant too -- try to keep your functions so that you can read most of them on one screen. Python allows you to do this more often (I find) without resorting to strange brace positions.

Example: Say you're debugging a script on an 80 character wide TTY display. Do you really want to go through each line, scrolling all the way to the end because your language has significant whitespace?

Example: Say you're debugging a script on an 80 character wide TTY display. Do you really want to go through each line, scrolling all the way to the end because your language has significant whitespace?

Uh, no, I don't... which is why I'm glad this is perfectly valid python syntax:

Say it's someone else's script. You'd have to write another script to process the original script to add the escape characters where they're needed and regular linebreaks where they're not.

Someone else's script? Written in what, python? Well then I don't need to process anything. It is either valid python or not. If you're just talking about including some pre-formatted text into a python script, then just enclose it in a multi-line string (which in python are inclosed by """ (three double quotes in a row)).

And I still think that if there's not a visible character there, it shouldn't mean anything. It goes against logic: Nothing isn't something.

I used to believe python's whitepsace-sensitive C syntax would really get on my nerves if I ever tried python, since I'm used to C and perl. Like many people, I wouldn't even give python a shot because of that whitespace issue. But one day I did because I needed to help a friend, and I actually found the language quite pleasant.

The whitepsace sensitivity didn't get in my way like I thought it would (which is to say it never got in my way, I didn't even notice it). Python's rules for whitespace are quite logical and intuitive and follow the way you're already used to indenting for readability in C or perl. When I started using python, I didn't even have to learn how the whitespace worked like I would have to learn any other syntax. The whitespace rules already worked the way I always programmed, so once I started python I had no trouble at all.

One of the biggest drawback of the whitespace issue is that you don't get a real lambda in Python for basically just that reason. The lambda that is in Python is limited to a single expression (ie. single line) and thus beside trivial stuff completly useless. Naming functions kind of really sucks if all you wanna do is something very little.

The whitespace issue makes of course other things like copy&paste a whole lot harder too.

Overall I just wish they would introduce some kind of construct to get awa

I read two Python books and (I still don't like this language but) whitespace is not one of its problems. <troll>You have to admit it's very confusing for n00bs (especially if you look into some real OSS programs)</troll> but it's definitely no more an issue after one chapter of a good book [diveintopython.org].

The What's New document gives a somewhat inaccurate description of the importance of the Decimal type. Naturally, switching from binary to decimal changes which fractions can be exactly represented, but it does not change the fact that some cannot.

The importance of decimal arithmetic is not that it is more accurate than binary, but rather that it conforms more closely to what people expect from using decimal in daily life. These expectations are codified into various social rules such as accounting practices.

There's been some heated discussion of Python 2.4's Decimal, in this very regard, on the Lambda the Ultimate [lambda-the-ultimate.org] languages blog.

That entire "debate" consists of one idiot flipping a shit because of a poorly worded snippet from a pre-release version of the "What's New?" overview. He didn't even bother to reference the appropriate PEPs and documentation before grossly misrepresenting the point of the new Decimal type.

No, the Decimal type isn't IEEE 754 standard. No, it doesn't solve, or intend to solve, all the world's numerical analysis problems. Yes, it is a good thing in a language intended to minimize the dissonance between user

Agreed: decimal representations are pretty useless. You might as well use binary integers and appropriate scaling. And that is what any implementation of decimal arithmetic should be: binary numbers with implicit decimal scale factors.

Python is a pretty good language for general use and for teaching. If fact, what got me interested in Python was a story (on Slashdot) about programmers at Sun saying that Python was a better language than Java in their environment.

Having said the above, there always seem to be 'issues'. Floating point numbers are one such issue. This release fixes that with 'Decimal'. The trouble is, you have to know about 'Decimal' before you can use it. This raises the difficulty of using Python.

The decimal type offers a fix for a (very slight) inaccuracy that's inherent with floating point values on a machine level. It's not a programming language "issue" at all.

Having said that, I agree it takes some time to become efficient in Python. I wish/hope (ought to check again) that Python's potentially excellent built-in documentation string system becomes/is more accessible and useful. That'd help a lot with getting to know modules.

The Boost [boost.org] project has Boost.Python [boost.org]. I haven't used it yet, but the docs make it sound very interesting. It looks quite simple to write base classes in C++ and then subclass them in Python.

One can write extensions in Pyrex [canterbury.ac.nz], which is a language that is very similar to Python. Pyrex code is converted to C, which is ccompiled into an extension to Python. One can also access any C libraries from Pyrex.

I had absolutely no interest in Python until I read Slashdots review of Dive Into Python [slashdot.org]. Its right on target and got me excited to run home and see what I could do. Of course that only lasted for about a month, but I'll be sure to head home and take a peak at it again.

There are GUI toolkits which typically are added on as an extension to something like Python. I am learning Ruby and can choose from Qt, Tk, Wx, Fox, etc. As long as someone has created the extensions you are good to go.

There are many gui widget libraries available for python, and several RAD tools to get into 'em quickly. Try Boa constructor http://boa-constructor.sourceforge.net/ [sourceforge.net] or google for pyglade

Can one tell me why I should learn python and not any other programming language anyway?

Well, some of the things I like about python:1: tightly structured code.2: less punctuation.3: more readable syntax.4: full OOP from the ground up (in contrast to perl where the OOP is bolted on with references).5: short production cycle.

Many of these things can be found in other languages as well. So it largely comes down a matter of preference.

From the little I have seen, python seems to be a command line language. Is it anywhere similar to Visual Basic, which I have come to see and experience through a GUI?

Can one tell me why I should learn python and not any other programming language anyway?...uh, because your brain is actually capable of learning more than one way to do things. Some people even know up to *three* programming languages!

Sarcasm aside, it's not uncommon for me to work in four vastly different programming languages in a single day (usually java, ocaml, python, and C). That's not counting the ``lesser'' languages like sh, javascript, etc...

Can one tell me why I should learn python and not any other programming language anyway?

I expect this isn't the answer that true Python devotees would express, but here goes anyway: It's a very high-level, very dynamic, language that's reliably cross-platform.

"Very high-level" differentiates it from Java, which I see as more mid-level. It's also different than Perl 5, which is higher-level than Python in some ways, but convoluted and crusty in other ways (anything involving nested data structures, for example).

"Dynamic" means you can test code interactively, you don't a build process, you don't waste time enumerating things and creating redundant headers and so on.

"Reliably cross-platform" is the key. This is where Scheme and Lisp and Haskell fall down. Lisp has a standard definition, but the community is fractured by there not being a standard implementation. You can argue that diversity is good and all that, but it does tend to hurt overall. Python has a huge number of standard library modules as a result.

The "eye opener" for me was reading the Python cookbook. Almost every recipe listed 2-3 ways to do everything, most of them hard to understand and using Python "tricks". Many of them depended on new 2.x features. I realized that there are now two kinds of classes in Python for instance.

The Python community has also grown arrogant like the Perl community. If a newbie writes something that can be re-written with a list comprehension for instance, it's pretty much given that nobody will act

If a newbie writes something that can be re-written with a list comprehension for instance, it's pretty much given that nobody will actually answer his question, choosing instead to re-write his program for him using tricks and shortcuts.

List comprehensions are not "tricks," they are an extremely powerful language feature. Newbies should be taught how to use them, and translating their loop-based code into list comprehensions is probably the best way to do that.

Python can be written from either an imperative standpoint or a pseudo-functional one. Most of the highly skilled Python programmers code in the pseudo-functional style, because it is more efficient and (arguably) more elegant.

Sure, you can get into some pretty scary territory with various combinations of sum(), map(), reduce(), and list comprehensions but that's your choice. I admit that there should not be such a big performance gap between the different styles... This is due to not enough effort being spent on improving the VM.

The 'one best way to do it' mostly mean basic language operations, like iterating over a list. Many recipes in the Python cookbook are really mildly complex utility programs. They are differ in features, complexity, performance and intented use. The are naturally many ways to do similar things. And if you take these variables into account, you will find they are not really doing the 'same' thing.

Since Python 2.x there is a move toward iterators and generators. Rather than seeing this as bloats I would say this Python is maturing to handle real world applications. If you are handling simple data and performance is not an issue, use regular lists. If you writing application to handle arbitrary large data set or performance is an issue, go for iterators and generators. List comprehension is a neat language construct but I don't see any need to rush rewriting anything.

Or has python strayed from it's original philosophy of 'one best way to do it'?

To a degree, yes. Largely to the extent it has it is a result of backwards-compatibility; while the Python designers do not make the mistake of enshrining reverse compatibility above all else, they do try to avoid gratuitously compromising it.

As a result, as better ways to do things have emerged, sometime the old ways hang around and muss things up. However, for any given version there is always a "core" that you can stick to that is very close to "one best way to do it"... and despite what a first reading tells you, you really don't throw much of the language away.

For instance, while in modern Python you can say either

a = []for i in range(5):
a.append(i*2)

or

a = [i*2 for i in range(5)]

the latter is the "one right way to do it". Few, if any, new additions truly leave you with two equally good choices.

(One of Guido's examples is the "lambda" statement, but a lot of people, including me, rather like not needing to replace

self.register("event", lambda event: self.keyPressed())

with

def handler(event):
self.keyPressed()self.register("event", handler)

but hey, that's life. (While that isn't code for any GUI toolkit in particular that is a pattern common to all of them.))

By and large, none of these things have affected the difficulty of learning Python from scratch and using its libraries. It has affected the difficulty of reading other people's code, but I find the alternative, "keeping the language stagnant indefinately", completely unacceptable, and frankly, reading code is hard anyhow. (I was writing code for others long before I was reading other's code.)

In fact, given as that is the alternative, "keeping the language stagnant indefinately", while I concede it is somewhat sad that we can't jump to an optimal language immediately so that nobody ever has to learn anything past their first impression (not sarcastic, that would be the ideal), that doesn't seem to be working for folks. You might try LISP, though, I gather that hasn't changed syntax, much. (Though I also gather mature programs in that language tend to start looking like their own languages themselves, so that just may move the pain...)

Take a look at this, and the very elegant Scheme dialect that was designed to go along with it.How To Design Programs [htdp.org]
One semi-unique feature is that it asks you what your skill level is when you launch the environment, and tailors the feature-set accordingly. Novices see a simple subset of the language, and experts get the full power inherent in a real-deal Lisp dialect.

I've always liked Python, but I don't think this update is enough to make me learn it.

In one respect, it is exactly what I've been hoping for. No more sweeping changes or vast syntactic variances, but they have eliminated some usability problems and silly errors. It's a very mature language now, and seems to be behaving as such, this makes me happy

Still, though, they seem to be competing for a niche that Perl has a deathgrip on for me. I use Bash whenever I can, Perl when I can't or it would be ugly,

I'm not sure what it means, but Python is looking an awful lot like CommonLisp, down to the somewhat controversial syntax.

There are two big differences. One is that CommonLisp made it a lot easier to treat programs as data and vice versa, and it had a built-in high-performance native compiler. On the other hand, Python integrates a lot better with Linux and UNIX, there are tons more libraries for it, and is easier for new users to learn.

You don't use lambda. You define a function (with def) under a temporary name and return it. The temporary name is local to the enclosing function, so it doesn't pollute the namespace: Python def differs in this regard from CL defun.

But seriously, Common Lisp is the Common Lisp for the 21st century. It's doing quite nicely; in the past couple of years folks have taken this "old" programming language and added excellent support for all kinds of current needs -- a Web application server (TBNL), a remarkable SQL interface (CL-SQL), general-purpose dataflow programming (Kenny Tilton's Cells), PDF typesetting and generation (CL-PDF, a rival to TeX!), OpenGL....

(Peter Seibel's Practical Common Lisp [gigamonkeys.com] is highly recommended for the "modern" programmer interested in this "old" language. Practical examples include spam filtering, an MP3 database, and a Shoutcast server.)

Python is a lot like CommonLisp: dynamic typing, reflection, eval, lexical scoping, extensive iteration and looping constructs, strings-as-sequences, and on and on.

For once thing is the "big difference" you describe: you can't transparently process code as data. That means no MACROS, which is what makes Lisp so damn powerful.

Go back and read the original papers on hygienic macros: you don't need Lisp syntax or code-as-data in order to have a macro facility as powerful as Scheme's (and Scheme got by for many years without macros anyway). I wouldn't be surprised if Python at some point gets a hygienic macro facility. Furthermore, there is a separate data syntax for Python that takes source that looks very similar to Python code and represents it as a DOM tree.

How do you return an anonymous function from a function in Python?

In the obvious way:

A ``def'' form executed inside a function definition defines a local function that can be returned or passed around. Free variables used in the nested function can access the local variables of the function containing the def.

That part is actually more natural than doing the same in CommonLisp (Python, like Scheme, but unlike CommonLisp, does not have separate value and function slots on symbols).

How do you build a function at run time? It's not easy or obvious.

There are two things you could mean by that. The first is to build it from source or structure. You can do this:

exec "def c(x): return x*x*x"

The second is to build complex functions using functions that take functions as arguments. You do that the same way you do in CommonLisp, since Python supports the same primitives: lexical closures, dynamic typing, and functions that take functions as arguments.

I think people see "lambda" and they somehow think that Python has something to do with Lisp.

I think Lisp zealots incorrectly think nothing that isn't exactly Lisp could even come close. Python, in fact, is very close to Lisp; the two big differences are syntax and lack of macros. Lack of macros can be addressed, and there are separate Python-like syntactic representations of data.

Currently in Python, the C floating point libraries used will produce this:

>>> 1.11.1000000000000001

Thus, the Decimal data type was born.

From PEP 327: "The inaccuracy isn't always visible when you print the number because the FP-to-decimal-string conversion is provided by the C library, and most C libraries try to produce sensible output. Even if it's not displayed, however, the inaccuracy is still there and subsequent operations can magnify the error."

No. This shows that interval arithemtic is needed. Why can't I know the precision of results? The printer should be able to know that the value is only accurate to n decimals (or binimals or whatever they're called:), and not print more. We let programs make a mistake that would have made us lose marks in Junior High, and (hopefully? I'm in health now, so I don't know) failed us in College.

Its nice to see a language evolve in favor of use without sacrificing readability and overall utility.

Before:

def foo(cls):
passfoo = synchronized(lock)(foo)foo = classmethod(foo)

After:

@classmethod@synchronized(lock)def foo(cls):
pass

Readable is of course in the eye of the beholder... and I am a C and Java programmer with a sizable fetish for D [digitalmars.com]. So while I don't find the syntax all that pleasing to me in terms of my own work, it certainly changes things for reading python code.

I was also stunned to learn how flexible decorators were in the previous version of python. It's refreshing to be see functions treated as objects... unless I'm mistaken about the concept.

However its a huge shame that the new decorator syntax isn't supported for classes in 2.4. Seems like that's going to become a wart on a rather consistent language syntax, IMO.

I was also stunned to learn how flexible decorators were in the previous version of python. It's refreshing to be see functions treated as objects... unless I'm mistaken about the concept.

Another important part has been Python 2.2's new-style classes and descriptors [rcn.com]. Descriptors are basically objects with special methods (__get__, __set__, __delete__). When you access a class or instance attribute, instead of returning the descriptor, the a method of the descriptor is called. This is what allows clas

So now that Python is using the '@' character for function decorators the Pythonistas can no longer crticize other languages like Ruby (which uses sigils responsibly) for their use of '@' (I'm not including Perl in the list of languages that use sigils responsibly, btw:)

Welcome to the sigil club! Pandoras box is now open! '$' can't be too far behind:)

In some of my last projects I had to analyze a Perl and a Java program. I programmed a few years in both of these, but now after a year with Python I was truly suprised of how primitive these languages felt.

All those funny symbols, casting back and forth in perl just getting in the way yet don't really say anything useful... here is an example:

or that uselessly verbose java where you have to write X number of lines before any action starts...

Python is a simple, clean and powerful language where the real value comes tomorrow or next month, when you have to understand and modify what you wrote today.
There are no objective measures of this quality you have to try it to believe it.

after Linus's comments I am inclined to get more profficient with Bash and C and almost ignore Python completely. It's so dissapointing though - I really wanted to learn Python; it's such a neat language.

Linus explicitly mentioned that he doesn't do anything "in the middle" - it's either kernel hacking or something trivial enough to do with bash. Just go ahead and learn Python - you will find that it's *easier* than bash, especially if your programs might have errors (which they do).

BTW, why would you want to get more proficient in C? Programmers are abandoning C in droves. It's just not programmer-time efficient to do things in C anymore. It's one thing if you are maintaining a project that was written in C originally, but for new projects, C is a non-starter.

Go read ESR's "The Art of Unix Programming", available online for free.

BTW, why would you want to get more proficient in C? Programmers are abandoning C in droves. It's just not programmer-time efficient to do things in C anymore. It's one thing if you are maintaining a project that was written in C originally, but for new projects, C is a non-starter.

I think one of the problems is too many people are still spreading the myth that it's essential to learn C before moving on to C++, which is totally false, C++ is a language in itself, and can be treated as such. Learning C (unless you intend to use it) is a waste of time, and I would go so far to say learning C first will make you less producive in C++, because it teaches concepts which are not applicable, or are actually bad habits when used in C++. At least that's my view on the subject.

That said, C still has its uses, but for many projects (like parent said) it's a "non-starter"

IMHO - there are still uses for C, there are absolutely no uses for C++ nowadays. Moreover - there is no C++. I mean - I've been tracking (and sometimes using) C++ developments since the early days till recent and I can tell you - the amount of pileup that they produced in modern C++ compared even to early C++'s (not to say about more sane languages) is totally ridiculous. No - inventing a new language construct for bloody every particular problem does not make a good l

If you're going to nitpick about how man CPU instructions it takes to return a std::string, you better not use any high-level language.

Take a look at the job market, good knowledge of C++ is still an invaluable skill.

Moreover take a look at libraries like boost [boost.org] or systems built on C++ like KDE. Well written C++ can be just as elegant as any language (it's always in the programmers hands, they can turn anything in to spaghetti). There's a reason why it's been so popular and many of its concepts are used

BTW, why would you want to get more proficient in C? Programmers are abandoning C in droves. It's just not programmer-time efficient to do things in C anymore.

Efficient use of programmer time is only one reason to select one language over another. I still code a lot in C on my own projects simply because I enjoy it. Yes, you heard right. I enjoy hunting down stupid off-by-one errors, iteration bugs, memory corruption, silently conflicting declarations, prototype errors, implicit casting errors, rounding

If all the C programmers vanish, what are we going to write the next generation of high-level languages in?

That's a very good question. However, just because programming in C is difficult doesn't mean that it's the best language to write high-level languages in. C has many design flaws that people have gotten so used to, they assume there's no other way of doing things.

If you like programming in C because it's a challenge, then by all means do so. However, I would classify that hobby with people who prog

BTW, why would you want to get more proficient in C? Programmers are abandoning C in droves. It's just not programmer-time efficient to do things in C anymore. It's one thing if you are maintaining a project that was written in C originally, but for new projects, C is a non-starter.

nice try.

C is the defacto for embedded systems. it's smaller and faster and there are libs available that make your app 1/10th the size of a regular C app.

If you are looking to do something that Bash can do but probably shouldn't (most things people use Bash for fall into that category) use perl, not python. Perl is very bash-like and porting things to perl is easy.

Of course, this is why so many perl programs look like a complete mess, people do quick ports and don't learn how to properly use the language. Do the next guy a favor and do "man perlstyle" early on.

Here's a novel idea: Use the right tool for the job. There's a very large class of problems for which neither c nor bash are particularly appropriate programming languages. I suspect that Linus was attempting to comment on the partcular classes of problems on which he typically works, not on the merits of any specific programming language. If everything you're doing now can be done with bash, then use bash. However, don't rule out Python just because Linus doesn't use it. One day, you'll need to do som

Linux is good at what he does, but what he does represents only a small slice of programming. So, his choice of language and tools really shouldn't have any bearing on your choice unless you happen to be in the business of writing big, monolithic kernels for PC hardware.

Actually, a lot of language and systems designers have blinders on when it comes to the needs of large chunks of their potential user communities. In particular, numerical and scientific users get short-changed in this because most people

Linus is not some kind of deity. He (by his own account) does nothing between kernelspace and small scripts.

I'm very proficient with C, Python and Bash, and I can tell you for a fact: Bash and Python barely compete against each other.

Bash has nothing in the way of nice datatypes. Bash is very slow while Python can be nearly as fast as C (I've gotten it up to 80% when relying heavily on C libraries like regex). Bash can't do more than the most trivial things without helper programs, which while useful, takes forever because it has to keep spawning processes.

For a high level language, Bash has pathetic memory management. Pretty much the only way to get some things done is tempfiles, which is worse than malloc because they're not removed if you don't clean them up.

If you can't hold more than two languages in your head, go home and learn Java and C#. You're only going to get made fun of on slashdot.

Don't worry about it. It's still a good book and very little (if anything) has changed that affects the book's accuracy. It won't tell you about new features of course, but those are all either dead easy to learn or not likely to matter to the average new pythonista.