Classic OOP in Python

Classic OOP in Python

I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.

Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?

For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?

Classic OOP in Python

On Wed, Jun 17, 2015 at 2:21 PM, Jason P. <suscricions at gmail.com> wrote:
> I'm gonna try to develop a modest application from ground up using
> TDD. If it had been done in Java for instance, I would made
> extensive use of interfaces to define the boundaries of my
> system. How would I do something like that in Python?

Classic OOP in Python

> Hello Python community.
>
> I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
>
> Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
>
> For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
>
>
> Many thanks!

What other languages do with interfaces, Python does with duck-typing. You
can build something like interfaces in Python, but many people don't bother.

I don't know if your project will be web-based, but here is an entire book
about developing Python web sites with a TDD approach:

TDD and interfaces are separate concepts, and I'm not sure they even
intersect. TDD is about writing tests as a way to design the best system,
and putting testing at the center of your development workflow. It works
great with Python even without interfaces.

Classic OOP in Python

> TDD is about writing tests as a way to design the best system, and
> putting testing at the center of your development workflow. It works
> great with Python even without interfaces.

I wonder how great it really is. Testing is important, that's for sure,
but to make it a dogmatic starting point of development is not that
convincing.

The way it was explained to me was that in TDD you actually don't write
code to any requirements or design: you simply do the least to pass the
tests. Thus, say you need to write a program that inputs a string and
outputs the same string surrounded by parentheses (the requirement), the
starting point might be this test case:

- run the program
- give it the word "hello" as input
- check that the program prints out "(hello)"

The right TDD thing would be to satisfy the test with this program:

input()
print("(hello)")

That *ought* to be the first version of the program until further test
cases are added that invalidate it.

Another interesting ism I have read about is the idea that the starting
point of any software project should be the user manual. The developers
should then go and build the product that fits the manual.

Classic OOP in Python

On 17/06/2015 21:39, Marko Rauhamaa wrote:

> Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.
>
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
>
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
>
> - run the program
> - give it the word "hello" as input
> - check that the program prints out "(hello)"
>
> The right TDD thing would be to satisfy the test with this program:
>
> input()
> print("(hello)")
>
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.
>
>
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.
>
>
> Marko
>

Awesome, one of the funniest things I've read in years.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Classic OOP in Python

> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
>
> - run the program
> - give it the word "hello" as input
> - check that the program prints out "(hello)"
>
> The right TDD thing would be to satisfy the test with this program:
>
> input()
> print("(hello)")
>
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.

The true TDD acolytes advocate a very idiosyncratic workflow, it's true.
I don't do this, but I also don't consider myself a TDD person. I value
tests a great deal, and put a lot of effort into them, but I don't write
trivial functions to get my tests to pass and then go back to change them.

But people whose opinion I value do advocate that, and it's possible that
in time I will understand their methods and use them myself. Stranger
things have happened...

Classic OOP in Python

> Hello Python community.
>
> I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
>
> Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
>
> For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
>
>
> Many thanks!

You don't need interfaces with Python. Duck typing makes that all possible. This is perfectly valid:

class Dog(object):
def move(self):
print "Dog is moving"

class Car(object):
def move(self):
print "Car is moving"

things = [Dog(), Car()]
for thing in things:
thing.move()

Despite Dog and Car being completely different classes with no relationship with each other, the code runs just fine.

Classic OOP in Python

> Ned Batchelder <ned at nedbatchelder.com>:
>
> > TDD is about writing tests as a way to design the best system, and
> > putting testing at the center of your development workflow. It works
> > great with Python even without interfaces.
>
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
>
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
>
> - run the program
> - give it the word "hello" as input
> - check that the program prints out "(hello)"
>
> The right TDD thing would be to satisfy the test with this program:
>
> input()
> print("(hello)")
>
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.
>
>
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.
>
>
> Marko

I had a Java class where we had to learn TDD, and that's the way TDD was taught to us, and I hated it. We watched a video of this guy explaining TDD with a hat that was red on the front and green on the back. It involved writing a simple failing unit test, then write code to fix it, then refactor the tests and/or code.

As an in-class exercise, we had to write an implementation of Conway's Game of Life. I threw TDD out the window and just wrote the damn program in under 15 minutes, then another 10 minutes to write unit tests that tested every possible code branch and several invalid inputs. Meanwhile, the people doing TDD the "right" way didn't even have a complete program after over an hour.

The brand of TTD we were taught would end up multiplying development time by at least a factor of 3, and by the time you were done, at least 75% of the tests you had written will have been removed due to rampant refactoring.

Classic OOP in Python

On 17/06/2015 22:33, sohcahtoa82 at gmail.com wrote:

> On Wednesday, June 17, 2015 at 1:39:31 PM UTC-7, Marko Rauhamaa wrote:
>> Ned Batchelder <ned at nedbatchelder.com>:
>>
>>> TDD is about writing tests as a way to design the best system, and
>>> putting testing at the center of your development workflow. It works
>>> great with Python even without interfaces.
>>
>> I wonder how great it really is. Testing is important, that's for sure,
>> but to make it a dogmatic starting point of development is not that
>> convincing.
>>
>> The way it was explained to me was that in TDD you actually don't write
>> code to any requirements or design: you simply do the least to pass the
>> tests. Thus, say you need to write a program that inputs a string and
>> outputs the same string surrounded by parentheses (the requirement), the
>> starting point might be this test case:
>>
>> - run the program
>> - give it the word "hello" as input
>> - check that the program prints out "(hello)"
>>
>> The right TDD thing would be to satisfy the test with this program:
>>
>> input()
>> print("(hello)")
>>
>> That *ought* to be the first version of the program until further test
>> cases are added that invalidate it.
>>
>>
>> Another interesting ism I have read about is the idea that the starting
>> point of any software project should be the user manual. The developers
>> should then go and build the product that fits the manual.
>>
>>
>> Marko
>
> I had a Java class where we had to learn TDD, and that's the way TDD was taught to us, and I hated it. We watched a video of this guy explaining TDD with a hat that was red on the front and green on the back. It involved writing a simple failing unit test, then write code to fix it, then refactor the tests and/or code.
>
> As an in-class exercise, we had to write an implementation of Conway's Game of Life. I threw TDD out the window and just wrote the damn program in under 15 minutes, then another 10 minutes to write unit tests that tested every possible code branch and several invalid inputs. Meanwhile, the people doing TDD the "right" way didn't even have a complete program after over an hour.
>
> The brand of TTD we were taught would end up multiplying development time by at least a factor of 3, and by the time you were done, at least 75% of the tests you had written will have been removed due to rampant refactoring.
>
> IMO, that kind of TTD is an utter waste of time.
>

Couldn't agree with you more. To once again quote my favourite part of
the Zen of Python, "practicality beats purity".

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Classic OOP in Python

In a message of Wed, 17 Jun 2015 23:39:17 +0300, Marko Rauhamaa writes:
>Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.
>
>I wonder how great it really is. Testing is important, that's for sure,
>but to make it a dogmatic starting point of development is not that
>convincing.

Try it and see.

>The way it was explained to me was that in TDD you actually don't write
>code to any requirements or design: you simply do the least to pass the
>tests. Thus, say you need to write a program that inputs a string and
>outputs the same string surrounded by parentheses (the requirement), the
>starting point might be this test case:
>
> - run the program
> - give it the word "hello" as input
> - check that the program prints out "(hello)"
>
>The right TDD thing would be to satisfy the test with this program:
>
> input()
> print("(hello)")
>
>That *ought* to be the first version of the program until further test
>cases are added that invalidate it.

You have been talking to people who are beginners at TDD. They have
been taught some hard and fast rules to keep them from the problem
of over-designing. Or you may have been taking to fools.

Both beginners and fools badly want to replace the wisdom that they
don't have in their heads with some rules from somebody else's head.
Beginners can work this way, and develop wisdom over time. Fools,
pretty much by definition cannot.

The way I usually work is to take a list of requirements from some
user(s) and turn them into test cases. And then the job of design
is to get from where I am 'have no code' to I can begin satisfying
some of these requirements.

Just because
> input()
> print("(hello)")

would satisfy the test you wrote doesn't mean it is a correct TTD thing
to do -- because it doesn't get you any closer to solving the real
problem your design is supposed to do.

This doesn't mean that you need to write your tests in a more
complicated way to prevent smart-asses from doing stuff like the
above. It means that your tests should make you get closer to your
goal. You will have to make a bridge of tests to get from 'I
have nothing' to the first user supplied test you need to meet,
but you do have a destination, and you need to go there.

I teach a bunch of 11 year olds to program in python using TTD and
pytest. A good number of them end up writing tests to test that
the python standard library works the way they expect it to. In some
sense, this is a waste of their time, but I resist the urge to
tell them so. They are developing wisdom and understanding of the
standard library, and confidence. When they are wiser, they can
delete some of these superfluous tests, but as for now, these tests
are a measure of their growing understanding. If you have
oversimplified what you have been told, then it may be that you have
missed this 'and I learned something here' bit they were telling you
about, even if you think that they ought to have known it all along.

Teaching design is hard.

Teaching design to smart-asses who don't want to learn design at all
but want some ruleset to be 'better than design' is impossible.

But teaching design using TTD as a tool for design. Well, that is
_wonderful_.

>Another interesting ism I have read about is the idea that the starting
>point of any software project should be the user manual. The developers
>should then go and build the product that fits the manual.

Unless the manual is bloody short, in which case it is a series of
acceptance tests -- this is a rotten idea. The marketing department
makes the user manual, the engineers implement it, and the users get
screwed as they get, not what they want, but what somebody in
marketing, or (only slightly better) engineering thinks that they
wanted, or ought to want.

If at all possible, you want the starting point of your software
development to be a conversation between the people who are going to
write the thing and the people who are going to use the thing, with
particular attention to what the users are using now, and why they
want something else instead of just more of the same.

>
>
>Marko

ps -- Marko, we have ample evidence that you are an extremely clever
person. But the purpose of TTD is not to make clever code, but
wise code. TTD in the hands of a fool will never produce that.
But how else do you have to check that your design, while clever
isn't _too damn clever_? Aren't you plagued with that problem? I am
and TDD is the best thing I ever learned to keep that tendancy I have
to write too clever code in check.

Classic OOP in Python

>I had a Java class where we had to learn TDD, and that's the way TDD
>was taught to us, and I hated it. We watched a video of this guy
>explaining TDD with a hat that was red on the front and green on the
>back. It involved writing a simple failing unit test, then write
>code to fix it, then refactor the tests and/or code. As an in-class
>exercise, we had to write an implementation of Conway's Game of Life.
>I threw TDD out the window and just wrote the damn program in under
>15 minutes, then another 10 minutes to write unit tests that tested
>every possible code branch and several invalid inputs. Meanwhile,
>the people doing TDD the "right" way didn't even have a complete
>program after over an hour. The brand of TTD we were taught would
>end up multiplying development time by at least a factor of 3, and by
>the time you were done, at least 75% of the tests you had written
>will have been removed due to rampant refactoring. IMO, that kind of
>TTD is an utter waste of time.

I'd hate that too. But that's not the TDD I know, or teach. Please
don't write off TDD as a result of this experience.

Classic OOP in Python

On 17/06/2015 23:09, Laura Creighton wrote:
>
> ps -- Marko, we have ample evidence that you are an extremely clever
> person. But the purpose of TTD is not to make clever code, but
> wise code. TTD in the hands of a fool will never produce that.
> But how else do you have to check that your design, while clever
> isn't _too damn clever_? Aren't you plagued with that problem? I am
> and TDD is the best thing I ever learned to keep that tendancy I have
> to write too clever code in check.
>

Strange, I see no such evidence, just a person who lives firmly in cloud
cuckoo land.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Classic OOP in Python

>The true TDD acolytes advocate a very idiosyncratic workflow, it's true.
>I don't do this, but I also don't consider myself a TDD person. I value
>tests a great deal, and put a lot of effort into them, but I don't write
>trivial functions to get my tests to pass and then go back to change them.

Grrr. I am a true TDD acolyte. And I don't do that either.

>But people whose opinion I value do advocate that, and it's possible that
>in time I will understand their methods and use them myself. Stranger
>things have happened...
>
>--Ned.

Seriously? Oh you humble man.

I have convinced a huge number of people who used to have that sort of
dogma that the reason they advocated such baby steps is that they
didn't have a good coverage tool. ***grin***

Classic OOP in Python

On Wednesday, June 17, 2015 at 6:34:23 PM UTC-4, Laura Creighton wrote:
> TDD is supposed to make you brave, not cowards, and it's
> Ned's most excellent tool
> http://nedbatchelder.com/code/coverage/> that I recommend to TDD dogmatic cowards.
>
> Even if you don't want to use TTD, you will enjoy Ned's tool. It's GREAT.
> Thank you, Ned.
>
> Laura

Classic OOP in Python

> Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.
>
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
>
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests.

You still need requirements and a design. Otherwise how do you know what
tests you need to write?

> Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
>
> - run the program
> - give it the word "hello" as input
> - check that the program prints out "(hello)"
>
> The right TDD thing would be to satisfy the test with this program:
>
> input()
> print("(hello)")
>
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.

That's certainly the more pedantic approach of TDD proponents. But frankly I
expect that outside of TDD tutorials, nobody *really* starts off that small
in practice. I know I don't.

And from there, I would build up extra doc tests and unit tests as needed.

The point is, I don't believe that there is any *necessity* to write
deliberately failing tests for *trivial* functions that are only one or two
lines. You can practically see that the function is correct without even a
test. Deliberately making it fail has only one advantage:

* Getting you used to failing tests and learning that they aren't in and of
themselves a bad thing or something to be ashamed of;

* Instilling the discipline to actually write tests.

Er, that's two advantages.

But really, in practice, how many trivial one liner functions do you end up
writing, as a percentage of your entire application? Unless the app is
pretty simple, chances are that percentage is very small.

Classic OOP in Python

On 6/17/2015 4:39 PM, Marko Rauhamaa wrote:
> Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.

I use what I might call 'smart TDD'. I think it a mistake to dismiss
TDD because 'stupid TDD' is possible, and maybe even extant.

The reason for having a test initially fail is to make sure that the
test is called. A unittest example:

Why does this 'pass'? Within the last year, it was discovered that a
TestCase subclass in one of the Python test files was not being run. I
made a similar mistake in my own code. The more I do TDD
(intelligently), the more I like it.

> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.

The bugs in Python, whether already fixed, on the tracker, or yet to be
recorded, are the result of CPython not originally using TDD of some
version. In my opinion, CPython maintenance and enhancement has
improved since some semblance of TDD was instituted. A runable test is
now the first stage for tracker issues.

> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design:

Writing sane code to pass tests based on requirements or a design is
intended to ensure that one is actually meeting the design.

> you simply do the least to pass the tests.

The incantation I have read is 'simplest', not 'least'. I personally
interpret 'simplest' to include some presumption of sanity in relation
to the ultimate goal.

> Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement),

One should write a reusable and easily testable *function* that takes a
string as input (and outputs the same string surrounded by parens).
Call the function paren_around.

> the starting point might be this test case:
>
> - run the program
> - give it the word "hello" as input
> - check that the program prints out "(hello)"

Doubly bad.

1. I consider mixing program i/o with function not related to i/o to be
a bad idea. TDD should cure one of this madness, not reinforce it.

2. The first test for smart TDD should best be with a falsey input (or
other corner or edge case). With this guideline, there is a good chance
the 'simplest code' will be augmented rather than replaced with
additional tests. A smart TDD test might be

assertEqual(paren_arount(''), '()')

> The right TDD thing would be to satisfy the test with this program:
>
> input()
> print("(hello)")

This does not work. The above adds '\n' to the end of the string, which
is not part of the specification. A human may not notice, but an
automated test that captures stdout and compares with "(hello)" will
notice and fail.

> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.

As a statistician, I know about the mistake of overfitting functions to
data. 'Teaching to the test' is a related mistake Smart TDD should not
reinforce bad practice. This simple code passes the initial testcase above.

> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.

The corresponding TDD approach is to write tests from the manual, and
then write code. One can also write tests from docs for existing code
- and then see if the code passes the doc-based tests. This is one way
people discover bugs.

Classic OOP in Python

On Thu, Jun 18, 2015 at 6:39 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.

I've seldom met a *user* manual that's truly the right way to start
building, but there have been times when I've built *API*
documentation prior to the code. That can work fairly well. I'll also
often start a project with a copious set of notes that aren't quite
user-facing, aren't quite programmer-friendly, but are somewhere in
between. Here's a new project I'm starting now:

Prior to actually creating that repo, I'd done some thrash testing of
the basic concepts (which is how I know that the basic idea will work
- my thrash test achieved 100tps, massively exceeding the 40tps that I
need, ergo it's worth proceeding to code), but the project itself
started with the README, then the .sql files giving a basic run-down
of the tables required. Now, and only now, I'm starting to look at
actual code. Is that starting with the user manual? Not quite, but I
think it captures the same concept that that's trying to capture.

Classic OOP in Python

On 17/06/2015 23:33, sohcahtoa82 at gmail.com wrote:
> I had a Java class where we had to learn TDD, and that's the way TDD was taught to us, and I hated it. We watched a video of this guy explaining TDD with a hat that was red on the front and green on the back. It involved writing a simple failing unit test, then write code to fix it, then refactor the tests and/or code.

I do not think it is important that the test fails before writing the
code, but for sure it is really wise to ensure every test fails at least
once. If I wrote a test and it has never failed before, then I always
change something in the test itself, in order to have an expected
failure. And sometimes the expected failures make me better understand
the behavior of the code.

> As an in-class exercise, we had to write an implementation of Conway's Game of Life. I threw TDD out the window and just wrote the damn program in under 15 minutes, then another 10 minutes to write unit tests that tested every possible code branch and several invalid inputs. Meanwhile, the people doing TDD the "right" way didn't even have a complete program after over an hour.
>
> The brand of TTD we were taught would end up multiplying development time by at least a factor of 3, and by the time you were done, at least 75% of the tests you had written will have been removed due to rampant refactoring.
>
> IMO, that kind of TTD is an utter waste of time.

I think TDD helps a lot to design the code to be "testable" as much as
possible. I believe good and experienced programmers could think in
advance the way to write testable code, without be driven by tests.
However, I also think that the bigger part of programmers, as me, do not
have this ability. That's the case TDD comes in handy helping us to
write testable code.

In case of bug fixing, I think also experienced and good programmers
that do not believe in TDD should always write the tests in advance, for
instance because:

* it is really the better way to understand the problem and point to it
* chances are that you will not write a test after fixing the bug, i.e.
because you may have other priorities, so you will miss a regression test