Augument assignment versus regular assignment

I do the following. First create lists x,y,z. Then add an element to x
using the augumented assignment operator. This causes all the other
lists to be changed also.
But if I use the assignment x=x+[4] instead of using the augumented
assignment, the y and z lists do not change.
Why is that?
This does not happen when I work with integer data type for example, as
shown below.

Advertisements

nagy wrote:
> I do the following. First create lists x,y,z. Then add an element to x
> using the augumented assignment operator. This causes all the other
> lists to be changed also.
> But if I use the assignment x=x+[4] instead of using the augumented
> assignment, the y and z lists do not change.
> Why is that?
> This does not happen when I work with integer data type for example, as
> shown below.
>
> Thanks for your help
> Nagy
>
>
>>>>x=y=z=[]

In this example, the '[]' creates a new list object. x, y, and z are all
set to reference that object.
>>>>x+=[2]

This creates a new list that is the concatenation of the list created
above (the list [2]) with a new list (the list [4]). This brand new list
is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That
is, they still point to the original list.
>>>>
>>>>x
>
> [2, 4]
>
>>>>y
>
> [2]
>
>>>>z
>
> [2]
>
>>>>a=b=4

This attempts to mutate the integer object 4, by adding 2 to it.
However, numbers in Python are immutable, and so the in-place operation
fails. Thus, it creates a new integer object equal to 6 (actually,
CPython keeps a cache of certain smaller integer objects and reuses
them, but this does not matter in practice). This new integer object is
bound to the name 'a'. The name 'b' remains bound to the original 4 object.
>>>>a
>
> 6
>
>>>>b
>
> 4
>

Advertisements

Thanks, Kirk.
I considered the += as only a shorthand notation for the assignment
operator.
Since for lists + is simply a concatetation, I am not sure it x=x+[2]
is creating a brand
new list. Could you refer me to any documentation on this?
Thanks,
Nagy
Kirk McDonald wrote:
> nagy wrote:
> > I do the following. First create lists x,y,z. Then add an element to x
> > using the augumented assignment operator. This causes all the other
> > lists to be changed also.
> > But if I use the assignment x=x+[4] instead of using the augumented
> > assignment, the y and z lists do not change.
> > Why is that?
> > This does not happen when I work with integer data type for example, as
> > shown below.
> >
> > Thanks for your help
> > Nagy
> >
> >
> >>>>x=y=z=[]
>
> In this example, the '[]' creates a new list object. x, y, and z are all
> set to reference that object.
>
> >>>>x+=[2]
>
> This does an "in-place" operation on that list, modifying (or
> "mutating") the object directly.
>
> >>>>x
> >
>
> > [2]
> >
> >>>>y
> >
> > [2]
> >
> >>>>z
> >
> > [2]
> >
> >>>>x=x+[4]
>
> This creates a new list that is the concatenation of the list created
> above (the list [2]) with a new list (the list [4]). This brand new list
> is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That
> is, they still point to the original list.
>
> >>>>
> >>>>x
> >
> > [2, 4]
> >
> >>>>y
> >
> > [2]
> >
> >>>>z
> >
> > [2]
> >
> >>>>a=b=4
>
> This binds the names 'a' and 'b' to the integer object 4.
>
> >>>>b
> >
> > 4
> >
> >>>>a+=2
>
> This attempts to mutate the integer object 4, by adding 2 to it.
> However, numbers in Python are immutable, and so the in-place operation
> fails. Thus, it creates a new integer object equal to 6 (actually,
> CPython keeps a cache of certain smaller integer objects and reuses
> them, but this does not matter in practice). This new integer object is
> bound to the name 'a'. The name 'b' remains bound to the original 4 object.
>
> >>>>a
> >
> > 6
> >
> >>>>b
> >
> > 4
> >

nagy wrote:
> Thanks, Kirk.
> I considered the += as only a shorthand notation for the assignment
> operator.
> Since for lists + is simply a concatetation, I am not sure it x=x+[2]
> is creating a brand
> new list. Could you refer me to any documentation on this?

Yes:

http://docs.python.org/ref/augassign.html
"An augmented assignment expression like x += 1 can be rewritten as x =
x + 1 to achieve a similar, but not exactly equal effect. In the
augmented version, x is only evaluated once. Also, when possible, the
actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead."

This behavior is only logical. Consider:
>>> x = [2]
>>> y = x + [4]

After these operations, we have two lists: x (the list [2]) and y (the
list [2, 4]). This is because the expression "x + [4]" creates a new
list. We then bind this new list to the name 'y', and leave the name 'x'
alone.

If we then say this:
>>> x = x + [6]

We are doing much the same operation. We are creating a new list (the
list [2, 6]), and binding it to the name 'x'. The list [2], previously
bound to 'x', is no longer bound to anything, so Python frees it.

The augmented assignment, as I went over previously, attempts to modify
the list object directly. Any names bound to the object (or any other
objects that reference the object) will see the changes.

nagy wrote:
> Thanks, Kirk.
> I considered the += as only a shorthand notation for the assignment
> operator.
> Since for lists + is simply a concatetation, I am not sure it x=x+[2]
> is creating a brand
> new list. Could you refer me to any documentation on this?
> Thanks,
> Nagy

> I am not sure if this is 100% guaranteed, as I have noticed in the past
> that id's can be reused under certain circumstances. Perhaps one of the
> resident gurus can comment.
>
Since the ID tends to be the memory address (in CPython, at least),
it can be reused if all prior references to the object have gone away.

Frank Millman wrote:
> So it looks as if x += [] modifies the list in place, while x = x + []
> creates a new list.

objects can override the += operator (by defining the __iadd__ method),
and the list type maps __iadd__ to extend. other containers may treat
+= differently, but in-place behaviour is recommended by the language
reference:

An augmented assignment expression like x += 1 can be rewritten as
x = x + 1 to achieve a similar, but not exactly equal effect. In
the augmented version, x is only evaluated once. Also, when possible,
the actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.

On 2006-07-09, Fredrik Lundh <> wrote:
> Frank Millman wrote:
>
>> So it looks as if x += [] modifies the list in place, while x = x + []
>> creates a new list.
>
> objects can override the += operator (by defining the __iadd__ method),
> and the list type maps __iadd__ to extend. other containers may treat
> += differently, but in-place behaviour is recommended by the language
> reference:
>
> An augmented assignment expression like x += 1 can be rewritten as
> x = x + 1 to achieve a similar, but not exactly equal effect. In
> the augmented version, x is only evaluated once. Also, when possible,
> the actual operation is performed in-place, meaning that rather than
> creating a new object and assigning that to the target, the old object
> is modified instead.

In this case, getindex() is a rather pointless function, but it could
be an expensive one or one with side effects or even one which alters
state, so that it gives different values on subsequent calls with the
same argument.

The += version finds the object to be operated upon once, the expanded
version does it twice.

On 2006-07-10, Jim Segrave <> wrote:
> In article <>,
> Antoon Pardon <> wrote:
>>On 2006-07-09, Fredrik Lundh <> wrote:
>>> Frank Millman wrote:
>>>
>>>> So it looks as if x += [] modifies the list in place, while x = x + []
>>>> creates a new list.
>>>
>>> objects can override the += operator (by defining the __iadd__ method),
>>> and the list type maps __iadd__ to extend. other containers may treat
>>> += differently, but in-place behaviour is recommended by the language
>>> reference:
>>>
>>> An augmented assignment expression like x += 1 can be rewritten as
>>> x = x + 1 to achieve a similar, but not exactly equal effect. In
>>> the augmented version, x is only evaluated once. Also, when possible,
>>> the actual operation is performed in-place, meaning that rather than
>>> creating a new object and assigning that to the target, the old object
>>> is modified instead.
>>
>>What does it mean that x is only evaluated once. I have an avltree module,
>>with an interface much like a directory. So I added print statements to
>>__setitem__ and __getitem__ and then tried the following code.
>>
>>>>> from avltree import Tree
>>>>> t=Tree()
>>>>> t['a'] = 1
>>__setitem__, key = a
>>>>> t['a']
>>__getitem__, key = a
>>1
>>>>> t['a'] = t['a'] + 1
>>__getitem__, key = a
>>__setitem__, key = a
>>>>> t['a'] += 1
>>__getitem__, key = a
>>__setitem__, key = a
>>>>> t['b'] = []
>>__setitem__, key = b
>>>>> t['b'] = t['b'] + [1]
>>__getitem__, key = b
>>__setitem__, key = b
>>>>> t['b'] += [2]
>>__getitem__, key = b
>>__setitem__, key = b
>>
>>So to me it seems that when we substitute t['a'] or t['b'] for x,
>>x is evaluated twice with the augmented version, just like it
>>is with the not augmented version.
>
> $ cat x.py
>
> def getindex(ind = 0):
> print 'getindex() called'
> return ind
>
> a = [0, 1, 2, 3, 4, 5]
> a[getindex(0)] = a[getindex(0)] + 17
> print a
> a[getindex(1)] += 22
> print a
>
> $ python x.py
> getindex() called
> getindex() called
> [17, 1, 2, 3, 4, 5]
> getindex() called
> [17, 23, 2, 3, 4, 5]
>
> In this case, getindex() is a rather pointless function, but it could
> be an expensive one or one with side effects or even one which alters
> state, so that it gives different values on subsequent calls with the
> same argument.
>
> The += version finds the object to be operated upon once, the expanded
> version does it twice.

I disagree. The += version only evaluates the index once, but still has
to find the object twice.

Let us start with the following:

t['b'] = []

Now with x being evaluated once, I would expect that

t[getindex('b')] += [1]

would be equivallent to:

_key = getindex('b')
_lst = t[_key]
_lst += [1]

And not to the following:

_key = getindex('b')
_lst = t[_key]
_lst += [1]
t[_key] = _lst

But as far as I can interpret what is happening from the printed lines
the second is happening and not the first. So in this example some
optimisation has happened, by calculating the key only once, but
the search for the object using that precalculated key happens twice,
once with __getitem__ and once with __setitem__.
>>> t['b'] = []
__setitem__, key = b
>>> t[getindex('b')] += [1]
getindex() called
__getitem__, key = b
__setitem__, key = b

"Antoon Pardon" <> wrote in message
news:...
> I disagree. The += version only evaluates the index once, but still has
> to find the object twice.

No it does not *have* to find the object twice and no it does *not* find
the object twice. From the viewpoint of the interpreter, the purpose of
abbreviating 'objectexpression = objectexpression op arg' as
'objectexpression op=arg' is to avoid unnecessary recalculation
> But as far as I can interpret what is happening from the printed lines

Your print get/set examples from your black-box testing miss the following
key point. While the interpreter still has to *use* the object(s) twice,
once to get and once to set, it no longer has to *calculate* the objects
twice. If we open the box and look at the compiled pycode, we get, for
example:
>>> dis(compile('x[j+k]=x[j+k]+1', '', 'single'))
1 0 LOAD_NAME 0 (x)
3 LOAD_NAME 1 (i)
6 BINARY_SUBSCR
7 LOAD_NAME 2 (j)
10 LOAD_NAME 3 (k)
13 BINARY_ADD

Both forms call binary_subscr to get the number to add to 1 and both call
store_subscr to set the result back in the list. But in the first case,
the expressions for both the source-target list and the index of the value
for the addition are duplicated and re-evaluated. In the second, the
*results* of the first evaluation are duplicated (on the stack) and saved
for the storage operation.

For understanding the detailed operation of CPython, dis is a great help

On 2006-07-10, Terry Reedy <> wrote:
>
> "Antoon Pardon" <> wrote in message
> news:...
>> I disagree. The += version only evaluates the index once, but still has
>> to find the object twice.
>
> No it does not *have* to find the object twice and no it does *not* find
> the object twice. From the viewpoint of the interpreter, the purpose of
> abbreviating 'objectexpression = objectexpression op arg' as
> 'objectexpression op=arg' is to avoid unnecessary recalculation

But is the viewpoint of the interpreter somehow relevant? IMO the
question is if the actual behaviour is compatible with what people
expect after having read the language reference. The viewpoint of
the interpreter IMO doesn't play a role in answering that question.
>> But as far as I can interpret what is happening from the printed lines
>
> Your print get/set examples from your black-box testing miss the following
> key point. While the interpreter still has to *use* the object(s) twice,
> once to get and once to set, it no longer has to *calculate* the objects
> twice.

The language reference doesn't talk about objects. And IMO you
should be carefull if you want to use the word "object" here.
In the line: "foo += 1", you can't talk about the object foo,
since foo will be bound to a different object after the assignment
than it was bound to before.

As I read the language reference the x stands for a target expression.
Now what does it mean to evaluate a target expression like col[key].
IMO it means finding the location of the item in the collection: the
bucket in the directory, the node in the tree ... grosso mode it
boils down to the call to __setitem__ or __getitem__ depending
on where the col[key] was located in the line (or if you prefer
the view from the interpreter it boils down to the BINARY_SUBSCR
and STORE_SUBSCR opcodes).

So if the language reference seems to implies that col[key] will
only be evaluated once in a line like: "col[key] += 1" I expect
only one call from __setitem__ or __getitem__ (or only one
from BINARY_SUBSCR or STORE_SUBSCR)

Now I know python doesn't behave this way, but how python
actually behave can't be used as an argument that this
is the behaviour as described by the language reference.

So my question is: suppose I write my own collector,
where __setitem__ and __getitem__ have the same side effect.
How many times should/will this side effect occur in code like
"col[key] += 1". As I read the language reference it should
happen only once, however that is not what happens. So if
the actual behaviour of python is what we want, which is
what I suspect, then the language reference should
clarify more what the supposed behaviour should be.

Now my reading of the language reference can be faulty,
but if you want to argue that, I would appreciate it
if you could explain how I have to read the language
reference in order to come to the conclusion that the
side effect in this example has to happen twice.

And even in this case would I suggest that the language
reference would better be made clearer, since I doubt
that I'm the only who will read the language reference
this way.

>>>>> Antoon Pardon <> (AP) wrote:
>AP> As I read the language reference the x stands for a target expression.
>AP> Now what does it mean to evaluate a target expression like col[key].
>AP> IMO it means finding the location of the item in the collection: the
>AP> bucket in the directory, the node in the tree ... grosso mode it
>AP> boils down to the call to __setitem__ or __getitem__ depending
>AP> on where the col[key] was located in the line (or if you prefer
>AP> the view from the interpreter it boils down to the BINARY_SUBSCR
>AP> and STORE_SUBSCR opcodes).
>AP> So if the language reference seems to implies that col[key] will
>AP> only be evaluated once in a line like: "col[key] += 1" I expect
>AP> only one call from __setitem__ or __getitem__ (or only one
>AP> from BINARY_SUBSCR or STORE_SUBSCR)

You need both the __setitem__ and the __getitem__ otherwise it won't work.
I think the "evaluated once" clause is for cases like:

f(a)[g(b)] += 1

where f and/or g have side effects. These side effects should then take
place only once. Side effects in __setitem__ and __getitem__ is not what it
is talking about.
--
Piet van Oostrum <>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email:

On 2006-07-11, Piet van Oostrum <> wrote:
>>>>>> Antoon Pardon <> (AP) wrote:
>
>>AP> As I read the language reference the x stands for a target expression.
>>AP> Now what does it mean to evaluate a target expression like col[key].
>>AP> IMO it means finding the location of the item in the collection: the
>>AP> bucket in the directory, the node in the tree ... grosso mode it
>>AP> boils down to the call to __setitem__ or __getitem__ depending
>>AP> on where the col[key] was located in the line (or if you prefer
>>AP> the view from the interpreter it boils down to the BINARY_SUBSCR
>>AP> and STORE_SUBSCR opcodes).
>
>>AP> So if the language reference seems to implies that col[key] will
>>AP> only be evaluated once in a line like: "col[key] += 1" I expect
>>AP> only one call from __setitem__ or __getitem__ (or only one
>>AP> from BINARY_SUBSCR or STORE_SUBSCR)
>
> You need both the __setitem__ and the __getitem__ otherwise it won't work.
> I think the "evaluated once" clause is for cases like:
>
> f(a)[g(b)] += 1
>
> where f and/or g have side effects. These side effects should then take
> place only once. Side effects in __setitem__ and __getitem__ is not what it
> is talking about.

Well I'll start on an possitive note and accept this. Now I'd like you
to answer some questions.

1) Do you think the langauge reference makes it clear that this is how
the reader has to understand things.

2a) In case you answer yes to question (1). Can you explain me how
I have to read the language reference in order to deduce this
is indeed the way things should be understood.

2b) In case you anser no to question (1). Do you find it unreasonable
that I ask that the language reference would be rewritten here so
that your explanation can be (more easily) deduced from it.

>>>>> Antoon Pardon <> (AP) wrote:
>AP> Well I'll start on an possitive note and accept this. Now I'd like you
>AP> to answer some questions.
>AP> 1) Do you think the langauge reference makes it clear that this is how
>AP> the reader has to understand things.

Yes.
>AP> 2a) In case you answer yes to question (1). Can you explain me how
>AP> I have to read the language reference in order to deduce this
>AP> is indeed the way things should be understood.

Just read what it says. `It is only evaluated once' is quite clear I would
say. Your problem is that you thought __setitem__ is part of evaluation,
but it isn't. It is part of assignment, while __getitem__ is part of
evaluation. See the definitions of __getitem__ and __setitem__ in the
language reference manual.

On 2006-07-14 16:07:28, Piet van Oostrum wrote:
>>AP> 2a) In case you answer yes to question (1). Can you explain me how
>>AP> I have to read the language reference in order to deduce this
>>AP> is indeed the way things should be understood.
>
> Just read what it says. `It is only evaluated once' is quite clear I would
> say. Your problem is that you thought __setitem__ is part of evaluation,
> but it isn't. It is part of assignment, while __getitem__ is part of
> evaluation. See the definitions of __getitem__ and __setitem__ in the
> language reference manual.

Sorry to butt in here... I really don't know much more about this than I
read in this thread

But wasn't stated earlier that one of the differences between a += b and a
= a + b is that a gets evaluated once in the first case and twice in the
second case? If __getitem__ was part of the evaluation (as you seem to
say), shouldn't it be called twice in the second case? It doesn't seem to
get called twice; see this snippet from an earlier message:
>>> t['a'] = t['a'] + 1
__getitem__, key = a
__setitem__, key = a
>>> t['a'] += 1
__getitem__, key = a
__setitem__, key = a

Seems like the __get/setitem__ thing has not much to do with what the
manual calls evaluation, but rather with what the name implies: setting and
getting the value of the item. And therefore, since in both the a += b case
and the a = a + b case the value of a is gotten once and set once,
__getitem__ gets called once and __setitem__ gets called once. No?

The problem with understanding augmented assignment is that it directs the
compiler and interpreter to do one or maybe two mostly invisible
optimizations. To me, the effective meaning of 'evalutating once versus
twice' is most easily seen in the byte code generated by what is, remember,
the reference implementation. What it does is what the
less-than-super-clear doc means.

I posted the difference for one expression after looking at three other
pairs. It is easy to examine such pairs in IDLE and, I presume, in other
IDEs.

>>>>> Gerhard Fiedler <> (GF) wrote:
>GF> On 2006-07-14 16:07:28, Piet van Oostrum wrote:
>AP> 2a) In case you answer yes to question (1). Can you explain me how
>AP> I have to read the language reference in order to deduce this
>AP> is indeed the way things should be understood.
>>>
>>> Just read what it says. `It is only evaluated once' is quite clear I would
>>> say. Your problem is that you thought __setitem__ is part of evaluation,
>>> but it isn't. It is part of assignment, while __getitem__ is part of
>>> evaluation. See the definitions of __getitem__ and __setitem__ in the
>>> language reference manual.
>GF> Sorry to butt in here... I really don't know much more about this than I
>GF> read in this thread
>GF> But wasn't stated earlier that one of the differences between a += b and a
>GF> = a + b is that a gets evaluated once in the first case and twice in the
>GF> second case? If __getitem__ was part of the evaluation (as you seem to
>GF> say), shouldn't it be called twice in the second case? It doesn't seem to
>GF> get called twice; see this snippet from an earlier message:
>>>>> t['a'] = t['a'] + 1
>GF> __getitem__, key = a
>GF> __setitem__, key = a
>>>>> t['a'] += 1
>GF> __getitem__, key = a
>GF> __setitem__, key = a

>GF> Seems like the __get/setitem__ thing has not much to do with what the
>GF> manual calls evaluation, but rather with what the name implies: setting and
>GF> getting the value of the item. And therefore, since in both the a += b case
>GF> and the a = a + b case the value of a is gotten once and set once,
>GF> __getitem__ gets called once and __setitem__ gets called once. No?

Yes, in both cases you get the value once, and you set the value once.

In an assignment, the lefthand side is evaluated differently from the
righthand side, of course. Because in the righthand side you need the value
of the object, but in the lefthand side you need only the 'location' (this
is not a Python term).
Therefore in the righthand side __getitem__ is part of the evaluation.
In the lefthand side when it is a, only a and i are evaluated, but then
the evaluation stops. Next the assignment is done with __setitem__.

Now if it is an augmented assignment like a+=b, It is only evaluated
once. But we need it both as a lefthand side and a righthand side. So this
means that a and i are evaluated (but only once!). For the lefthand side
this completes the evaluation. And then the results of these are used as
parameters to __getitem__ to complete the evaluation of the righthand side.
Assuming b had already been evaluated, next the assignment is performed
by calling __setitem__.

Your example above doesn't show any difference because t['a'] doesn't have
any side effects. But if you use for both t and the index a function that
prints something you will see the difference.
--
Piet van Oostrum <>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email:

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!