Commit Message

There are long standing complaints about the fact its very difficult
to remove a portion of a variable. The immediate request is for a -=
and =- operator. The trouble is that += and =+ are "immediate"
operators and are applied straight away. Most people would expect
-= and =- to be deferred to have the effect most people desire and
therefore implementing -= and =- would just make the situation more
confusing.
This deferred operation is much more similar to the override syntax
which happens at data store finalisation. The _remove operator is
therefore in keeping with the _append and _prepend operations.
This code is loosely based on a patch from Peter Seebach although it
has been rewritten to be simpler, more efficient and avoid some
potential bugs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
bitbake/lib/bb/data_smart.py | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)

Comments

On Sat, Mar 9, 2013 at 5:47 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:
> There are long standing complaints about the fact its very difficult> to remove a portion of a variable. The immediate request is for a -=> and =- operator. The trouble is that += and =+ are "immediate"> operators and are applied straight away. Most people would expect> -= and =- to be deferred to have the effect most people desire and> therefore implementing -= and =- would just make the situation more> confusing.>> This deferred operation is much more similar to the override syntax> which happens at data store finalisation. The _remove operator is> therefore in keeping with the _append and _prepend operations.>> This code is loosely based on a patch from Peter Seebach although it> has been rewritten to be simpler, more efficient and avoid some> potential bugs.>
This is a nice idea, though I'm slightly concerned about the naive
implementation (use of str.replace). Without it being either word-based
(limits flexibility to optimize for the common case) or regex-based (more
complex), we may be greatly limiting the usefulness. I wouldn't want to try
to remove a word from DISTRO_FEATURES and end up removing part of a word,
for example.

On Saturday 09 March 2013 16:20:10 Chris Larson wrote:
> On Sat, Mar 9, 2013 at 5:47 AM, Richard Purdie <> > richard.purdie@linuxfoundation.org> wrote:> > There are long standing complaints about the fact its very difficult> > to remove a portion of a variable. The immediate request is for a -=> > and =- operator. The trouble is that += and =+ are "immediate"> > operators and are applied straight away. Most people would expect> > -= and =- to be deferred to have the effect most people desire and> > therefore implementing -= and =- would just make the situation more> > confusing.> > > > This deferred operation is much more similar to the override syntax> > which happens at data store finalisation. The _remove operator is> > therefore in keeping with the _append and _prepend operations.> > > > This code is loosely based on a patch from Peter Seebach although it> > has been rewritten to be simpler, more efficient and avoid some> > potential bugs.> > This is a nice idea, though I'm slightly concerned about the naive> implementation (use of str.replace). Without it being either word-based> (limits flexibility to optimize for the common case) or regex-based (more> complex), we may be greatly limiting the usefulness. I wouldn't want to try> to remove a word from DISTRO_FEATURES and end up removing part of a word,> for example.
I can't recall if we've discussed this before, but shouldn't we be looking at
introducing some understanding of list type variables into BitBake itself? If
BitBake knew the variable was a list it would be possible to have language
elements that understood how to behave in this situation.
(I've been thinking about this for a while. It could also help to implement
better dependencies on values of list variables, to only rebuild what's really
necessary when DISTRO_FEATURES changes for example).
Cheers,
Paul

On Sun, 10 Mar 2013 20:39:07 +0000
Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
> I can't recall if we've discussed this before, but shouldn't we be looking at > introducing some understanding of list type variables into BitBake itself? If > BitBake knew the variable was a list it would be possible to have language > elements that understood how to behave in this situation.
I rather like this. I would like to have _remove, and it may even make sense
to adopt _remove in the short term, but really I think most of the things
where we're using append/remove should probably be lists, which would also
eliminate the entire category of bugs in which it is unclear whether or not
to add a space.
Rough thoughts about implementation: Listness could be a flag.
LIST[list] = " "
LIST = "blah blah blah"
# Oh, hey, what if...
PATH[list] = ":"
or as shorthand:
LIST = [blah blah blah]
If this is done, the behavior of append/prepend is slightly altered (to
always ensure separators are added when needed), and then we could more
easily handle the remove case, too.
That said: I'd rather see a plain _remove go in than wait on a larger and
fancier design.
-s

On Wed, Mar 13, 2013 at 11:28 AM, Peter Seebach <peter.seebach@windriver.com
> wrote:
> On Sun, 10 Mar 2013 20:39:07 +0000> Paul Eggleton <paul.eggleton@linux.intel.com> wrote:>> > I can't recall if we've discussed this before, but shouldn't we be> looking at> > introducing some understanding of list type variables into BitBake> itself? If> > BitBake knew the variable was a list it would be possible to have> language> > elements that understood how to behave in this situation.>
Yeah, this gets brought up from time to time, it's something we've wanted
for a while, but nobody has taken the time to make it happen. I would love
to see some form of typing go in, and I know Richard has expressed a desire
to see this as well in the past.
I rather like this. I would like to have _remove, and it may even make sense
> to adopt _remove in the short term, but really I think most of the things> where we're using append/remove should probably be lists, which would also> eliminate the entire category of bugs in which it is unclear whether or not> to add a space.>> Rough thoughts about implementation: Listness could be a flag.>> LIST[list] = " "> LIST = "blah blah blah"> # Oh, hey, what if...> PATH[list] = ":">
We do have a flag-based typing implementation in the metadata, just not in
bitbake itself, see
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=e4921fd for the
original implementation. I particularly like the bits that automatically
convert the function signature of the type callback (or class) into a
request for additional information via flags. An added required argument to
the function = a required flag of that name, same for optional. See
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/types.py#n27
or
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/types.py#n13
for
examples of that particular implementation.
or as shorthand:
>> LIST = [blah blah blah]>
I played briefly around with a *really* preliminary prototype of something
similar to this. See https://gist.github.com/kergoth/2788410 - any unquoted
values are interpreted as json types, and the append/prepend operations are
switched to better work with non-string values. It works surprisingly well
considering how naive the implementation is, see the functional local.conf
snippet in the gist.