Comments

We're looking to base an embedded development on oe-core in the near
future, and one of the things we must have is the ability to configure
features in/out of recipes per distro.
At the moment some recipes have a configuration / set of dependencies
baked in to the recipe. For example, gstreamer has theora, ogg and
vorbis hard-coded in to it's EXTRA_OECONF and DEPENDS. Currently, you'd
have to override the DEPENDS and EXTRA_OECONF for the recipe in the
distro layer - which means moving knowledge about what EXTRA_OECONF
flags bring in which dependencies outside of the recipe. It seems that
this metadata is better contained in the recipe.
I'm proposing something along the lines of the following:
+PACKAGE_FEATURES ?= "ogg vorbis theora"
+
SRC_URI += " file://gst-plugins-base-tremor.patch"
SRC_URI[md5sum] = "2920af2b3162f3d9fbaa7fabc8ed4d38"
@@ -21,6 +23,12 @@ inherit gettext
EXTRA_OECONF += "--disable-freetypetest --disable-pango"
+python () {
+ oe_package_feature_switch("ogg", "--enable-ogg", "--disable-ogg",
"libogg", d)
+ oe_package_feature_switch("vorbis", "--enable-vorbis",
"--disable-vorbis", "libvorbis", d)
+ oe_package_feature_switch("theora", "--enable-theora",
"--disable-theora", "libtheora", d)
+}
+
do_configure_prepend() {
# This m4 file contains nastiness which conflicts with libtool 2.2.2
rm -f ${S}/m4/lib-link.m4
Then in your layer, you could configure which plugins gstreamer should
be built with like this:
PACKAGE_FEATURES_pn-gst-plugins-base = "ogg theora"
Or just filter out the package features that you don't want.
With a little extra work, it would also be possible to check the list of
package features a distro layer requests against those the recipe
actually implements. Then we could get an early warning when we pull
oe-core and a recipe has stopped implementing a feature.
Cheers,
Chris.

On Thu, 2011-06-30 at 18:10 +0100, Chris Elston wrote:
> We're looking to base an embedded development on oe-core in the near> future, and one of the things we must have is the ability to configure> features in/out of recipes per distro.> > At the moment some recipes have a configuration / set of dependencies> baked in to the recipe. For example, gstreamer has theora, ogg and> vorbis hard-coded in to it's EXTRA_OECONF and DEPENDS. Currently, you'd> have to override the DEPENDS and EXTRA_OECONF for the recipe in the> distro layer - which means moving knowledge about what EXTRA_OECONF> flags bring in which dependencies outside of the recipe. It seems that> this metadata is better contained in the recipe.> > I'm proposing something along the lines of the following:
This is actually along the lines of what I've been thinking too!
> diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass> index 9930a24..4f1c54f 100644> --- a/meta/classes/utils.bbclass> +++ b/meta/classes/utils.bbclass> @@ -84,6 +84,15 @@ def oe_system(d, cmd, **kwargs):> kwargs["shell"] = True> return oe_popen(d, cmd, **kwargs).wait()> > +def oe_package_feature_switch(feature, switch_on, switch_off,> dependencies, d):> + oeconf = d.getVar("EXTRA_OECONF", True)> + if feature in d.getVar("PACKAGE_FEATURES", True).split():> + d.setVar("EXTRA_OECONF", oeconf + " " + switch_on)> + depends = d.getVar("DEPENDS", True)> + d.setVar("DEPENDS", depends + " " + dependencies)> + else:> + d.setVar("EXTRA_OECONF", oeconf + " " + switch_off)> +> oe_soinstall() {> # Purpose: Install shared library file and> # create the necessary links> diff --git> a/meta/recipes-multimedia/gstreamer/gst-plugins-base_0.10.32.bb> b/meta/recipes-multimedia/gstreamer/gst-plugins-base_0.10.32.bb> index f81011f..6fc6405 100644> --- a/meta/recipes-multimedia/gstreamer/gst-plugins-base_0.10.32.bb> +++ b/meta/recipes-multimedia/gstreamer/gst-plugins-base_0.10.32.bb> @@ -6,10 +6,12 @@ LIC_FILES_CHKSUM => "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \> > file://COPYING.LIB;md5=55ca817ccb7d5b5b66355690e9abc605 \> > file://gst/ffmpegcolorspace/utils.c;beginline=1;endline=20;md5=9c83a200b8e597b26ca29df20fc6ecd0"> > -DEPENDS += "virtual/libx11 alsa-lib freetype gnome-vfs liboil libogg> libvorbis libxv libtheora avahi util-linux tremor"> +DEPENDS += "virtual/libx11 alsa-lib freetype gnome-vfs liboil libxv> avahi util-linux tremor"> RDEPENDS_${PN} += "gnome-vfs-plugin-file gnome-vfs-plugin-http> gnome-vfs-plugin-ftp \> gnome-vfs-plugin-sftp"> > +PACKAGE_FEATURES ?= "ogg vorbis theora" > +> SRC_URI += " file://gst-plugins-base-tremor.patch"> > SRC_URI[md5sum] = "2920af2b3162f3d9fbaa7fabc8ed4d38"> @@ -21,6 +23,12 @@ inherit gettext> > EXTRA_OECONF += "--disable-freetypetest --disable-pango"> > +python () {> + oe_package_feature_switch("ogg", "--enable-ogg", "--disable-ogg",> "libogg", d)> + oe_package_feature_switch("vorbis", "--enable-vorbis",> "--disable-vorbis", "libvorbis", d)> + oe_package_feature_switch("theora", "--enable-theora",> "--disable-theora", "libtheora", d)> +}> +> do_configure_prepend() {> # This m4 file contains nastiness which conflicts with libtool 2.2.2> rm -f ${S}/m4/lib-link.m4> > Then in your layer, you could configure which plugins gstreamer should> be built with like this:> > PACKAGE_FEATURES_pn-gst-plugins-base = "ogg theora"> > Or just filter out the package features that you don't want.> > With a little extra work, it would also be possible to check the list of> package features a distro layer requests against those the recipe> actually implements. Then we could get an early warning when we pull> oe-core and a recipe has stopped implementing a feature.
How about something like the syntax for the recipe:
PACKAGECONFIG[ogg] = "--enable-ogg, --disable-ogg, libogg"
PACKAGECONFIG[vorbis] = "--enable-vorbis, --disable-vorbis, libvorbis"
PACKAGECONFIG[theora] = "--enable-theora, --disable-theora, libtheora"
and then in base.bbclass we can simply:
for flag in (d.getVarFlags("PACKAGECONFIG") or {}):
<manipulate DEPENDS/EXTRA_OECONF>
I'm also strongly tempted to tie this into the DISTRO_FEATURES variable
instead of individual PACKAGE_FEATURES variables to make it clear how
this is intended to be used.
If we do this, we are going to need to exercise some measure of control
as someone is going to want an enable/disable option for every config
option for every package. How do we decide which make sense and which
don't?
Note that the more options you have, the more combinations there are to
test and the less testing the system likely gets. I would therefore like
to see some kind of natural pressure to only add very useful
DISTRO_FEATURES. Maybe a criteria like it affecting 5 or more recipes?
Cheers,
Richard

2011/7/1 Koen Kooi <koen@dominion.thruhere.net>
>> Op 1 jul 2011, om 10:55 heeft Frans Meulenbroeks het volgende geschreven:>> >> > Good idea.> > Personally I'd like to also bring footprint into the equation. If a> feature drags in lots of additional packages, it is interesting to make it> configurable.> > My favourite example: bluez dragging in all kind of rendering stuff> (through DEPENDS) even though the hardware functionality might not be there> (e.g. you have BT but not audio).>> Which is a great example, since that doesn't impact footprint at all, it's> an alsa *plugin* that will get produced.>>> bluez.inc:DEPENDS = "gstreamer gst-plugins-base dbus glib-2.0"
I don't think gstreamer is really needed or desired if you e.g. just want to
do some tethering over bluetooth, or in my case, connect to a BT HID, and
they do contribute to both the build time and the footprint.
Frans.

Oh by the way, and the message you might be see saying:
This message may not have been sent by: fransmeulenbroeks@gmail.com Learn
more Report phishing
This is really google not having their stuff properly configured.
I'm sending email from their web client (https://mail.google.com/....)
And yes, it is me :-)
Frans

Op 1 jul 2011, om 11:26 heeft Frans Meulenbroeks het volgende geschreven:
> > > 2011/7/1 Koen Kooi <koen@dominion.thruhere.net>> > Op 1 jul 2011, om 10:55 heeft Frans Meulenbroeks het volgende geschreven:> > >> > Good idea.> > Personally I'd like to also bring footprint into the equation. If a feature drags in lots of additional packages, it is interesting to make it configurable.> > My favourite example: bluez dragging in all kind of rendering stuff (through DEPENDS) even though the hardware functionality might not be there (e.g. you have BT but not audio).> > Which is a great example, since that doesn't impact footprint at all, it's an alsa *plugin* that will get produced.> > > bluez.inc:DEPENDS = "gstreamer gst-plugins-base dbus glib-2.0"> > I don't think gstreamer is really needed or desired if you e.g. just want to do some tethering over bluetooth, or in my case, connect to a BT HID, and they do contribute to both the build time and the footprint.
Again, a plugin, so no footprint issues.

2011/7/1 Koen Kooi <koen@dominion.thruhere.net>
>> Op 1 jul 2011, om 11:26 heeft Frans Meulenbroeks het volgende geschreven:>> >> >> > 2011/7/1 Koen Kooi <koen@dominion.thruhere.net>> >> > Op 1 jul 2011, om 10:55 heeft Frans Meulenbroeks het volgende geschreven:> >> > >> > > Good idea.> > > Personally I'd like to also bring footprint into the equation. If a> feature drags in lots of additional packages, it is interesting to make it> configurable.> > > My favourite example: bluez dragging in all kind of rendering stuff> (through DEPENDS) even though the hardware functionality might not be there> (e.g. you have BT but not audio).> >> > Which is a great example, since that doesn't impact footprint at all,> it's an alsa *plugin* that will get produced.> >> >> > bluez.inc:DEPENDS = "gstreamer gst-plugins-base dbus glib-2.0"> >> > I don't think gstreamer is really needed or desired if you e.g. just want> to do some tethering over bluetooth, or in my case, connect to a BT HID, and> they do contribute to both the build time and the footprint.>> Again, a plugin, so no footprint issues.>> Well, as far as I know if it is in DEPENDS it will implicitly end up in
RDEPENDS and will become part of the image that contains bluez, unless
special precautions are taken, so it seems to me the image will grow as
plugins do consume space.
Anyway, my current project does not use or need bluez and I use a hand
crafted image picking only the things that I need so haven't really been
bitten by this recently.
Frans

On Fri, 2011-07-01 at 12:19 +0200, Frans Meulenbroeks wrote:
> Well, as far as I know if it is in DEPENDS it will implicitly end up> in RDEPENDS
No, you've got it backwards. Items appearing in RDEPENDS are (to a
first approximation) treated as if they are also in DEPENDS, but not the
converse.
p.

> > I'm proposing something along the lines of the following:> > This is actually along the lines of what I've been thinking too!
Cool, I'm glad I'm not alone on this one :)
> How about something like the syntax for the recipe:> > PACKAGECONFIG[ogg] = "--enable-ogg, --disable-ogg, libogg"> PACKAGECONFIG[vorbis] = "--enable-vorbis, --disable-vorbis, libvorbis"> PACKAGECONFIG[theora] = "--enable-theora, --disable-theora, libtheora"> > and then in base.bbclass we can simply:> > for flag in (d.getVarFlags("PACKAGECONFIG") or {}):> <manipulate DEPENDS/EXTRA_OECONF>
This is much neater than having to have an anonymous function in the
recipe to call oe_package_feature_switch.
> I'm also strongly tempted to tie this into the DISTRO_FEATURES variable> instead of individual PACKAGE_FEATURES variables to make it clear how> this is intended to be used.
I'd imagined that there were some cases which weren't covered by things
implemented as DISTRO_FEATURES. To stick with the gstreamer example -
it's possible that the distribution might have a feature, but you don't
want gstreamer plugin support for that feature. For example if the
distro needs 'theora', but you don't want the gstreamer theora plugin
building. When you're trying to slim your embedded platform down, these
things can really make a difference. It makes sense to me to provide
consistent and intuitive ways to do this, otherwise people building
platforms on oe-core end up with fragile hacks in their own layers that
break when you pull a later oe-core.
Some worthwhile configuration can't be expressed in terms of high-level
distro features (i.e.: usb, bluetooth, alsa etc...). What about "I want
SSL support, but NOT in library foo". That case can't be covered by
distro feature 'ssl'.
> If we do this, we are going to need to exercise some measure of control> as someone is going to want an enable/disable option for every config> option for every package. How do we decide which make sense and which> don't?
How could we tell what oe users are likely to want to configure?
> Note that the more options you have, the more combinations there are to> test and the less testing the system likely gets. I would therefore like> to see some kind of natural pressure to only add very useful> DISTRO_FEATURES. Maybe a criteria like it affecting 5 or more recipes?
Agreed - this is where the default set of package features comes in. If
you're not using the default set of features specified by oe-core, then
you're on your own (similar to using a tainted kernel). This doesn't
seem like a reason to prevent configuration.
Embedded developers are going to need to configure packages for their
own application, if we provide an interface to do it, then that has to
be better than everyone inventing their own methods. It would even be
possible to implement something which detects that a layer is adding
non-standard package config, and warn about it (perhaps in the sanity
check?).
Cheers,
Chris.

<cut>
> Some worthwhile configuration can't be expressed in terms of high-level> distro features (i.e.: usb, bluetooth, alsa etc...). What about "I want> SSL support, but NOT in library foo". That case can't be covered by> distro feature 'ssl'.
see http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=2
"Declaring USE flags for individual packages"
Sometimes you want to declare a certain USE flag for one (or a couple)
of applications but not system-wide. ...
.."
>> Embedded developers are going to need to configure packages for their> own application, if we provide an interface to do it, then that has to> be better than everyone inventing their own methods.
Why reinvent the wheel? See above.
<cut>
This discussion about flags/switches is recurring now and then. It's
becoming a classic :)
Now, the detractors have argued that those flags would be a nightmare
for people packaging feeds, with no way for the package manager to
detect those different recipes with the same name.
Are we still standing there?
Regards
Andrea

On Fri, 2011-07-01 at 12:53 +0200, Andrea Adami wrote:
> Now, the detractors have argued that those flags would be a nightmare> for people packaging feeds, with no way for the package manager to> detect those different recipes with the same name.
That does indeed come up frequently but I think this objection is
misplaced. The people building the feeds just need to make sure that
their particular DISTRO is using a consistent set of flags and refrain
from changing them capriciously in places like local.conf.
Or, to look at it from another perspective, there are already plenty of
ways in which you can generate two .ipk files with the same name but
different/incompatible contents by changing the contents of your
configuration files. (For example, a DISTRO can already clobber
EXTRA_OECONF_pn-foo in any way that it wants by using an override.)
It's not at all obvious that introducing per-recipe USE flags would make
things any worse in that respect.
I think Chris's proposal is basically a good one. In one sense it's
just syntactic sugar, since (as above) it doesn't actually allow you to
do anything that you couldn't already achieve by other means, but it
certainly makes it neater.
p.

On 07/01/2011 02:41 AM, Koen Kooi wrote:
> > Op 1 jul 2011, om 11:26 heeft Frans Meulenbroeks het volgende geschreven:> >>>>>> 2011/7/1 Koen Kooi <koen@dominion.thruhere.net>>>>> Op 1 jul 2011, om 10:55 heeft Frans Meulenbroeks het volgende geschreven:>>>>>>>> Good idea.>>> Personally I'd like to also bring footprint into the equation. If a feature drags in lots of additional packages, it is interesting to make it configurable.>>> My favourite example: bluez dragging in all kind of rendering stuff (through DEPENDS) even though the hardware functionality might not be there (e.g. you have BT but not audio).>>>> Which is a great example, since that doesn't impact footprint at all, it's an alsa *plugin* that will get produced.>>>>>> bluez.inc:DEPENDS = "gstreamer gst-plugins-base dbus glib-2.0">>>> I don't think gstreamer is really needed or desired if you e.g. just want to do some tethering over bluetooth, or in my case, connect to a BT HID, and they do contribute to both the build time and the footprint.> > Again, a plugin, so no footprint issues.
I disagree. I care about the footprint of sstate/packaged-staging
files. And build time is still a concern without
sstate/packaged-staging being used/found.

Op 6 jul. 2011 om 18:53 heeft Tom Rini <tom_rini@mentor.com> het volgende geschreven:
> On 07/01/2011 02:41 AM, Koen Kooi wrote:>> >> Op 1 jul 2011, om 11:26 heeft Frans Meulenbroeks het volgende geschreven:>> >>> >>> >>> 2011/7/1 Koen Kooi <koen@dominion.thruhere.net>>>> >>> Op 1 jul 2011, om 10:55 heeft Frans Meulenbroeks het volgende geschreven:>>> >>>> >>>> Good idea.>>>> Personally I'd like to also bring footprint into the equation. If a feature drags in lots of additional packages, it is interesting to make it configurable.>>>> My favourite example: bluez dragging in all kind of rendering stuff (through DEPENDS) even though the hardware functionality might not be there (e.g. you have BT but not audio).>>> >>> Which is a great example, since that doesn't impact footprint at all, it's an alsa *plugin* that will get produced.>>> >>> >>> bluez.inc:DEPENDS = "gstreamer gst-plugins-base dbus glib-2.0">>> >>> I don't think gstreamer is really needed or desired if you e.g. just want to do some tethering over bluetooth, or in my case, connect to a BT HID, and they do contribute to both the build time and the footprint.>> >> Again, a plugin, so no footprint issues.> > I disagree. I care about the footprint of sstate/packaged-staging> files. And build time is still a concern without> sstate/packaged-staging being used/found.
I was talking about target footprint, not build footprint
> > -- > Tom Rini> Mentor Graphics Corporation> > _______________________________________________> Openembedded-core mailing list> Openembedded-core@lists.openembedded.org> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

* Tom Rini Tom Rini <tom_rini@mentor.com> [07/06/11 07:53 PM]:
> On 07/01/2011 02:41 AM, Koen Kooi wrote:> > Op 1 jul 2011, om 11:26 heeft Frans Meulenbroeks het volgende geschreven:> >> 2011/7/1 Koen Kooi <koen@dominion.thruhere.net>> >> > >> Op 1 jul 2011, om 10:55 heeft Frans Meulenbroeks het volgende geschreven:> >>> Good idea.> >>> Personally I'd like to also bring footprint into the equation. If a> >>> feature drags in lots of additional packages, it is interesting to> >>> make it configurable. My favourite example: bluez dragging in all kind> >>> of rendering stuff (through DEPENDS) even though the hardware> >>> functionality might not be there (e.g. you have BT but not audio).> >> > >> Which is a great example, since that doesn't impact footprint at all,> >> it's an alsa *plugin* that will get produced.> >> > >> > >> bluez.inc:DEPENDS = "gstreamer gst-plugins-base dbus glib-2.0"> >> > >> I don't think gstreamer is really needed or desired if you e.g. just> >> want to do some tethering over bluetooth, or in my case, connect to a> >> BT HID, and they do contribute to both the build time and the> >> footprint.> > > > Again, a plugin, so no footprint issues.> > I disagree. I care about the footprint of sstate/packaged-staging> files. And build time is still a concern without> sstate/packaged-staging being used/found.
I agree with Tom.
While I understand the concerns of others, that more configurability in
building packages makes it harder to test and support oe-core, and in the end
yocto, I still like to see more granularity when it comes to building. Sure,
disk space is mostly cheap and we often have powerfull computers to build on,
but having the ability to reduce build time and space is still something that
I'd like to see. It's not that fun if a complete rebuild of a system takes
half a day or a day to complete, when it easily could have build in 2 hours if
less "unneeded" stuff was being built.
(Yes, for my embedded systems, I often consider everything related to sound,
graphics etc as unneeded. I'm not advocating removing anything litek that, as
it definitely is usefull; but I'd like an easy way to disable things like
that).
Cheers,
Anders