decided to use the “logging” module:
this way we can easily add “debug” and “info” etc. messages and it’s up to
the “client” which messages it wants to display (I’m not talking about
the messages which are displayed when “osc up” is called or something like
that)

as a part of our Google Summer of Code Project to cleanup osc our
first task was to define a new commandline user interface for osc.
The current user interface is quite “inconsistent” (with regard to
the expected arguments for different commands) and has some other
“flaws”.
Here are some examples to show some flaws of the current user
interface:

Additionally we support lots of commands and the output of﻿
“osc –help”is quite long. In order to tackle this problem we decided
to introduce “groups” with subcommands, for instance:
attribute list
attribute create…
attribute set…
…

As a result we get rid of “god commands” (commands which supported
lots of different options for different things (like osc meta)) and
the new commands are easier to use because they support less arguments
and/or options (note: this doesn’t mean we lose functionality – the
functionality is just moved to another command/command group).

The attached table is just a _proposal_ for a new commandline user
interface.

Some additional explanations:

The biggest change with regard to our current user interface is the
introduction of an url-like syntax:
For example:
osc ls api://project/package
instead of
osc ls project package

In this case “api” means that the request is issued to the default
apiurl (in most cases https://api.opensuse.org) which can be
configured in the ~/.oscrc.
To issue the “list” request to a different obs instance one can use:
osc ls https://api.somehost/project/package
or
osc ls alias://project/package
“alias” is an alias for this apiurl which can be configured in the
~/.oscrc.

If you see this url-like syntax for the first time you might think
that it makes things much more complicated (and even more to type)
but it is advantageous:

– in most cases it is obvious if a command is a remote command or
local working copy command
– this way we get rid of ambiguities:
Suppose we support the following command (“<foo>” indicate that
“foo” is an optional argument): binaries get api://project/package <repo/arch>
binaries get <repo/arch> (if $PWD is a package working copy
the project and package arguments will be read from it)

Possible invocations are:osc binaries get api://foo/bar standard/i586 # get all binaries
for this repo and archosc binaries get api://foo/bar # get all binaries for all repos
and all arches

# now suppose $PWD is a package working copy:osc binaries get standard/i586 # get all binaries for this repo
and arch (project and package are read from the working copy)

osc binaries get api://foo/bar # gets all binaries for all repos
and all arches (project is “foo” and package is “bar”)

In the latter invocation we’re still in a package working copy
but ignore it and use the project and package arguments which
were specified.

Note: in this case a special (like the url-like) syntax is required
otherwise osc is unable to distinguish between a project/package and
repo/arch argument.

Once again this is just a _proposal_ – feedback is very welcome!
(please send the feedback to the opensuse-buildservice@opensuse.org mailinglist – thread)

I’m blatantly abusing GSoC for a project that I would like to see in openSUSE but that I’ve never had time to work on. But really it’s a worthwhile thing to have: a set of Plasma widgets that users and developers can add to their workspace to make it easy to see what’s going on in OBS in the projects that matter to them. If you want to work on a fun project with cutting edge technologies such as Qt, QML, Plasma then head on over to the GSoC 2011 Ideas Page.

OK I’ll take it that there are several hands raised in the audience (I reckon I’m being overly cautious, I’m sure there are loads of hands up but as I don’t have my glasses on I can only see the first two rows). So what do we need from our lovely community to help make GSoC 2010 a success?

* We need some admins for openSUSE in GSoC 2010. This mainly involves making sure that we do everything we need to participate in GSoC; making sure students feel comfortable in the project, and push our contributors a bit to publish ideas and mentor students. Basically the GoTo contact points.

* We need people to maintain the GSoC 2010 wiki page. I have already started the GSoC 2010 page on the wiki, yes it is pretty much a copy/paste of last years but it gets the ball rolling 😉

* We need people to start thinking about ideas that students could work on. If you have a good idea, why not put it in openFATE and put it on the wiki too (with a link to the openFate entry)? That way we can utilise the voting feature of openFate and gauge how much the community would appreciate the student’s hard work.

So there’s nothing stopping you from joining in, so get to it! Oh and if you’re looking for a way to contribute to openSUSE but aren’t a coder this is a great way to get your feet wet with the community 🙂

Since the rails oauth-plugin got support for oauth 1.0a I started to migrate the frontend so that it also supports 1.0a. This was a nice exercise to learn how certain things are done with rails. Additionally I did some code cleanups, bugfixing etc.

The goal for this week is more testing, bugfixing and writing a user documentation.

Tags: gsoc Posted in Build Service | Comments Off on GSoC – summary of this week’s meeting

The task for this week was to add support to the frontend so that desktop clients like osc can add the oauth specific parameters to the http “Authorization” header. The ruby library was already able to handle this and therefore I only needed to do a very small change in our urllib2 OAuthHandler which is used by osc.

Using the Authorization header has one drawback:
– the current flow looks like the following: a client makes an unauthorized API request, the API sends back a 401 to tell the client that it needs to authenticate. Therefore the response also contains the following http header: ‘WWW-Authenticate: basic realm=”Frontend login” ‘. This indicates that the client should use basic auth to authenticate with the API. The question is how we can tell the client that it could also use oauth? Sending back something like ‘WWW-Authenticate: basic, oauth realm=”Frontend login”‘ will probably break some clients. Fortunately darix had a great idea: the client simply tells the server which auth methods it supports. This can be done by adding a new http header like ‘Accept-Authentication: OpenID; OAuth;q=0.8, digest;q=0.7, Basic;q=0.5″ ‘ to each request (q indicates which method is preferred, see other http headers like ‘Accept-Language’ for the details). If the API needs authorization it looks at this header and picks the “preferred” method from this list and sends back ‘WWW-Authenticate: <preferred_and_supported_method>, realm=”Frontend login”‘ ‘. In case the Accept-Authentication header is omitted the application’s default method is used (in our case basic auth). Another thing which needs to be discussed is how the API should behave if the client only accepts methods which aren’t supported by the API (e.g. should the API send back a 401 or 406?).

Apart from thinking about this the other task for this week(end) is to add an UI for managing oauth tokens etc. The first part of this task is to decide which tasks the UI should support (like revoking tokens, authorize tokens etc.).

The next meeting will be on monday to discuss the first results.

Tags: gsoc Posted in Build Service | Comments Off on GSoC – summary of this week’s meeting

The goals for the last week were to implement oauth support into osc and add something like a “ttl” so that an access token expires after some time.

In order to implement it into osc I decided to write a simple OAuthHandler class which can be added as an “opener” to urllib2. So it should be possible to add custom “openers” for other protocols (but the interface might change again).

The next action item was to add a ttl for an access token. In fact this was just a “one-liner” (apart from a small migration script). I’m really impressed how easy it was to do this with rails.

One note about the osc integration:
At the moment osc sends all required authentification stuff (e.g. oauth_token etc.) via url parameters: http://0.0.0.0:3000/source/home:Admin?oauth_consumer_key=<key>&oauth_signature_method=HMAC-SHA1… because we cannot use POST requests. It might be “nicer” to add this kind of parameters to the http header – so our plan is to use the standard http authorization or www-authenticate headers (see also here).

Action item for the next week:

add support to the frontend so that it can handle oauth via the authorization header.

Tags: gsoc Posted in Build Service | Comments Off on GSoC – summary of this week’s meeting

This weeks topic was the integration of the cross-compilation mode into the build environment. But it’s more than just a cross-toolchain – it’s a speed-boost for our ARM build environment. As of today, the source is deployed in the repository Base:build:arm:cross. It’s not fully bootstrapped because of the current high load and the upcoming downtime – so watch out for changes there and in Base:build:arm.

But what are these “speedup’s” ? First, you’ve to know that in our build environment the ARM binaries are executed through an emulation-layer. This works on the cost of speed. The goal is now, to exchange some key parts in a transparent manner with native x86 binaries: no emulation, no slowdown. Sounds reasonable, but is it easily possible ?
I had to take care not to mix stuff too much because the environment would break. But now I’ve to say: WOW, this worked incredibly well 😉 .

The distinctive feature of our approach in comparison to usual cross-build environments is that we use the best of native environment emulation and the speed of cross-compilation. Because of this combination we don’t have to patch the individual packages to make them cross-compilation ready. This is a new way of cross-compiling suitable also for large number of packages. A detailed overview about the different crossbuild types can be found on this page.
Another feature to note is that the exchanged binaries (replacing ARM with x86 in the build environment) also don’t need heavy patching and there’s no need to compile them as static binaries. All of them are normal distribution packages.

A switch in the project enables/disables the new features. With the new changes in place, the speed could be vastly increased. Some figures:
* package rpm
* package glibc w/o locales

Build time in minutes

x86 native

armv5tel native

armv5tel cross

factor native

factor cross

rpm

8

107

17

13,38

2,13

glibc

33

505

63

15,3

1,91

Thats a drop from about x15 to x2 in comparison to the native x86 build-time !! See it yourself when the “crosscompiled” repo in Base:build:arm is up and running.

In the last days I had a closer look at the oauth rails plugin which requires some methods from the restful_authentification module. As the obs frontend doesn’t use this module we need to provide our own implementations of these methods. Fortunately it only uses a handful of methods (like authorized?, login_required, logged_in?, current_user etc.) so it shouldn’t be too hard to get it working without the restful_authentication module.

Another thing on my todo list was to look for possible workarounds for the session fixation attack. According to this thread it’ll be fixed in a new revision of the oauth spec. So after the user grants access to a specific application the oauth provider redirects the user to a callback url (if it’s specified by the consumer). Additionally it adds a parameter to this url (called oauth_verifier) which has an unpredictable value – so an attacker has no chance to “take over the session” (this is just a short summary – for more details have a look at the spec).

Last but not least I finished the test application and played around with it.

TODO:

start integrating oauth into the frontend

play around with the python library

Btw. my mentor pointed me to an interesting railscast about authlogic – it gives a great overview about this module.

Tags: gsoc Posted in Build Service | Comments Off on GSoC – summary of this week’s meeting