Change Log

Contributing

Supported build systems

Koji

Koji is the software that builds RPM packages from source tarballs and
SPEC files. It uses its own Mock to create chroot environments to
perform builds.

MBS comes with its own koji.conf config file which allows configuring
for your custom Koji instance(s).

Mock

Mock is a simple program that will build source RPMs inside a chroot. It
doesn't do anything terribly fancy other than populate a chroot with the
contents specified by a configuration file, then build any input SRPM(s)
in that chroot.

Client tooling

mbs-manager

There are subcommands for running the MBS server, performing database
migrations, generating certificates and submitting local module
builds. For more info, there's an existing help available.

Client-side API

The MBS implements a RESTful interface for module build submission and state
querying. Not all REST methods are supported. See below for details. For client
tooling which utilizes the API, please refer to Client tooling section.

The response, in case of a successful submission, would include the task ID.

HTTP 201 Created

{
"id": 42,
"state": "wait",
...
}

Options:

buildrequire_overrides - the buildrequires to override the modulemd with. The overrides must
be to existing buildrequires on the modulemd. The expected format is
{'platform': ['f28', 'f29']}.

require_overrides - the requires to override the modulemd with. The overrides must be to
existing requires on the modulemd. The expected format is {'platform': ['f28', 'f29']}.

scratch - a boolean indicating if a scratch module build should be performed.
Only allowed to be True if the MBS setting MODULES_ALLOW_SCRATCH is True.

yaml - a string of the input file when submitting a YAML modulemd file directly in a
multipart/form-data request. Only allowed if scratch is True or if the MBS
setting YAML_SUBMIT_ALLOWED is True. The basename of the file will be used as
the module name.

modulemd - a string for submitting a YAML modulemd file as a parameter in the JSON data as
an alternative to sending it in a multipart/form-data request. Only allowed if
scratch is True or if the MBS setting YAML_SUBMIT_ALLOWED is True.

module_name - a string to use as the module name if a scratch build is requested and the
YAML modulemd is submitted using the modulemd parameter.

srpms - an optional list of Koji upload URLs of SRPMs to include in a module scratch build.
Only allowed if scratch is True.

rebuild_strategy - a string of the desired rebuild strategy (affects what components get
rebuilt). For the available options, please look at the "Rebuild Strategies" section below.

As described in the API, the following rebuild strategies are supported in MBS:

all - all components will be rebuilt. This means that even if the components have not changed
since the previous build of the module, all components will be rebuilt and not reused.

changed-and-after - all components that have changed and those in subsequent batches will be
rebuilt. Take for example a module with two batches, and each batch has two components. If one of
the two components in the first batch is changed, the other component in the batch will be reused
while all other components in the module will be rebuilt. By default, MBS only allows this
rebuild strategy.

only-changed - all changed components will be rebuilt. This means that all components,
regardless of what happened in previous batches, will be reused if they haven't been changed.
This strategy is a compromise between all and changed-and-after.

To configure the rebuild strategies in MBS, you may configure the following options:

rebuild_strategy - a string of the rebuild strategy to use by default. This defaults to
changed-and-after.

rebuild_strategy_allow_override - a boolean that determines if a user is allowed to specify
the rebuild strategy they want to use when submitting a module build. This defaults to False.

rebuild_strategies_allowed - a list of rebuild strategies that are allowed to be used. This
only takes effect if rebuild_strategy_allow_override is set to True. This defaults to
allowing all rebuild strategies that MBS supports.

Module build state query

Once created, the client can query the current build state by requesting the
build task's URL.

GET /module-build-service/1/module-builds/1042

The response, if the task exists, would include various pieces of information
about the referenced build task.

time_modified - Zulu ISO 8601 timestamp of when the module build was last modified.

time_submitted - Zulu ISO 8601 timestamp of when the module build was submitted.

version - the module build's version.

Listing all module builds

The list of all tracked builds and their states can be obtained by
querying the "module-builds" resource.
There are a number of configurable GET parameters to change how the
module builds are displayed. These parameters are:

verbose - Shows the builds with additional detail such as the modulemd
and state trace (i.e. verbose=True). This value defaults to False.

short - Shows the builds with a minimum amount of information
(i.e. short=True). This value defaults to False.

page - Specifies which page should be displayed (e.g. page=3). This
value defaults to 1.

per_page - Specifies how many items per page should be displayed
(e.g. per_page=20). This value defaults to 10.

order_by - a database column to order the API by in ascending order. Multiple can be provided.

order_desc_by - a database column to order the API by in descending order. Multiple can be
provided. This defaults to id.

An example of querying the "module-builds" resource with the "per_page" and the "page"
parameters:

Filtering component builds

The component builds can be filtered by a variety of GET parameters. Some of these
parameters include:

batch

build_time_only - boolean e.g. "true" or "false"

format

module_id or module_build

nvr

package

ref

scmurl

state - Can be the state name or the state ID. Koji states are used
for resolving to IDs. This parameter can be given multiple times, in which
case or-ing will be used.

state_reason

tagged - boolean e.g. "true" or "false"

tagged_in_final - boolean e.g. "true" or "false"

task_id

Import module

Importing of modules is done via posting the SCM URL of a repository
which contains the generated modulemd YAML file. Name, stream, version,
context and other important information must be present in the metadata.

init

When a user first submits a module build, it enters this state. We parse the
modulemd file, learn the NVR, and create a record for the module build.

Then, we validate that the components are available, and that we can fetch
them. If this is all good, then we set the build to the 'wait' state. If
anything goes wrong, we jump immediately to the 'failed' state.

wait

Here, the scheduler picks up tasks in wait and switches to build immediately.
Eventually, we'll add throttling logic here so we don't submit too many
builds for the build system to handle.

build

The scheduler works on builds in this state. We prepare the buildroot, submit
builds for all the components, and wait for the results to come back.

done

Once all components have succeeded, we set the top-level module build to 'done'.

failed

If any of the component builds fail, then we set the top-level module
build to 'failed' also.

ready

This is a state to be set when a module is ready to be part of a
larger compose. perhaps it is set by an external service that knows
about the Grand Plan.

Bus messages

Supported messaging backends:

fedmsg - Federated Messaging with ZeroMQ

in_memory - Local/internal messaging only

amq - Apache ActiveMQ

Message Topic

The suffix for message topics concerning changes in module state is
module.state.change. Currently, it is expected that these messages are sent
from koji or module_build_service_daemon, i.e. the topic is prefixed with
*.buildsys. or *.module_build_service., respectively.

Message Body

The message body is a dictionary with these fields:

state

This is the current state of the module, corresponding with the states
described above in Module Build States.

It is possible to set these values in httpd using SetEnv,
anywhere in /etc/profile.d/ etc.

If test-runtime environment is detected,
TestConfiguration is used, otherwise...

if MODULE_BUILD_SERVICE_DEVELOPER_ENV is set to some reasonable
value, DevConfiguration is forced and config.py is used directly from the
MBS's develop instance. For more information see docs/CONTRIBUTING.rst.

Setting Up Kerberos + LDAP Authentication

MBS defaults to using OIDC as its authentication mechanism. It additionally
supports Kerberos + LDAP, where Kerberos proves the user's identity and LDAP
is used to determine the user's group membership. To configure this, the following
must be set in /etc/module-build-service/config.py:

AUTH_METHOD must be set to 'kerberos'.

KERBEROS_HTTP_HOST can override the hostname MBS will present itself as when
performing Kerberos authentication. If this is not set, Python will try to guess the
hostname of the server.

KERBEROS_KEYTAB is the path to the keytab used by MBS. If this is not set,
the environment variable KRB5_KTNAME will be used.

LDAP_URI is the URI to connect to LDAP (e.g. 'ldaps://ldap.domain.local:636'
or 'ldap://ldap.domain.local').

LDAP_GROUPS_DN is the distinguished name of the container or organizational unit where groups
are located (e.g. 'ou=groups,dc=domain,dc=local'). MBS does not search the tree below the
distinguished name specified here for security reasons because it ensures common names are
unique.

ALLOWED_GROUPS and ADMIN_GROUPS both need to declare the common name of the LDAP groups,
not the distinguished name.