It has become clear that it is important to define the roles of each individual/community when it comes to our BuildBot. Please read the excerpt below, taken from Joel Brobecker's e-mail message at https://sourceware.org/ml/gdb-patches/2017-12/msg00417.html:

{{{The GDB project being a volunteer-based community-driven project,a port, and the associated resources, can only survive if there isa group of people with time and motivation to match both the portand the available resources. When resources are put in place forthose platforms, and those resources fail to be useful, the peopleto turn to for answers are those who pushed for them to be putin place. Asking the community in general to be responsible doesnot seem like an attainable goal, nor is it attractive to me.

That is not to say that the community is entirely care free either.I think we see nearly every day people cooperating on subjectsthat do not directly matter to them, and as far as I am concernedgoing well beyond the call of duty. But there has to be a reasonablelimit to what we ask of people to help for other platforms.Contributing hardware is not a fire-and-forget thing. For the issueswith the buildbots, there has to be someone who has a stake inthose platforms, and willing to drive the effort in getting thoseback in shape. These are the people we should turn to, not theperson who provides the service for those who want to. And if thereare none and the resource becomes a nuisance rather than a help,then the logical answer is to remove it.

If anything, what this discussion leads me to, is that the lessonslearned here is that we should be clearer about an identified groupwho would be responsible for what before a new buildbot is being putin place, and also about what level of "service" can be expectedfrom the people maintaining the infrastructure, vs what concernsthe platforms the buildbots build and test.}}}

In other words:

* Please '''DO NOT''' assume that contributing a machine to the GDB BuildBot project means that the GDB community will automatically be responsible for maintaining that specific port in good shape. * If you care about '''XYZ''' and contribute a '''XYZ''' machine/system to the GDB BuildBot, please keep in mind that the GDB community and the GDB BuildBot maintainer reserve the right to disable the e-mail notifications generated by the '''XYZ''' builders if they are not reliable. * Do not expect the GDB BuildBot maintainer to keep track of all the failures happening when building GDB on your system. * Do not expect the GDB community or the GDB BuildBot maintainer to fix the problems happening when building GDB on your system. We will help you investigate the issue whenever we can, though.

== TL;DR ==

The web interface for the GDB BuildBot is https://gdb-build.sergiodj.net/. Specifically, you may be interested in going directly to the [[http://gdb-build.sergiodj.net/waterfall|Waterfall page]], which displays in a nice manner the builds.

The URL for connecting your buildslave to the GDB BuildBot is '''gdb-build.sergiodj.net:16123'''. You must contact us first in order to obtain permission to do that (and a password); send a message to <gdb@sourceware.org> '''and''' put <sergiodj@redhat.com> in the Cc.

The interface for viewing the results of the builds is https://gdb-build.sergiodj.net/results .

Regression reports are sent to the [[https://sourceware.org/ml/gdb-testers/|gdb-testers]] mailing list.

* '''BuildBot''': This is the name of the project we use to test GDB (see [[more info here|http://buildbot.net/]]).

* '''buildmaster''': BuildBot has a '''master''' node, which is responsible for deciding which tasks each slave will perform, collecting the results of each task, and providing the interface for viewing the results.

* '''buildslave'''': This is the '''slave''' node of BuildBot. Slaves are "dumb", in the sense that they only execute things when master tells them to, and they usually do not have any knowledge of the entire task being performed. A typical configuration for BuildBot is to have one buildmaster, and one or more buildslaves connected to it. This is what GDB does.

* '''build step''': A '''step''' is what the name says: a step in the build process. For example, checking out the source code of the project that will be tested is a step.

* '''build factory''': A '''factory''' is like a recipe of how to perform a build. It can contain instructions for how to prepare the build directory, how to checkout the sources, how to compile them, how to test them, and how to compare the test results. A build factory is composed by one or more build steps.

* '''builder''': A '''builder''' is an instance of a factory. It is the build process itself. For example, one possible builder for GDB is '''Fedora-x86_64-native-gdbserver-m64''', which is responsible for building and testing a 64-bit GDB on a x86_64 machine, and testing the '''native-gdbserver''' there.

* '''BuildBot''': This is the name of the project we use to test GDB (see [[http://buildbot.net/|more info here]]).

* '''buildmaster''': BuildBot has a '''master''' node, which is responsible for deciding which tasks each slave will perform, collecting the results of each task, and providing the interface for viewing the results.

* '''buildslave'''': This is the '''slave''' node of BuildBot. Slaves are "dumb", in the sense that they only execute things when master tells them to, and they usually do not have any knowledge of the entire task being performed. A typical configuration for BuildBot is to have one buildmaster, and one or more buildslaves connected to it. This is what GDB does.

* '''build step''': A '''step''' is what the name says: a step in the build process. For example, checking out the source code of the project that will be tested is a step.

* '''build factory''': A '''factory''' is like a recipe of how to perform a build. It can contain instructions for how to prepare the build directory, how to checkout the sources, how to compile them, how to test them, and how to compare the test results. A build factory is composed by one or more build steps.

* '''builder''': A '''builder''' is an instance of a factory. It is the build process itself. For example, one possible builder for GDB is '''Fedora-x86_64-native-gdbserver-m64''', which is responsible for building and testing a 64-bit GDB on a x86_64 machine, and testing the '''native-gdbserver''' there.== How to view previous build logs ==

All the logs for all builds '''in the last 6 months''' are stored at https://gdb-build.sergiodj.net/results/. As you can see when you enter the page, the logs are organized by builder, then by the first two letters of the commit hash, and then by the commit hash itself. So, for example, if you wanted to see the build logs generated by the '''Fedora-x864_64-m64''' builder for a fictitious commit {{{0a1b2c3d4e}}}, you would have to go to https://gdb-build.sergiodj.net/results/Fedora-x864_64-m64/0a/0a1b2c3d4e.

The logs are compressed using {{{xz}}} in order to save space, so you have to uncompress before viewing them. A direct way to look at the {{{gdb.log}}} of the commit mentioned above, for example, is:

We also store logs for '''try builds''' (see below for more information on '''try builds'''). To access them, go to the {{{try}}} directory inside the builder directory. For example, https://gdb-build.sergiodj.net/results/Fedora-x864_64-m64/try/0a/0a1b2c3d4e.

* The {{{name}}} of the builder. Please pick a descriptive name; for example, if the builder will build and test native-gdbserver 32-bit on Fedora, with a 64-bit GDB, your builder should be called {{{Fedora-x86_64-native-gdbserver-m32}}} (actually, this is the name of an existing builder). Do not use whitespaces or special characters.

* The {{{type}}} of the builder. This is one of the most important fields. It will be used as the prefix for the {{{RunTestGDB*}}} set of classes, in order to determine how to build and test GDB. Using the same example as above, the {{{type}}} would be {{{NativeGDBServer_c64t32}}}. The {{{c64t32}}} part means that GDB will be compiled as a 64-bit binary, but the tests will be run in 32-bit mode (i.e., {{{RUNTESTFLAGS='--target_board=native-gdbserver/-m32'}}}).

* The {{{builddir}}}, which is the name of the directory that will be created on the buildmaster to store information about this build. Please use the name of the builder as the builddir, but with lowercase chars.

* The {{{slavenames}}} property, which is a list of buildslaves that can perform this build. It is extremely important to include only buildslaves capable of building for the specified architecture.

This is what a new addittion should look like:

* The {{{name}}} of the builder. Please pick a descriptive name; for example, if the builder will build and test native-gdbserver 32-bit on Fedora, with a 64-bit GDB, your builder should be called {{{Fedora-x86_64-native-gdbserver-m32}}} (actually, this is the name of an existing builder). Do not use whitespaces or special characters.

* The {{{type}}} of the builder. This is one of the most important fields. It will be used as the prefix for the {{{RunTestGDB*}}} set of classes, in order to determine how to build and test GDB. Using the same example as above, the {{{type}}} would be {{{NativeGDBServer_c64t32}}}. The {{{c64t32}}} part means that GDB will be compiled as a 64-bit binary, but the tests will be run in 32-bit mode (i.e., {{{RUNTESTFLAGS='--target_board=native-gdbserver/-m32'}}}).

* The {{{builddir}}}, which is the name of the directory that will be created on the buildmaster to store information about this build. Please use the name of the builder as the builddir, but with lowercase chars.

* The {{{slavenames}}} property, which is a list of buildslaves that can perform this build. It is extremely important to include only buildslaves capable of building for the specified architecture.This is what a new addittion should look like:

If this doesn't work, you will have to find the specific version on[[http://trac.buildbot.net/wiki/DownloadInstall|BuildBot's website]].If you are running Windows, take a look at[[http://trac.buildbot.net/wiki/RunningBuildbotOnWindows|BuildBot'sinstructions]].

{{{pip install buildbot-slave==0.8.14}}}

/!\ You may need to install the Python development files in order to build {{{buildbot-buildslave}}}.

If this doesn't work, you will have to find the specific version on [[http://trac.buildbot.net/wiki/DownloadInstall|BuildBot's website]]. If you are running Windows, take a look at [[http://trac.buildbot.net/wiki/RunningBuildbotOnWindows|BuildBot's instructions]].

==== Fedora-specific instructions ====

You will need to install the dependencies to compile and test GDB. For Fedora systems:

/!\ Please make sure to '''re-run''' this command whenever you upgrade your Fedora! New dependencies can be added to GDB, and it is important to keep up-to-date with them.

Do not forget to disable the {{{abrt-ccpp}}} service on Fedora. It will make ABRT be invoked every time a coredump is generate, which consumes time and is totally unnecessary. To disable it, do (as root):

/!\ Please make sure to '''re-run''' this command whenever you upgrade your Debian! New dependencies can be added to GDB, and it is important to keep up-to-date with them.

==== General instructions ====

/!\ Because of the {{{make TAGS}}} step, you will also have to install {{{emacs}}}.

/!\ If your machine will not test 32-bit packages, you do not need to install the 32-bit (i.e., .i686) versions of the packages above.

/!\ It has been [[https://sourceware.org/ml/gdb-patches/2015-07/msg00603.html|reported]] that the tests won't run properly if the SHELL variable is not set to a valid shell (e.g. /sbin/nologin instead of /bin/bash). Make sure that the user running the buildslave has a valid shell or override the SHELL environment variable somehow.

{{{URL:PORT}}} is the address of the buildmaster '''TO BE DEFINED''',and {{{NAME}}} is the name of the buildslave.

{{{URL:PORT}}} is the address of the buildmaster,and {{{NAME}}} is the name of the buildslave. The {{{PASSWORD}}} needs to be obtained from the BuildBot admin.

==== Adding information about yourself ====

You must add your contact information on your buildslave. To do that, edit the file named {{{DIR/info/admin}}} (assuming that {{{DIR}}} is the name of your buildslave, as per the example above) and add your name and e-mail address there:

{{{$ cat DIR/info/adminMy Name <my@email.com>}}}

==== Adding information about the machine ====

You should also edit the file {{{DIR/info/host}}} and include information about the machine that is running the buildslave. This is very useful when a failure happens and someone is trying to reproduce it, because knowing the versions of the Linux kernel, g++ and other tools can help narrow down the problem. In order to do that, it is recommended that you put '''at least''' the following information in your {{{DIR/info/host}}} file:

{{{<Distro name and release><Linux kernel information (from uname)>

<Version of g++><Version of binutils>}}}

To see an example of how this information is displayed, see the "Slave information" section here: https://gdb-build.sergiodj.net/buildslaves/fedora-x86-64-1

A good idea is to create a script that updates this information every time the system reboots, because that is usually a sign that some packages have been upgraded. In other words, it's your responsibility to keep this information up-to-date on your buildslave :-).

some special cronjob to restart this on every reboot. An example would be:

{{{$ crontab -l# restart buildslaves on every reboot

@reboot buildslave restart NAME}}}

== Try Jobs ==

The Try scheduler is now configured on our BuildBot. What this means is that adeveloper will be able to upload a patch to our BuildBot, and this patchwill then be tested against a set of builders we have there.

=== Concepts ===

As you know, our BuildBot master runs on my (Sergio) personal server, and we haveseveral "slaves" running on different locations. The master doesn't doany builds itself, but rather controls all the details about the buildsthat need to be performed. The slaves do not take any decision bythemselves; they just do what the master tells them to do.

On top of the slaves, there is an abstraction called "builder". Thebuilder is basically a recipe of how to do a build. This includes thecompilation flags used, environment variables, and testsuite options. Aslave can have multiple builders; a builder can also be assigned tomultiple slaves (for example, when we have more than one slave with thesame hardware configuration, as is the case of the Fedora x86_64slaves).

Then, we have the schedulers. They are responsible for monitoringevents and triggering builds. Our main scheduler monitors ourrepository, and triggers a full build on all builders whenever a newcommit has been pushed. We also have "cron job" schedulers, used totrigger racy testcase detection builds every week at a certain time.And now, we have the "try job" scheduler, responsible for triggering abuild on a set of builders when a developer has submitted a patch fortesting.

=== Build priority ===

Before the try scheduler, builds were basically treated in a "firstcome, first served" basis. No special priority was assigned to a build.

Now, I decided to try a new algorithm. Considering that a developerusually want to test patches when they are almost ready for submission,it would be really annoying if she had to wait in line until her buildis processed; it is not rare to see some builders having 10 or morebuilds in the queue. So, my decision was to "move" the try builds tothe top of the queue. This will obviously delay the testing of theofficial repository, but I am hoping that the impact will not be verybig. If it is, then this decision will have to be revisited.

=== Security ===

There are a fewconsiderations to be made about the security of a try job. The obviousone is that a developer may upload a malicious patch to be "tested", andthere is no easy way for BuildBot to detect this. Since the BuildBotmaster does not build anything, it would be the slaves who would sufferfrom this kind of attack.

But there is another thing to consider. In order to be able to upload apatch to be tested, the developer must have some kind of access to theBuildBot master. Yes, my personal server. No, I am not happy aboutthis. I will write more on the authentication mechanism below.

So, after giving a long thought about this problem, I decided that '''Iwill only provide accounts to people who already have commit access to the GDB repository'''.

If you want access, you need to contact me and provide a proof that you have commit access (a commit made by you should be enough).

=== Authentication ===

The BuildBot try command can use SSH for authentication, so that is whatI have chosen. Basically, you will need to send me your '''public''' SSHkey; no passwords are needed/accepted. You will be using a common userto log in, and will not receive a PTY (i.e., will not have shellaccess). If you ever used gitolite, then you know the drill. If not,it's pretty simple.

I will provide a configuration file below.

=== Installing buildbot ===

In order to use the new feature, you will have to install the "buildbot"package. You don't need the "slave" package! On Debian, an "apt-getinstall buildbot" should do the trick. On Fedora, "dnf installbuildbot". However, Fedora enables the BuildBot systemd service afterthe installation; I strongly recommend you disable that, unless youintend to run a BuildBot master on your machine.

=== The "buildbot try" command ===

After installing the package, the only command you need to know is"buildbot try". It will be responsible for generating a patch out ofthe git tree you're in (assuming you don't provide a patch explicitly),connecting to the server and submitting the patch to be tested.

I strongly recommend that you read the command's documentation at:

http://docs.buildbot.net/current/manual/cmdline.html#try

However, the usage is pretty basic. There are two modes: you can eitherbe in a local branch and ask "buildbot try" to automatically figure outthe patch to be tested, or you can provide the patch file explicitly.

In the first mode, "buildbot try" will generate a diff of your localbranch against your "master" branch, and will instruct the BuildBotmaster to test the patch *using the master's branch HEAD revision*.This means that if you want to generate a patch to be tested against thelatest commit of the official repository, you have to do a "git pull" onyour master branch first (and rebase your local branch, of course).

The second mode accepts a patch file via the "--diff" parameter.Originally, "buildbot try" will not specify a revision against which thepatch needs to be tested; however, I have modified that behaviour on theserver side, and the provided patch will always be tested against thelatest commit of the official repository.

=== Configuration file for "buildbot try" ===

Here is the configuration file that you need to use for the "buildbottry" command to work. This file needs to be named{{{$HOME/.buildbot/options}}}.

{{{$> cat $HOME/.buildbot/options# The method of connection to the BuildBot master.try_connect = 'ssh'

# The host of the BuildBot master.try_host = 'gdb-build.sergiodj.net'

# Our username. We will always use this username!try_username = 'gdb-buildbot'

# The jobdir on BuildBot master (i.e., the directory where "buildbot# try" will put the request files.try_jobdir = 'gdb/try_ssh_jobdir'

# The VCS we are using for the project.try_vc = 'git'

# The branch against which we will try our patches. Currently, we# only support 'master'.try_branch = 'master'

# Who am I?# NOTE: YOU NEED TO USE THE FORMAT "NAME <EMAIL>" IN THIS FIELD!try_who = 'John Smith <john@example.com>'

It is good to mention that most of these options can be overriden bycommand line options, so you can (for example) choose to test your patchagainst only a subset of the available try_builders.

=== Notifications ===

The "buildbot try" command offers a way to wait for the builds tocomplete on the command line, but this doesn't work when you're usingSSH. However, you will receive e-mails from our BuildBot notifying youabout the results of all of your try builds. The e-mails will be verysimilar to those sent to the gdb-testers mailing list.

=== Build specifics ===

As mentioned above, the try builds will almost always be performedagainst the latest commit on the official repository. They will alsorun using the very same flags that are being used for each builder(unfortunately, there is no way to change these flags on try builds).And the results will be compared against the last results that ourBuildBot has on file.

For example, if our BuildBot has just tested the commit 0abcd on acertain builder, and then you request it to test your patch on the samebuilder, BuildBot will run the full build of your patch, generate thetestsuite logs, and compare them against the testsuite logs generated bytesting 0abcd. The known XFAIL files for that specific builder willalso be taken into account, i.e., they will be discarded when comparingthe testsuite logs.

Last, but not least, your try build will generate its own testsuitelogs, which will be stored inside the builder's directory, at:

{{{https://gdb-build.sergiodj.net/results/<<BUILDER>>/try/<<FIRST TWO LETTERS OF COMMIT>>/<<COMMIT>>/<<TRY COUNT>>/}}}

The {{{TRY COUNT}}} can be obtained from the build status page. Just search for {{{try_count}}}there.

=== Available Builders ===

The list of available builders for a try build can be found at:

https://git.sergiodj.net/gdb-buildbot.git/tree/lib/config.json

Search for the entry named {{{GDB_Try_Jobdir}}}, and look at the {{{builderNames}}} list. That is the most updated list of try builders.

This wiki page describes how the GDB BuildBot works. Read it if you are interested in adding a new buildslave to the system, if you want to know who is responsible for each existing buildslave, or if you are just curious about how the BuildBot is configured.

The GDB project being a volunteer-based community-driven project,
a port, and the associated resources, can only survive if there is
a group of people with time and motivation to match both the port
and the available resources. When resources are put in place for
those platforms, and those resources fail to be useful, the people
to turn to for answers are those who pushed for them to be put
in place. Asking the community in general to be responsible does
not seem like an attainable goal, nor is it attractive to me.
That is not to say that the community is entirely care free either.
I think we see nearly every day people cooperating on subjects
that do not directly matter to them, and as far as I am concerned
going well beyond the call of duty. But there has to be a reasonable
limit to what we ask of people to help for other platforms.
Contributing hardware is not a fire-and-forget thing. For the issues
with the buildbots, there has to be someone who has a stake in
those platforms, and willing to drive the effort in getting those
back in shape. These are the people we should turn to, not the
person who provides the service for those who want to. And if there
are none and the resource becomes a nuisance rather than a help,
then the logical answer is to remove it.
If anything, what this discussion leads me to, is that the lessons
learned here is that we should be clearer about an identified group
who would be responsible for what before a new buildbot is being put
in place, and also about what level of "service" can be expected
from the people maintaining the infrastructure, vs what concerns
the platforms the buildbots build and test.

In other words:

Please DO NOT assume that contributing a machine to the GDB BuildBot project means that the GDB community will automatically be responsible for maintaining that specific port in good shape.

If you care about XYZ and contribute a XYZ machine/system to the GDB BuildBot, please keep in mind that the GDB community and the GDB BuildBot maintainer reserve the right to disable the e-mail notifications generated by the XYZ builders if they are not reliable.

Do not expect the GDB BuildBot maintainer to keep track of all the failures happening when building GDB on your system.

Do not expect the GDB community or the GDB BuildBot maintainer to fix the problems happening when building GDB on your system. We will help you investigate the issue whenever we can, though.

2. TL;DR

The URL for connecting your buildslave to the GDB BuildBot is gdb-build.sergiodj.net:16123. You must contact us first in order to obtain permission to do that (and a password); send a message to <gdb@sourceware.org> and put <sergiodj@redhat.com> in the Cc.

3. New terms

In this page, we will use a few specific terms to describe things, so it is good to understand what they mean.

BuildBot: This is the name of the project we use to test GDB (see more info here).

buildmaster: BuildBot has a master node, which is responsible for deciding which tasks each slave will perform, collecting the results of each task, and providing the interface for viewing the results.

buildslave': This is the slave node of BuildBot. Slaves are "dumb", in the sense that they only execute things when master tells them to, and they usually do not have any knowledge of the entire task being performed. A typical configuration for BuildBot is to have one buildmaster, and one or more buildslaves connected to it. This is what GDB does.

build step: A step is what the name says: a step in the build process. For example, checking out the source code of the project that will be tested is a step.

build factory: A factory is like a recipe of how to perform a build. It can contain instructions for how to prepare the build directory, how to checkout the sources, how to compile them, how to test them, and how to compare the test results. A build factory is composed by one or more build steps.

builder: A builder is an instance of a factory. It is the build process itself. For example, one possible builder for GDB is Fedora-x86_64-native-gdbserver-m64, which is responsible for building and testing a 64-bit GDB on a x86_64 machine, and testing the native-gdbserver there.

5. How to add your buildslave

If you are interested in adding a new buildslave (and possibly a new builder) to the GDB BuildBot, thank you! There are some steps you need to perform in order to make sure everything is OK.

5.1. Buildslave specs

Your machine needs to:

Be able to properly compile and test GDB. Make sure all the necessary dependencies are installed in your system.

Be able to run Python, and to run the buildbot-slave == 0.8.14 package (previous version might work, but we have not tested them).

Have a decent amount of disk space to perform the git clone, the build and the test.

It is also recommended that your machine is constantly updated (e.g., on Fedora you might consider running a cronjob to dnf updateevery day), because GDB's test results are impacted by updates on GCC, glibc, binutils, etc.

5.2. Buildslave owner responsibilities

You will be the responsible for the buildslave, which means that we may contact you if there is anything you should do (i.e., update the buildbot-slave package, restart your buildslave, etc.). Be sure to fill the contact fields on your buildslave configuration file (see below).

5.3. Contact us first

Please, contact us in order to discuss further details, and to let us know about you. The best way to contact the GDB community is to write to <gdb@sourceware.org>. Please, make sure you also include Sergio Durigan Junior in the Cc (e-mail: <sergiodj@redhat.com>), because he is the current maintainer of the GDB BuildBot.

5.4. Adding your buildslave to the configuration

You will need to patch the following files and add your buildslave configuration there:

config.json

master.cfg

On config.json, you will have to add your buildslave information on the slaves section. You should provide:

The name of your buildslave. Please pick a descriptive name. Do not use whitespaces nor special characters.

Then, you will have to edit the builders section. In this section we describe what kind of build we are interested in, and what are the buildslaves that will perform it. If you are adding a buildslave with a new architecture, then you will most probably want to create a new builder for it. However, you can also be interested in adding more buildslaves to an existing builder. In both cases, you will have to edit the following properties:

The name of the builder. Please pick a descriptive name; for example, if the builder will build and test native-gdbserver 32-bit on Fedora, with a 64-bit GDB, your builder should be called Fedora-x86_64-native-gdbserver-m32 (actually, this is the name of an existing builder). Do not use whitespaces or special characters.

The type of the builder. This is one of the most important fields. It will be used as the prefix for the RunTestGDB* set of classes, in order to determine how to build and test GDB. Using the same example as above, the type would be NativeGDBServer_c64t32. The c64t32 part means that GDB will be compiled as a 64-bit binary, but the tests will be run in 32-bit mode (i.e., RUNTESTFLAGS='--target_board=native-gdbserver/-m32').

The builddir, which is the name of the directory that will be created on the buildmaster to store information about this build. Please use the name of the builder as the builddir, but with lowercase chars.

The slavenames property, which is a list of buildslaves that can perform this build. It is extremely important to include only buildslaves capable of building for the specified architecture. This is what a new addittion should look like:

5.5.2. Debian/Ubuntu-specific instructions

Please make sure to re-run this command whenever you upgrade your Debian! New dependencies can be added to GDB, and it is important to keep up-to-date with them.

5.5.3. General instructions

Because of the make TAGS step, you will also have to install emacs.

If your machine will not test 32-bit packages, you do not need to install the 32-bit (i.e., .i686) versions of the packages above.

It has been reported that the tests won't run properly if the SHELL variable is not set to a valid shell (e.g. /sbin/nologin instead of /bin/bash). Make sure that the user running the buildslave has a valid shell or override the SHELL environment variable somehow.

It is also recommended that you create a separate user to run the buildslave. This user should not need/have root nor SSH access; it only needs to be able to build and test GDB.

For example, let's assume this new user is called buildbot. Log into its account in the system, and create a new buildslave. To do that, you are going to need three things:

The name of the buildslave (should be the same name you put in the JSON configuration file)

The address of the buildmaster, which is gdb-build.sergiodj.net:16123

The password that was provided to you

Then, you can issue the command:

$ buildslave create-slave DIR URL:PORT NAME PASSWORD

Where DIR is the directory that will be used by the buildslave to store local files (it can be the same as NAME), URL:PORT is the address of the buildmaster, and NAME is the name of the buildslave. The PASSWORD needs to be obtained from the BuildBot admin.

5.5.4. Adding information about yourself

You must add your contact information on your buildslave. To do that, edit the file named DIR/info/admin (assuming that DIR is the name of your buildslave, as per the example above) and add your name and e-mail address there:

$ cat DIR/info/admin
My Name <my@email.com>

5.5.5. Adding information about the machine

You should also edit the file DIR/info/host and include information about the machine that is running the buildslave. This is very useful when a failure happens and someone is trying to reproduce it, because knowing the versions of the Linux kernel, g++ and other tools can help narrow down the problem. In order to do that, it is recommended that you put at least the following information in your DIR/info/host file:

A good idea is to create a script that updates this information every time the system reboots, because that is usually a sign that some packages have been upgraded. In other words, it's your responsibility to keep this information up-to-date on your buildslave :-).

5.5.6. Starting the buildslave

After that, you can start the buildslave:

$ buildslave start NAME

Make sure it is started OK. It is also recommended that you create some special cronjob to restart this on every reboot. An example would be:

6. Try Jobs

The Try scheduler is now configured on our BuildBot. What this means is that a developer will be able to upload a patch to our BuildBot, and this patch will then be tested against a set of builders we have there.

6.1. Concepts

As you know, our BuildBot master runs on my (Sergio) personal server, and we have several "slaves" running on different locations. The master doesn't do any builds itself, but rather controls all the details about the builds that need to be performed. The slaves do not take any decision by themselves; they just do what the master tells them to do.

On top of the slaves, there is an abstraction called "builder". The builder is basically a recipe of how to do a build. This includes the compilation flags used, environment variables, and testsuite options. A slave can have multiple builders; a builder can also be assigned to multiple slaves (for example, when we have more than one slave with the same hardware configuration, as is the case of the Fedora x86_64 slaves).

Then, we have the schedulers. They are responsible for monitoring events and triggering builds. Our main scheduler monitors our repository, and triggers a full build on all builders whenever a new commit has been pushed. We also have "cron job" schedulers, used to trigger racy testcase detection builds every week at a certain time. And now, we have the "try job" scheduler, responsible for triggering a build on a set of builders when a developer has submitted a patch for testing.

6.2. Build priority

Before the try scheduler, builds were basically treated in a "first come, first served" basis. No special priority was assigned to a build.

Now, I decided to try a new algorithm. Considering that a developer usually want to test patches when they are almost ready for submission, it would be really annoying if she had to wait in line until her build is processed; it is not rare to see some builders having 10 or more builds in the queue. So, my decision was to "move" the try builds to the top of the queue. This will obviously delay the testing of the official repository, but I am hoping that the impact will not be very big. If it is, then this decision will have to be revisited.

6.3. Security

There are a few considerations to be made about the security of a try job. The obvious one is that a developer may upload a malicious patch to be "tested", and there is no easy way for BuildBot to detect this. Since the BuildBot master does not build anything, it would be the slaves who would suffer from this kind of attack.

But there is another thing to consider. In order to be able to upload a patch to be tested, the developer must have some kind of access to the BuildBot master. Yes, my personal server. No, I am not happy about this. I will write more on the authentication mechanism below.

So, after giving a long thought about this problem, I decided that I will only provide accounts to people who already have commit access to the GDB repository.

If you want access, you need to contact me and provide a proof that you have commit access (a commit made by you should be enough).

6.4. Authentication

The BuildBot try command can use SSH for authentication, so that is what I have chosen. Basically, you will need to send me your public SSH key; no passwords are needed/accepted. You will be using a common user to log in, and will not receive a PTY (i.e., will not have shell access). If you ever used gitolite, then you know the drill. If not, it's pretty simple.

I will provide a configuration file below.

6.5. Installing buildbot

In order to use the new feature, you will have to install the "buildbot" package. You don't need the "slave" package! On Debian, an "apt-get install buildbot" should do the trick. On Fedora, "dnf install buildbot". However, Fedora enables the BuildBot systemd service after the installation; I strongly recommend you disable that, unless you intend to run a BuildBot master on your machine.

6.6. The "buildbot try" command

After installing the package, the only command you need to know is "buildbot try". It will be responsible for generating a patch out of the git tree you're in (assuming you don't provide a patch explicitly), connecting to the server and submitting the patch to be tested.

However, the usage is pretty basic. There are two modes: you can either be in a local branch and ask "buildbot try" to automatically figure out the patch to be tested, or you can provide the patch file explicitly.

In the first mode, "buildbot try" will generate a diff of your local branch against your "master" branch, and will instruct the BuildBot master to test the patch *using the master's branch HEAD revision*. This means that if you want to generate a patch to be tested against the latest commit of the official repository, you have to do a "git pull" on your master branch first (and rebase your local branch, of course).

The second mode accepts a patch file via the "--diff" parameter. Originally, "buildbot try" will not specify a revision against which the patch needs to be tested; however, I have modified that behaviour on the server side, and the provided patch will always be tested against the latest commit of the official repository.

6.7. Configuration file for "buildbot try"

Here is the configuration file that you need to use for the "buildbot try" command to work. This file needs to be named $HOME/.buildbot/options.

It is good to mention that most of these options can be overriden by command line options, so you can (for example) choose to test your patch against only a subset of the available try_builders.

6.8. Notifications

The "buildbot try" command offers a way to wait for the builds to complete on the command line, but this doesn't work when you're using SSH. However, you will receive e-mails from our BuildBot notifying you about the results of all of your try builds. The e-mails will be very similar to those sent to the gdb-testers mailing list.

6.9. Build specifics

As mentioned above, the try builds will almost always be performed against the latest commit on the official repository. They will also run using the very same flags that are being used for each builder (unfortunately, there is no way to change these flags on try builds). And the results will be compared against the last results that our BuildBot has on file.

For example, if our BuildBot has just tested the commit 0abcd on a certain builder, and then you request it to test your patch on the same builder, BuildBot will run the full build of your patch, generate the testsuite logs, and compare them against the testsuite logs generated by testing 0abcd. The known XFAIL files for that specific builder will also be taken into account, i.e., they will be discarded when comparing the testsuite logs.

Last, but not least, your try build will generate its own testsuite logs, which will be stored inside the builder's directory, at:

https://gdb-build.sergiodj.net/results/<<BUILDER>>/try/<<FIRST TWO LETTERS OF COMMIT>>/<<COMMIT>>/<<TRY COUNT>>/

The TRY COUNT can be obtained from the build status page. Just search for try_countthere.