Repository use

Policies about Repository Use

These topics have been discussed at the WR workshop in Madrid, on Nov
27-28 2012, and later refined on Dec 2013, on the wr-dev list and
private email.

This page is not specific
to White Rabbit but generic about development on ohwr.org (both
software and gateware). Please link it from other projects when
it makes sense, its location won't change.

The Resources and the Problem

Unfortunately, the engine that drives ohwr.org only supports one
repository per project. This means that concurrent development
must work on a single repository, where stable releases and unclean
tests coexist.

To be able to freely experiment, people needs to change
history. For example to fix a bug in an already-committed patch,
or to change a commit message to make it more useful to others.

On the other hand, we need persistent commit identifiers, especially
when a project is used as a "git submodule" of another project. If
one commit is used as a submodule, we cannot risk losing it.

The Tools

Development (i.e., volatile commit identifiers) may just happen
on other repositories, outside ohwr. This means github.com,
gitorious.com or one's own machine with ssh access for mates. However,
the overhead for users is not considered worth the effort.

Therefore, we can count on the following tools to implement a
sensible policy. Most users should already know them:

Named branches. Different names have different meanings: people
should know that names like "master" or "stable-2012-10" are
different
from "devel-rubi-121101" or "testing-new-pll".

Renames. Renaming a branch doesn't change any commit identifier.

Tags. If a commit is pinned by a tag, it won't disappear until the
tag
is removed. The tag can be like "v2.1" or "projectname-v2.1". The
latter convention allows to tag external projects when needed
(for example, the fine-delay projects used tags in spec-sw and
zio to pin down the commits it relied on).

Submodules. A submodule makes the supermodule refer to a
specific commit of another project. You must ensure that that
very commit must remain available in the future.

Merges. When a merge happens, the commit identifiers of both
branches
will remain.

Rebases. A rebase, interactive or not, changes all commit
identifiers. Previous numbers are lost, and you can't go back
to that specific checkout. This happens even if you only rephrase
a commit message.

The Rules

Based on the above, we defined a set of rules for best collaboration
between developers. We identified two different set of repositories:
the ones that are fast moving, where different people concurrently
commit serious code and users rely on the master branch, and "slower"
repositories, where we make official releases any now and then, and
introduction of new features is somewhat slower.

The two set of rules are not conflicting; the main difference is
the role of the "proposed_master" branch.

Rules for "fast" repositories (e.g.: wr-cores, general-cores).

All developers work against a branch called "proposed_master".

New features are developed either in a local proposed_master or in
a
branch and only pushed to ohwr/proposed_master when they are
done+tested.

The repository maintainer moves proposed_master to master when it
seems in a good state.

When a release happens, a branch of master is created by the
repository maintainer at that time.

The repository maintainer backports critical bug-fixes from
proposed_master to the released branch.

Rules for "slower" repositories (e.g. wr-switch-sw, spec-sw).

Master is holy for normal developers: only the maintainer should
push
to master, or approve others to push specific commits.

Master is holy for the maintainer: no rebase is ever
permitted. Everybody can base their development or fetch a
submodule from a commit that is an ancestor of master. (I'd add
a special exception, allowing the maintainer to change master
right
after a wrong push, like giving an half-an-hour window; this means
it's
safe to base on commit that are at least a little old).

All development must happen in private branches.

Before requesting a merge to master, consider rebasing to the
current tip of master. This is a controversial point, and
different
people have strong and differing ideas. Please consider both
options
and choose what to follow case by case.

Git submodules must only refer to master commits, tags or
branches of stable forks (e.g., a backport).

Nobody can remove branches or tags by other developers, unless
authorized by said developer.

If your branch or tag needs to be persistent, because it is a
submodule of something else, name it clearly as stable or
important stuff. This will save you from being asked by others
to remove them. You can rename a branch without losing
information or commits.

Whenever a binary is shipped (software or vhdl), it should
claim which commit it was compiled from -- for example, for
software we can use "git describe" to get a useful string.
Binaries
from uncommited code or non-precious commits should be only used
temporarily for experimenting.

If your repository refers to external binaries, make sure the name
of those binaries is not generic. If later releases of your
repository
will refer to different binaries, you can't just overwrite them,
or
users running the previous version will get in trouble.

Keeping backups

I (Alessandro) strongly suggest to keep a backup of your
development on your personal machines, so any rebase made by
others won't lose your commits (such rebases are errors, for
sure, but they still may happen). See
Local-GIT-Backups
for details.

Thanks

Thanks to all WR developers (too many at this point to list),
who helped reaching agreement, after we faced the unavoidable issues
of imperfect commits and rebases.