You have all your computing privileges and accounts, including an account on the lbne gpvm nodes (e.g. lbnegpvm01.fnal.gov).

You are working on one of the lbne gpvm nodes. Many of these should work on other machines/sites as well, if they are configured in the standard way, but there are no guarantees for that. Talk to your system/local administrator if you’re having trouble, to make sure there’s no differences in the basic setup.

You have a working knowledge of UNIX.

You are using the bash shell (most of the things for csh should be clear)

You know something about the general structure of LArSoft. If you don’t know what a fcl file is, and/or don’t know how to make/modify one, then you will need a more basic tutorial than what’s here.

You should see a list (possible long list…) of versions of the code to set up. Each line should look something like this:

“lbnecode” “v1_00_01” “Linux64bit+2.6-2.5” “debug:e4” “”

Pick which version you want to setup (probably the latest, and probably the profile version), and do something like this (with the version and qualifiers you want):

setup lbnecode v1_00_01 -q debug:e4

This will automatically setup larsoft as well, so don’t worry about anything else. Meaning, you don't need to also do setup larsoft v1_00_01 -q debug:e4. lbnecode of a particular version requires larsoft of that version, and thus it sets it up too. You can think of “lbnecode” as LBNE’s own particular implementation of larsoft. You should then be able to run your usual larsoft job:

We advise strongly that each new directory you would like to make corresponding to a new development environment you will create should only be performed once per login. So mkdir larsoft_mydev2 below should only happen after logging out and logging back in. The reason is that your $PRODUCTS variable tends to grow to include all working areas you visit per login. It can happen across logins too. This makes for many weird effects upon saying mrbsetenv. Check your $PRODUCTS when confused.

For developing your code, we recommend you use the latest LArSoft frozen release. The LArSoft frozen release is tagged and built once a week. By using the latest frozen release, you get the most up-to-date features and a stable release. We have instructions below on how to update your local release when a new LArSoft release is available.

Like above, the first thing to do is

source /grid/fermiapp/lbne/software/setup_lbne.sh

But now, things get different! First, check the larsoft releases:

ups list -aK+ larsoft

Then setup the latest version of LArSoft, e.g. (note v03_04_04 -q e6:prof is just an example, setup the latest larsoft version you find from this command. Note that the list from the ups command is NOT sorted, so the latest version is not the last one in the list)

Now, you should make a new directory where you intend to put your code (not your AFS home, as the libraries you build can get large, but somewhere beneath /lbne/app/users/USER/). Let’s say that new directory is going to be called larsoft_mydev. Then do

and then, if this is your first time building with these local packages, do

mrbsetenv

You can omit this if you have already done this, but if you at any point add a new package, you should do mrb z; mrbsetenv. Meaning, you want to redefine the CMakeList.txt files and hence the Makefiles. Once properly set up, you can build/install:

mrb i -j4

That latter part tells it to build on 4 cores, if available. Check for build errors! It may be easier to break that step above into two parts---mrb build -j4; mrb install---in order to more easily check for compilation issues.

Once you're stable and the code you're working on is well defined, and you're not git pull'ing new repositories, the development cycle can be shortened. Go make your edits in ../srcs/lbnecode/PackageA/AnalysisCode.cpp and jump back to ../build. You can now replace the onerous

Now, if you want to just build, see below. However, you will also want to develop some stuff. Before touching the code, do this in the srcs/<package> directory (e.g. src/larreco/). If you cd into that, you will then be inside a particular repository. Git commands make sense only if you cd there.

git flow feature start $USER_testFeature

where $USER is your username, and testFeature is up to you. You should now see that you are on this new branch (feature/$USER_testFeature) if you issue git branch -a. This helps you comply with the First rule: never push to master branch.. Make all of your changes, and commit as you will. You will need to create a new branch like this for every repository/package in which you are changing code! This is all done manually now, but expect nice scripts that do this for you in the future.

The reason for making your own branch and working on it is because you always need to be in a position to only commit and push a fraction of what you have. First rule: never push to master branch. You may realize some part of what you'd like to push is not ready yet. Or you may wish to suddenly fix one other totally orthogonal thing in origin/develop and push it -- a hot fix. You need to be adaptable to commit/push'ing half-measures. The easiest way to accommodate that is to be on your own new branch. Stay off your own develop branch until you know you want to push into origin develop. (See below git fetch, rebase where that's done.) It's just a good idea.

You have now defined a local branch for you to work in. The git repository of origin as yet knows nothing about this branch. We will come back to this.

We will not pretend that there is not a short cut around all this manufacture of extra branches and git flow usage. Bare git commands can be used to make quick changes. Namely, if you know you are going to fix one piece of code in, e.g., larcore/SimpleTypesAndConstants/PhysicalConstants.h maybe there's no reason to make this extra branch. You will fix develop, and that's it. Here's how to do that. Remember the First rule: never push to master branch..

In git, you have your own “repository” in your development area. You can and should make frequent commits to it. Note that this, unlike svn, won’t change the global repository that everyone uses, for two reasons:

because you are good at following best practices, you have made your own branch in which all of your changes are going into; and,

you are only making commits/changes to your own local repository.

So, suppose you have changed (or created!) my_file.cc in some srcs/<package> area. To commit this change, do

git add my_file.cc
git commit -m ‘message about the commit here; if you do not use -m, it will open a text editor and allow you to make a very long commit message’

You can add multiple files at once (git add my_file.cc my_file.h) and use just one commit message for them. Or git add my_dir, and it adds recursively everything in directory my_dir. Or simply skip the git add by doing git commit -a -m "blah blah blah".

Also incredibly helpful is the git status command, which will tell you all of the files that you have changed, new files that you have created, and any files you have staged---i.e., done git add on but not yet committed.

More gitology: Update your feature branch with updates to the repository¶

It’s likely that people will make updates to the repository (develop branch) while you are working on your feature. It’s a good idea to incorporate those regularly, and it’s a requirement that you do that before pushing in your own commits. To do that, go to the package you want to update and rebase

The first two commands will fetch whatever changes from origin (on all branches) that you've missed out on whilst working in your local area and shove them into your changes -- or rebase you to origin/develop. Then the git flow feature finish will take your feature branch and merge it back into yourdevelop branch (remember, you have your own repository!). Sometimes you'll see git pull to accomplish the first two things above. pull is roughly a combo of fetch and merge. rebase is a more sophisticated version of merge, which works along the time axis first, before merging -- I think. Now you need to push your changes in develop to the main (origin) repository:

git push origin develop

and congratulations, you've unleashed your terror on the LArSoft world!

If you are making breaking changes, e.g. you are changing code in two repositories and people need to checkout both repositories in order to compile code, it is good practice to put your changes in feature branches and ask the release manger to merge them back into develop when he/she is tagging a new release.

This would merge your feature branch into develop and delete the feature branch. You are on local develop branch now.Resolve conflicts if necessary.If you want to push your changes to remote develop, you can do "git push" now.

The above should be enough to do what you are typically used to doing on a day-to-day basis, and it’s a good place to start. Once you think you have those down, there are some further things you may want to be able to do, outlined below.

Imagine you're working in your directory in your git clone'd, pull'd version of larsim/LArG4. LArG4 is a package -- it's a top-level directory underneath a project, which makes it a package in our parlance. Say you would like to create histograms or a TTree that characterizes some truth level information. (You're somehow unhappy with the module LArG4Ana_module.cc that already does this and want to start fresh.) You thus want an ED::analyzer module.

To accomplish this you would proceed like any right-thinking physicist and cp another *Ana_module.cc file from somewhere else and pull out all the un-needed bits, leaving the key structure of the module -- the methods you must over-write to make this thing work. You would then declare, initialize, and Fill your histograms/TTrees in the right places. (You don't need to Write() them because the TFileService in ART takes care of that, as usual.)

Now just go back to your build directory and mrb z; mrbsetenv; mrb i -j4. You hopefully don't need to touch any of the files that do the building, because the drilling down to find your new .cc just does this as a feature in the CMakeLists.txt. Now, if you had added stuff that needs access to header files that were not already included in larsim/LArG4/CMakeList.txt you would need to edit that file to add them. Same if you added new functions/methods not already in Libraries in that CMakeList.txt.

Okay, instead, let's add a new directory under the existing repository, itself containing one or more modules and perhaps other .cpp and .h files. First, to get it to compile, your Makefile needs to know about it. As yet, it does not know about it. Remember, the cmake that happens in the first layer of doing an mrb b or mrb i takes your CMakeLists.txt files and creates the Makefiles. So that's what happened above, and merely adding a file in an existing directory, as we did there, did not complicate matters too much. Here, we have at least one other step. We'll need to inform the top level CMakeLists.txt that you have a new directory into which to drill down. You'll then need a CMakeLists.txt in that directory too.

Let's do this in lbnecode. Add a new add_subdirectory(MyNewDirectory) line in lbnecode/lbne/CMakeLists.txt. That is, you are directing it to drill down into lbnecode/lbne/MyNewDirectory to look for the next CMakeLists.txt file to generate the Makefile to build all the fabulous code you've put there. cp a CMakeLists.txt file that is nearby. Use the cetbuildtools macros already in there and from other CMakeLists.txt files to specify any new include directories to compile against and libraries to link against.

In fact, we don't see too common a need for this from the lay-collaborator. Imagine coming along with a whole new package of code (which itself might live in nusoft, say) for which you've written specific algorithms, e.g. Pandora. That use case is one for code that might merit a whole new repository. But, any new wire simulation or optical scintillation package or calorimetry reco module can go into an existing repository: lbnecode or larsim or larreco, correspondingly.

Use multiple install areas when you want to build flavors using different ups qualifiers (-q setup option). This method would typically be used when compiling your code using debug compiler options (-q debug:e4) vs. profiled/optimized compiler options (-q e4:prof). Each install area should be initialized using the command mrb newDev.

The -T option instructs mrb to put the build and install areas into the specified subdirectory. The -f option tells mrb that it is OK to use an existing source aras. Here are a couple of caveats about this method.

You can omit the -v and -q options if you have larsoft setup (mrb will inherit the version and qualifiers from your setup version of larsoft).

Your specified qualifiers are hardwired into the generated localProductsXXXX/setup initialization script. This initialization script is not reentrant. That is, you can not switch flavors simply by sourcing a different localProductsXXXX/setup script. You should always source localProductsXXXX/setup in a fresh shell.