Some times ago, this idea of spliting the tools which are in the distribution was raised and I wasn't sure whenever it was a good idea or not for ocamlbuild. My main con, was that I think that one reason why ocamlbuild is used is that it's distributed with ocaml.

But now, after thinking of it a little bit more, I think this con can be erased by « massive » contributions and a shorter release cycle.

I tried to do the split last week but I didn't had the time to finish it.

Glad to see you working on this! It would be best to leave it as a private fork until there is agreement from the development team that this is a direction we want a future release of OCaml to go in.

OCaml 4.02 is in feature freeze right now in preparation for branching for release, so it would probably be best to bring this up in a week or so after that branch has been created, and trunk is 4.03dev.

ACK, I don't play against the core team. I suppose hhugo will agree that if the core team is opposed to this decision we will just withdraw the ocamlbuild git repository and that will be the end of the story.

Not sure about this; I think it's a good thing that a proper build tool ships with Ocaml itself. If you move it to its own repo, people will inevitably want to use this or that external module, and you can't ship that with the compiler.

I seem to recall that I had to write Ocamlbuild's glob module in pure Ocaml to avoid a dependency on Str. And Nicolas wrote a My_unix module so that Ocamlbuild would work even with "degraded" Unix environments.

On the other hand it is true that the plugin/rule/tag system is difficult to use.

Why not fork Ocaml, work on Ocamlbuild, and use pull requests on Github?

I think ocamlbuild is a quite separate project and easily splitable. Given the fact that you don't use ocamlbuild inside OCaml sources that a good candidate to be moved away.

If you have a different project, you'll be able to follow a different release cycle (shorter hopefully).

E.g. fix the .cmxs rule and release rather than wait for all other patches on all other topics to be merged.

Given the fact that camlp4 and tk lib has been moved to their own project, I didn't think it was a problem.

On the topic of using external module and rewriting code like "glob" or "my_unix", I don't share your opinion... I would use Str and Unix and if mandatory I would enforce thing like depending on findlib. I don't have enough time to redo everything.

The main problem is that there are currently extremely few people remotely motivated to contribute to ocamlbuild's development. I count two, jpdeplaix and myself. That remains a problem whether or not ocamlbuild is split.

It's very good to be able to make releases independently of OCaml's release schedule. But on how many occasion would it have made sense, between the 4.01 and the 4.02 release? Answer: zero times. Nobody gave a damn about helping with ocamlbuild development (except jpdeplaix above) since the 4.01 release, so there was nothing to release.

Will having ocamlbuild in its own github repository encourage people to contribute? Well:
- ocaml already has a github repository, and we even monitor pull requests now, where would the added convenience be?
- camlp4 has been split into a separate github repository just before 4.01. How does being split as its own github work for camlp4? Besides Jérémie Dimino's maintenance work, there have been precisely four external contributions, three of them coming from regular contributors to the OCaml distribution itself. I can't see a wave of new contributors here.

(I also have a draft of improved OCamlbuild documentation, publicly announced 8 months ago, with a github repository. So far, besides one batch of typo fixes, no substantial contribution.)

I feel neutral about splitting ocamlbuild as a separate project: I see no reason why that would make any difference -- besides making "which build system to use?" discussions even funnier.

What about taking it the other way: is keeping ocamlbuild inside OCaml sources providing a significant benefit to the core team?

Actually, I would be motivated to work on it. Esp. fixing .cmxs incremental build which is a pain point for OASIS. I don't contribute because I always postpone it given the time frame of releases. Now you may argue that I can do it and wait but it will not account for the thousands other urgent matters that I have to process. Lets call it lack of "immediate reward", something which our brain is pretty accustomed to.

Anyway, all these are just wild guesses and I suppose the best way to see if there will be an improvement in contributors is to try.

On the topic of "which build system to use?", I think this is the usual flamewar. I stopped reading caml-list 2 years ago because of this kind of discussion. Having "ocamlbuild" as an external project will not prevent 10 new "ultimate, parallel, fast, so much better and written by me" build systems to be created in 2014.

Regardless of whether ocamlbuild is split, I'm certainly interested in trying to make OASIS's life easier. However, I have very little feedback about what your needs about OASIS would be; the little I get being in the form of occasional contributions from Jacques-Pascal (jpdeplaix), drawn out of any original OASIS context. Is your "cmxs incremental build" issue reported anywhere?

> It's very good to be able to make releases independently of OCaml's release schedule.
> But on how many occasion would it have made sense, between the 4.01 and the
> 4.02 release? Answer: zero times.

That's circular: the reason that I (for example) haven't contributed ocamlbuild bugfixes
back in the past is that I can't wait a year for the fix to show up in a release to unblock
my build. If ocamlbuild were easier to recompile separately (e.g. via an opam pin),
then working on it is a viable option. Otherwise, a Makefile or an alternative build
tool is the only way to get on with real work.

Given that ocamlbuild isn't used by the compiler itself, I think splitting it out after
this release would at least give it a fighting chance to become more usable.

Now that camlp4 is no longer in the distribution, the is no real point in keeping ocamlbuild there.
Moreover, ocamlbuild includes support for findlib, which is not part of the distribution.
So yes, I think there are very good reasons to split it, and let users improve it.

It's been more than one year that I've stated I would prefer to have camlp4, (labltk,) ocamlbuild, ocamldoc separated from other bits of the compiler.
Separated or at least that it's possible to configure, build and install them on their own. So overall, I'm in favor of that.

That said, I'm worried about how the split would be done. Part of these worries is a general feeling that doing a proper split for ocamlbuild is more difficult than for camlp4 or labltk.
Camlp4 was something that could be disabled for quite some time and there was camlp5 to prove such a tool could live outside of the compiler. Labltk was an optional component for an even longer time and there was again something (lablgtk) to prove it could live nicely outside.
Now, ocamlbuild, most libraries and programs rely on it and the situation regarding build systems is weird enough that it justifies taking extra time and thinking _before_ moving it.

For starters, I believe the split of ocamlbuild should be decided before 4.02 and that a notice should be put in the release notes and in ./configure if it is to be split for 4.03.

I don't have strong justifictions for my worries; it's mainly a feeling but a feeling of someone who's dived very deep into build systems, including OCaml's.

One thing that definitely has to be changed is with the various "config" files: myocamlbuild_config.ml and config/Makefile. They hold the result of ./configure in two different formats. The information is valuable and has to be made available in a proper format to subsequent tools. I consider this a blocking issue before ocamlbuild can be split.

I'll try to come up with the various other issues in the coming weeks but with the preparation of the new version of http://win-builds.org [^] I've clearly fallen behind the cross-compilation work.

> One thing that definitely has to be changed is with the various "config" files:
myocamlbuild_config.ml and config/Makefile. They hold the result of ./configure
in two different formats. The information is valuable and has to be made
available in a proper format to subsequent tools. I consider this a blocking
issue before ocamlbuild can be split.

This comment above is the biggest concern imho and has not been answered yet. Those of you (gildor, jpdeplaix) who want to kick ocamlbuild out, what is your opinion on this particular point?

I just had a quick look at myocamlbuild_config.ml and I think the best is to register what will be missing in 'ocamlc -config' (which means that it will help not only ocamlbuild but other build tools BTW) but we can already be in a pretty good shape with just what we have in ocamlc.

The other good point on relying on the output of 'ocamlc -config' is that we can use ocamlbuild with older version of ocaml...

I hadn't thought about this earlier on but separating ocamlbuild from this data might be quite nice for the cross-compilation support.

There is no reason to have such values hardcoded into a build system and being able to point ocamlbuild to a different configuration file at runtime both through environment variables and commandl-line switches would be a plus imho.

And now I notice how much that could overlap with what ocamlfind is able to do. Maybe the data could be read by ocamlfind and then re-used by other tools?

PS: currently I cross-compile oasis/ocamlbuild-based build systems by pointing ocamlfind to specific configuration files that are setup for cross-compilation through environment variables; this is very nice but I'd hate to add another environment variable for a config file that duplicates some of the information.

edit: the process to create this file is: configure runs, creates config/Makefile at some point later on this file is transformed into myocamlbuild_config.ml; a solution that would unify both into a nicer format be nice to have (thinking about it, it is also possible to have configure write both files at the same time painlessly so this might not be a huge concern).

hhugo, I don't know if you already did that. Maybe you have something a bit different.

Anyway, I agree with the plan.
An other possible discussion would be: Do we want to have the separated ocamlbuild compatible with older ocaml versions (older than 4.03) or do we follow the same line as camlp4 does ?

> Do we want to have the separated ocamlbuild compatible with older ocaml versions (older than 4.03) or do we follow the same line as camlp4 does ?

I'd say yes, at least for 4.02. It will make the transition much smoother and will make initiali development much easier: packages would use the version in the compiler tree for now and in a few months can switch to the external one by adding the package and configuring the compiler sources without ocamlbuild (unless I'm mistaken, that bit is in 4.02).

By « older version » I meant all versions up to 3.12.1, because the with the external ocamlbuild, the api would be the same and it would be pretty convenient for packages that uses ocamlbuild to have a good backward compatibility.

Not to tell anyone how to spend their time, but I would encourage spending some time dealing with the 4.02 breaking changes before starting yet another feature breakout in the middle of a release cycle.

For instance, it woud be very useful for the 4.02 branch right now identify packages that do not compile with camlp4 correctly (see http://github.com/avsm/opam-bulk-logs [^]), and fix upstream packages (such as OASIS) that warn due to the new Bytes module. Once all this is stable, it would be much easier to use 4.02 as a base on which to break out ocamlbuild and deal with all the subsequent fallout.

What is the current opinion on this? Is it worth bringing up the question again?

As said above, I asked the dev team, before the 4.02 release, about the idea of splitting off ocamlbuild, but there was no clear consensus. I'm now considering bringing that question to the table again, but I haven't heard for anyone willing to work on ocamlbuild matters since -- while hhugo, jpdeplaix and gildor previously expressed interest in helping with ocamlbuild's maintenance if it was moved of the repository.

I still have no strong opinion about the move, but currently I'm doing 100% of the maintenance work going in ocamlbuild. Among the people that would support a move out of the repository, are there still some that would commit some time to maintenance work? Is there any particular reason why these people wouldn't consider contributing for now? For example, a couple of bugs have been fixed between 4.02.0 and 4.02.1, and some other may still be fixable.

I will not talk as them but I think the main reason is that the bugs or the features they can implement is delayed to the next ocaml release. This is too complicated if you want to support several version of the compiler and have this feature/bug fixed. Plus, you have to wait a considerable amount of time.

I'd like opinions on an intermediate proposal/step: separate the build ocamlbuild from the build of the compiler, i.e. the compiler would have to be built and installed before ocamlbuild can be built.

The only drawback I can think of involves testing ocamlbuild when doing changes in the compiler but thanks to gasche's DESTDIR support in the compiler it should only be a matter of "make install DESTDIR=some/dir" plus setting PATH and OCAMLPATH. A couple additional steps but simple ones.

That work would be required if ocamlbuild is moved to its own repository anyway so no work lost in any case and it would simplify support for cross-compilation (both build-time and because it would ensure a single ocamlbuild executable isn't tied to the compiler used to build it) while being fairly simple to implement.