JamPlus

JamPlus 3.0 branch

(This information is deprecated. See the JamPlus 3.0 branch take two topic for the most recent updates.)

Hello.

I have recently pushed a branch called 'jamplus3'. This branch contains changes for the next major evolution of JamPlus, something I've been working on over the past year. In particular, it greatly enhances the multiple toolchain support.

I open this up for discussion. What do you think of the new toolchain support? What other features, breaking, if necessary, would you like to see?

The Jambase.jam file will auto-launch a toolchain called (on Windows) c/win32/release. The user can override the default toolchain by passing the TOOLCHAIN=c/platform/config key/value pair. For instance, to build win64 debug for the C toolchain, you would specify TOOLCHAIN=c/win64/debug on the Jam command-line.

jam TOOLCHAIN=c/win64/debug

Additionally, as a backward compatibility feature, passing PLATFORM=abc and CONFIG=defg will result in a toolchain called c/abc/defg.

jam PLATFORM=abc CONFIG=defg

A new rule, Toolchain TOOLCHAIN_SPEC, allows the current toolchain to be changed. Unlike JamPlus 2.x, the toolchains can be something other than C language stuff. The format of TOOLCHAIN_SPEC (c/win32/release) is slash-separated where the word before the first slash is the toolchain category. Everything the follows is parsed by the category's C._Toolchain rule and is category-dependent. See samples/toolchains-helloworld/Jamfile.jam for usages.

C.ToolchainSpecKeys is used as before, but it now parses a / separator to assign the same value to multiple keys. For instance, C.ToolchainSpecKeys C.PLATFORM/PLATFORM assigns the value to both C.PLATFORM and PLATFORM.

C.ToolchainHelper is used to load modules from bin/modules/toolchains/c/_helpers. C.ToolchainHelperOnce will load the specified file only once, whereas C.ToolchainHelper will load it at every request.

The majority of the files residing in the bin/modules/c-compilers/ directory have been moved to bin/modules/toolchains/c/'. The @bin/modules/c-compilers/ directory tree has been removed.

The order of toolchain file loads is as follows when Toolchain c/win32/release is called:

The category of toolchain is parsed from the front of the toolchain spec (c/win32/release). In this case, it is c, and bin/modules/toolchains/c/_helpers/c.jam is loaded.

Jam will attempt to load bin/modules/c/win32-release.jam. If that fails, Jam will attempt to load bin/modules/c/win32.jam. If that fails, Jam falls back to loading bin/modules/c/_default_.jam.

Toolchain spec keys are added for C.PLATFORM/PLATFORM and C.CONFIG/CONFIG. For the toolchain c/win32/release, the following assignments are made: C.PLATFORM=win32, PLATFORM=win32, C.CONFIG=release, CONFIG=release

Next, C.ToolchainHelper $(C.PLATFORM) is called. For c/win32/release, this loads bin/modules/c/_helpers/win32.jam.

For the C toolchain, auto detection of the compiler is performed. For c/win32/release, the latest Visual Studio version is detected. Failing that, MinGW is detected. C.COMPILER_SUITE is set to vc for a successful Visual Studio detection.

Next, C.ToolchainHelper $(C.COMPILER_SUITE)-$(C.PLATFORM)-$(C.CONFIG) is called. For c/win32/release with a successful Visual Studio detection, this loads bin/modules/c/_helpers/vc-win32-release.jam which sets up some optimization flags and #defines.

There are no examples at the moment for a toolchain category other than c, but it would be easy to create one. If you wanted a toolchain category called assets, you would create a toolchains/assets/_helpers/assets.jam file and fill in the blanks.

samples/fakeps3/toolchains/c/fakeps3.jam shows an example of handling multiple configurations and architectures within a single file.

Be aware that the Jam variable called C.ACTIVE_TOOLCHAIN has been renamed to C.ACTIVE_TOOLCHAIN_GRIST.

Instead of faking 'derived' classes of compiler hierarchies within Jam (that never worked well before), settings are now added to $(C.COMPILER_SUITE_SYMBOL) with the rule names that should be called. More than one rule name can be attached per setting; the listed rules will executed in order. The names of the settings are as follows: C.ApplicationFromObjects_CleanIntermediates, C.ApplicationFromObjects_LinkFlags, C.ApplicationFromObjects_PostBuild, C.ApplicationFromObjects_Setup, C.C++Exceptions, C.LibraryFromObjects_LibFlags, C.LinkPrebuiltLibraries, C.MultiCppCompile_PreCompile, C.MultiCppCompile_PchDeps, C.MultiCppCompile_PostCompile, C.MultiCppCompile_SetupFlags, C.RuntimeTypeHelper, C.SharedLibraryFromObjects_CleanIntermediates, C.SharedLibraryFromObjects_ExportLib, C.SharedLibraryFromObjects_LinkFlags, C.SharedLibraryFromObjects_PostBuild, C.SharedLibraryFromObjects_RegServer, C.SharedLibraryFromObjects_UnRegServer,

Replies (5)

I've only just been upgrading to jamplus-nextgen, so this will take some time to get used to.

On the plus side, I really like the idea of what you've done with the toolchains ... I've been wanting to move our asset building out of custom batch files, so that will be great ... when I can figure it out.

Moving to jamplus-nextgen, I've been confused by where to put platform-specific settings, such as C.CC and C.C++, so I'll be interested to see what the startup process looks like now.

Now ... what I don't understand here, is where to set up C.CC and C.C++.

They're set to generic settings for the compiler in "_mycompiler.jam" destroying any settings that I put in "myplatform-mycompiler.jam" ... and the logic of this flow confuses me, I just can't figure out the purpose of "_mycompiler.jam"!

The flow itself is subtly different in the MSVC-based, APPLE-based and LINUX-based toolchains, making things even more interesting.

I've compiled the latest jamplus3 from today's sources and am have a couple of initial questions/problems.

1) Intellisense is broken on win32/win64/xbox360 builds ... and I can't fix it anymore.

This is because the MSSDK_STDHDRS aren't written out with the Visual Studio workspace, and so intellisense can't find them.

I could fix this on jamplus-nextgen by modifying DumpJamTargetInfo.jam (line 144 in jamplus3) ...hdrs += $(SUBDIRHDRS) $(STDHDRS) ;needs to be hdrs += $(SUBDIRHDRS) $(STDHDRS) $(C.STDHDRS:Z=$(C.COMPILER_SUITE_SYMBOL)) ;

This doesn't work anymore because it looks like the workspace builds shortcut the compiler setup, and things like C.STDHDRS and C.COMPILER_SUITE_SYMBOL seem to never be initialized.

Can you please tell me the thinking behind this change in behavior, and how I can get around it and get the SDK paths added to the workspace again?

2) Which brings me to a question ... what is the thinking behind having both STDHDRS and C.STDHDRS?

clang and gcc both set STDHDRS, and then it's never used anywhere except in DumpJamTargetInfo.jam on line 144.

msvc sets C.STDHDRS, and it's used heavily in vc-shared.jam to do a SET INCLUDE=$(C.STDHDRS:\\J=;)before calling the compiler/linker/etc, rather than adding the include directories to the command line.

But at c.jam line 366, in C._MultiCppCompile, there'scollectHdrs += $(postHDRS) $(C.STDHDRS) ;which seems to sets the directories redundantly ... but only for multi-compile.

Is there some thinking behind these apparent inconsistencies that I'm missing?

The normal msvc setup also defines MSSDK_STDHDRS, but it's global on win32/win64, and local-only on xbox360. Is MSSDK_STDHDRS supposed to be something that I can refer to?

Jam now makes a good attempt to automatically find rules that have not been specifically loaded by the script via IncludeModule or include or a similar mechanism. It does this by splitting the rule name into its components (separated by periods) and then searching the disk for a file of the component name. Each component file is loaded and tested for the rule name.

If the script asks for C.Application, Jam loads modules/c.jam and looks for a C.Application rule. It finds the C.Application rule and executes it.

If the script calls CopyFile, Jam will look in modules/copyfile.jam for the CopyFile rule.

If the script wants to execute the rule Assets.Build.Something, the following happens:

modules/assets.jam is loaded. A test is performed for the rule Assets.Build.Something. If it is found, it is executed.

modules/assets/build.jam is loaded. A test is performed for the existence of the rule Assets.Build.Something.

modules/assets/build/something.jam is loaded. A test is performed for the existence of the rule Assets.Build.Something.

Assuming the search made it this far, the rule Assets.Build.Something does not exist. Now, Jam begins looking for a more generic rule called Assets.Build.*.

If that is not found, Assets.* is searched for.

If either is found, $(__MISSING_RULE_COMPONENTS) is filled in with the string list Assets Build Something.

As a result of this new rule searching facility, a number of the rules in the provided C modules have undergone some renames: