The Eclipse Foundation runs a Hudson continuous integration server that eclipse hosted projects can take advantage of for their builds.&nbsp; Currently this is hosted here: https://build.eclipse.org/hudson, but due to stability issues a new 'stable' instance has been created and is hosted here: https://hudson.eclipse.org/hudson . The Hudson server allows for the execution of Continuous Integration Builds, Nightly Builds, Integration Builds, Release Builds, and Testing. The 'stable' Hudson instance is maintained by the Eclipse Webmasters. The planned Hudson sandbox instance will be maintained by the eclipse community. Job creation will be limited to committers and admin rights are administered by the community as a whole.

+

The Eclipse Foundation runs a Hudson continuous integration server that Eclipse-hosted projects can take advantage of for their builds. Currently this is hosted here: https://hudson.eclipse.org/ . The Hudson server allows for the execution of Continuous Integration Builds, Nightly Builds, Integration Builds, Release Builds, and Testing. Hudson is maintained by the Eclipse Webmasters. A Hudson sandbox instance is available for those who like the bleeding edge; however, its stability may suffer due to frequent restarts, untested configurations and upgrades.

+

*Hudson "Stable" instance: https://hudson.eclipse.org/

+

*Hudson Sandbox instance: https://hudson.eclipse.org/sandbox/

+

*Hudson Sandbox instance: https://hudson.eclipse.org/perftests/

−

== Migrating to the 'new' Eclipse.org Hudson service ==

+

Note if you are looking for information on the Hudson software project itself at Eclipse then that can be found at [[Hudson-ci|hudson-ci]].

−

If you have a build that is currently running on the 'unstable' Hudson instance(hosted on https://build.eclipse.org/hudson) and you wish to move to the Hudson instance managed by the Foundation(hosted on https://hudson.eclipse.org/hudson) you will need to file a bug against community->hudson with a subject like 'Move job job-name to hudson.eclipse.org'. Webmaster will then import the job and close the bug. Please note that the stable instance does not provide the same plugins as the unstable instance. You may also need to update your job configuration to account for differences in the Hudson deployment.

+

== Hudson configuration and tools ==

−

== Hudson configuration and tools ==

+

The Hudson setup uses a combination of SLES 11 x86_64 machines for Linux slaves, Windows 7 for the Windows slave(s) and Mac OS X. These machines are behind a firewall so any outbound http(s) connections are proxied.

−

Presently the 'new' hudson setup uses 3 SLES 11 x86_64 machines (1 master and 2 slaves), with bash as the default shell. These machines are behind a firewall so any outbound http(s) connections are proxied.

+

The following global variables are set(identically across installs):

−

The following global variables are set(identically across installs):

+

*JVM_OPTS: proxy data (see "Accessing the Internet" below)

−

+

*ANT_ARGS: proxy data

−

*JVM_OPTS: proxy data (see "Accessing the Internet" below)

+

−

*ANT_ARGS: proxy data

+

*ANT_OPTS: proxy data

*ANT_OPTS: proxy data

−

Each node also has a .m2/settings.xml file with the proxy data.

+

Each node also has a .m2/settings.xml file with the proxy data.

+

+

=== Choosing the right slave ===

+

+

* '''hudson-slave1, hudson-slave2, hudson-slave4 and hudson-slave6''' - these are the main build nodes for Hudson jobs. You can specify them by name or by using the 'build2' label.

+

* '''hudson-slave5''' - this is an ia64 slave.

+

* '''hudson-slave7, hudson-slave8''' - these are ppc64 slaves located at OSUOSL.

+

* '''fastlane''' - this slave is intended for usage during a release train crunch when re-spins may require more capacity than hudson-slave1&amp;2 can provide. By default jobs should not run here.

+

* '''hudson-perf1-tests''' - this slave is used for running performance tests ONLY.

+

* '''mac-tests and windows7tests''' - these 2 slaves are meant for running UI tests for their respective OS versions. By default jobs should not run on either slave.

+

+

See also: [[Hudson server performance metrics]]

+

+

=== Server Storage ===

+

+

[[Image:Build infra layout.png|thumb|Build and Hudson storage layout]] Three tiers of storage are available for storing Workspaces, build artifacts, nightly and release builds. For optimal build performance and service availability, it is important that you use each storage device according to its intended purpose.

+

+

The image on the right illustrates the three storage tiers and their intended purpose.

−

==Tools(and locations)==

+

== Tools(and locations) ==

−

*Maven 2.2.1 (installed automatically)

+

*Maven 2.2.1 (installed automatically)

−

*Maven 3.0 alpha 5 (installed automatically)

+

*Maven 3.0 alpha 5 (installed automatically)

−

*Maven 3.0-alpha-5-local (/shared/common/apache-maven-3.0-alpha-5)

+

*Maven 3.0-alpha-5-local (/shared/common/apache-maven-3.0-alpha-5)

−

*Maven 3.0-alpha-6-local (/shared/common/apache-maven-3.0-alpha-6)

+

*Maven 3.0-alpha-6-local (/shared/common/apache-maven-3.0-alpha-6)

*Maven 3.0-Beta-1 (/shared/common/apache-maven-3.0-beta-1)

*Maven 3.0-Beta-1 (/shared/common/apache-maven-3.0-beta-1)

−

*Sun Java 5 SR 22 64bit (/shared/common/jdk-1.5.0-22.x86_64)

+

*Sun Java 5 SR 22 64bit (/shared/common/jdk-1.5.0-22.x86_64)

−

*Sun Java 5 R 16 32bit (/shared/common/jdk-1.5.0_16)

+

*Sun Java 5 R 16 32bit (/shared/common/jdk-1.5.0_16)

−

*Sun Java 5 R 22 64bit (/shared/common/jdk-1.5.0-22.x86_64)

+

*Sun Java 5 R 22 64bit (/shared/common/jdk-1.5.0-22.x86_64)

−

*Sun Java 6 R 21 32bit (/shared/common/sun-jdk1.6.0_21_i586)

+

*Sun Java 6 R 21 32bit (/shared/common/sun-jdk1.6.0_21_i586)

*Sun Java 6 R 21 64bit (/shared/common/sun-jdk1.6.0_21_x64)

*Sun Java 6 R 21 64bit (/shared/common/sun-jdk1.6.0_21_x64)

−

*Apache-ant-1.7.1 (/shared/common/apache-ant-1.7.1)

+

*Apache-ant-1.8.1 (/shared/common/apache-ant-1.8.1)

−

*Apache-ant-1.7.0 (/shared/common/apache-ant-1.7.0)

+

*Apache-ant-1.7.1 (/shared/common/apache-ant-1.7.1)

+

*Apache-ant-1.7.0 (/shared/common/apache-ant-1.7.0)

*Apache-ant-1.6.5 (/shared/common/apache-ant-1.6.5)

*Apache-ant-1.6.5 (/shared/common/apache-ant-1.6.5)

−

*Headless Buckminster 3.6 (/shared/common/buckminster-3.6)

+

*Headless Buckminster 3.6 (/shared/common/buckminster-3.6)

*Buckminster 3.6 Integration (installed automatically)

*Buckminster 3.6 Integration (installed automatically)

−

== Accessing the Internet using Proxy ==

+

== Accessing the Internet using Proxy ==

−

Each Hudson instance has unrestricted access to the Internet by using proxy.eclipse.org. The shell environment variables below are set for the Hudson build user. If your build process overrides, or bypasses these variables, you must instruct your tools to use the proxy service to access external sites.

+

Each Hudson instance has unrestricted access to the Internet by using proxy.eclipse.org. The shell environment variables below are set for the Hudson build user. If your build process overrides, or bypasses these variables, you must instruct your tools to use the proxy service to access external sites.

The purpose of the Proxy for Hudson is not for security -- we use firewalls for that. The proxy is used for three specific reasons:

+

The purpose of the Proxy for Hudson is not for security -- we use firewalls for that. The proxy is used for three specific reasons:

−

# The Eclipse Hudson environment is expected to grow to a large number of slaves for builds and for tests. If each of those slaves requires a routable IP address, the Foundaton will be required to acquire (at cost) additional IP blocks, which further complicates routing and firewall setups.

+

#The Eclipse Hudson environment is expected to grow to a large number of slaves for builds and for tests. If each of those slaves requires a routable IP address, the Foundaton will be required to acquire (at cost) additional IP blocks, which further complicates routing and firewall setups.

−

# A proxy will allow us to track and monitor external dependencies that are downloaded at build time, for IP purposes.

+

#A proxy will allow us to track and monitor external dependencies that are downloaded at build time, for IP purposes.

−

# A proxy will enable us to implement caching at the proxy level, should the CI mechanism begin to download the entire world and consume too much bandwidth.

+

#A proxy will enable us to implement caching at the proxy level, should the CI mechanism begin to download the entire world and consume too much bandwidth.

+

=== Configuring a proxy for the p2 director ===

+

The p2 director does not respect the "http.proxyHost" etc. options passed in on command line. But, since the p2 director is an Eclipse application, one way to configure the proxy settings is to set the configuration file '''configuration/.settings/org.eclipse.core.net.prefs''' in the director installation as below. [Note: this are roughly accurate as of 03/15/2013. They may change from time to time, over the years, so if they don't seem to work, contact the webmasters to ask if they have changed or differ for different machines.] Notice this example has "systemProxiesEnabled" set to true. This is because on Hudson, from time to time, the webmasters may change the proxy configuration for each slave differently, so if you can take advantage of the system-set proxies, you will be better off. If not, the provided values will apply if you set "systemProxiesEnabled" to false. Also note that the "socks proxy" should not be set in Eclipse, unless you know for sure you have a true socks proxy, or else all traffic will be routed through that, so if its not a true socks-level proxy none of the others will work. The proxy for Hudson is not a socks proxy, just HTTP, HTTPS, and FTP protocols. Also note, a full "Eclipse Platform", where native providers are provided, such as on Windows, will automatically detect and fill-in the system proxies. But, this auto-detection won't work if you have a "bare" p2-director app or something like the platform's "basebuilder"; in which case you must provide them with a script such as the following. See {{bug|401964}} for some discussion of these issues, among many other network and infrastructure issues.

The easiest way to obtain this file is to configure an Eclipse installation on your own computer (in Eclipse, go to '''Window > Preferences > General > Network Connections'''), and then copy the configuration that Eclipse wrote in '''<eclipse>/configuration/.settings/org.eclipse.core.net.prefs'''.

On the Windows operating system, proxy settings (and exceptions to using the proxy) can be set in "Internet Options". These are "detected" by Eclipse and set in "native" values of proxy preferences, but, apparently, from searching eclipse bugs for "proxies", some functions in Eclipse use these preferences and others do not. In any case, you might HAVE to set the Windows Internet Options proxy exceptions and in some cases it might make things easier. (For one case of details/history, see bug {{bug|372880}}.

−

Buckminster cvs materializing, uses a proxy, how is this configured?

+

== Additional Troubleshooting Tips ==

−

To finish this thread. Michael Wenz pointed me to a change made in the cdo build (to solve this issue), a snippet from

+

=== Buckminster CVS materializing: proxy error: Forbidden ===

−

his email to me:

+

−

But I saw that the CDO build is green again and they still do an Ant call from Hudson that again triggers Buckminster.

But I saw that the CDO build is green again and they still do an Ant call from Hudson that again triggers Buckminster. Previously that build failed with the same exception as ours did or do.

−

<env key="no_proxy" value="${no.proxy}" />

+

−

<properties />

+

Not sure what these guys changed, but I saw that they added something in their build.xml that seems to fix this. I found 2 snippets that appear to be in connection with this: ... &lt;condition property="no.proxy" value="${env.no_proxy}, dev.eclipse.org" else="dev.eclipse.org"&gt; &lt;isset property="env.no_proxy" /&gt; &lt;/condition&gt; ...

Normally "project level" administration is defined for a Hudson job. This allows for only one or a few committers to have "full access" to the job, to do builds, change the configuration, or even delete the job. To give access to everyone, say to "read" the builds, you can add the user "anonymous" and mark the "read" check box. Typically, it is desired to have some "in between" access to all the committers of a project, for example, to maybe any committer can kick off a build, but only the project-level administrator can change the configuration. If this is desired, there is a "role" groups that can be used instead of listing all committers by name. The "role" name is formed by perpending "ROLE_" to the upper case version of the Linux group that defines the committers. For example, EPP committers are authorized using the Linux group ''technology.packaging'', so their Hudson group would be ROLE_TECHNOLOGY.PACKAGING. So, as an example, the project level authorization might look like the following, from the Hudson "configure project" page: <br>

+

+

[[Image:Projectlevel.png|Example Project Level Security settings]]

== Hudson for Administrators ==

== Hudson for Administrators ==

Line 110:

Line 142:

*[[Common Build Infrastructure/Managing Hudson|Manage Hudson]]

*[[Common Build Infrastructure/Managing Hudson|Manage Hudson]]

*[[Hudson/Admin/UpgradeHudson|Upgrade Hudson]]

*[[Hudson/Admin/UpgradeHudson|Upgrade Hudson]]

−

*[[Hudson/Admin/Installed_Plugins|Installed Plugins]]

+

*[[Hudson/Admin/Installed Plugins|Installed Plugins]]

−

=== Duties of Administrators ===

+

=== Duties of Administrators ===

#Hudson upgrades and restarts

#Hudson upgrades and restarts

Line 121:

Line 153:

#Monitor the Hudson Inbox.

#Monitor the Hudson Inbox.

−

=== Who are the Adminstrators ===

+

=== Who are the Adminstrators ===

−

*Nick Boldt

+

* Eclipse Webmasters - webmaster@eclipse.org

−

*Kim Moir

+

* David Williams

−

*David Carver

+

−

*Thomas Hallgren

+

−

*Chris Aniszczyk

+

−

Please add other known administrators.

+

You can contact the Hudson admins by opening [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=Hudson a bug]

General Information

The Eclipse Foundation runs a Hudson continuous integration server that Eclipse-hosted projects can take advantage of for their builds. Currently this is hosted here: https://hudson.eclipse.org/ . The Hudson server allows for the execution of Continuous Integration Builds, Nightly Builds, Integration Builds, Release Builds, and Testing. Hudson is maintained by the Eclipse Webmasters. A Hudson sandbox instance is available for those who like the bleeding edge; however, its stability may suffer due to frequent restarts, untested configurations and upgrades.

Note if you are looking for information on the Hudson software project itself at Eclipse then that can be found at hudson-ci.

Hudson configuration and tools

The Hudson setup uses a combination of SLES 11 x86_64 machines for Linux slaves, Windows 7 for the Windows slave(s) and Mac OS X. These machines are behind a firewall so any outbound http(s) connections are proxied.

The following global variables are set(identically across installs):

JVM_OPTS: proxy data (see "Accessing the Internet" below)

ANT_ARGS: proxy data

ANT_OPTS: proxy data

Each node also has a .m2/settings.xml file with the proxy data.

Choosing the right slave

hudson-slave1, hudson-slave2, hudson-slave4 and hudson-slave6 - these are the main build nodes for Hudson jobs. You can specify them by name or by using the 'build2' label.

hudson-slave5 - this is an ia64 slave.

hudson-slave7, hudson-slave8 - these are ppc64 slaves located at OSUOSL.

fastlane - this slave is intended for usage during a release train crunch when re-spins may require more capacity than hudson-slave1&2 can provide. By default jobs should not run here.

hudson-perf1-tests - this slave is used for running performance tests ONLY.

mac-tests and windows7tests - these 2 slaves are meant for running UI tests for their respective OS versions. By default jobs should not run on either slave.

Server Storage

Build and Hudson storage layout

Three tiers of storage are available for storing Workspaces, build artifacts, nightly and release builds. For optimal build performance and service availability, it is important that you use each storage device according to its intended purpose.

The image on the right illustrates the three storage tiers and their intended purpose.

Tools(and locations)

Maven 2.2.1 (installed automatically)

Maven 3.0 alpha 5 (installed automatically)

Maven 3.0-alpha-5-local (/shared/common/apache-maven-3.0-alpha-5)

Maven 3.0-alpha-6-local (/shared/common/apache-maven-3.0-alpha-6)

Maven 3.0-Beta-1 (/shared/common/apache-maven-3.0-beta-1)

Sun Java 5 SR 22 64bit (/shared/common/jdk-1.5.0-22.x86_64)

Sun Java 5 R 16 32bit (/shared/common/jdk-1.5.0_16)

Sun Java 5 R 22 64bit (/shared/common/jdk-1.5.0-22.x86_64)

Sun Java 6 R 21 32bit (/shared/common/sun-jdk1.6.0_21_i586)

Sun Java 6 R 21 64bit (/shared/common/sun-jdk1.6.0_21_x64)

Apache-ant-1.8.1 (/shared/common/apache-ant-1.8.1)

Apache-ant-1.7.1 (/shared/common/apache-ant-1.7.1)

Apache-ant-1.7.0 (/shared/common/apache-ant-1.7.0)

Apache-ant-1.6.5 (/shared/common/apache-ant-1.6.5)

Headless Buckminster 3.6 (/shared/common/buckminster-3.6)

Buckminster 3.6 Integration (installed automatically)

Accessing the Internet using Proxy

Each Hudson instance has unrestricted access to the Internet by using proxy.eclipse.org. The shell environment variables below are set for the Hudson build user. If your build process overrides, or bypasses these variables, you must instruct your tools to use the proxy service to access external sites.

Why use a Proxy?

The purpose of the Proxy for Hudson is not for security -- we use firewalls for that. The proxy is used for three specific reasons:

The Eclipse Hudson environment is expected to grow to a large number of slaves for builds and for tests. If each of those slaves requires a routable IP address, the Foundaton will be required to acquire (at cost) additional IP blocks, which further complicates routing and firewall setups.

A proxy will allow us to track and monitor external dependencies that are downloaded at build time, for IP purposes.

A proxy will enable us to implement caching at the proxy level, should the CI mechanism begin to download the entire world and consume too much bandwidth.

Configuring a proxy for the p2 director

The p2 director does not respect the "http.proxyHost" etc. options passed in on command line. But, since the p2 director is an Eclipse application, one way to configure the proxy settings is to set the configuration file configuration/.settings/org.eclipse.core.net.prefs in the director installation as below. [Note: this are roughly accurate as of 03/15/2013. They may change from time to time, over the years, so if they don't seem to work, contact the webmasters to ask if they have changed or differ for different machines.] Notice this example has "systemProxiesEnabled" set to true. This is because on Hudson, from time to time, the webmasters may change the proxy configuration for each slave differently, so if you can take advantage of the system-set proxies, you will be better off. If not, the provided values will apply if you set "systemProxiesEnabled" to false. Also note that the "socks proxy" should not be set in Eclipse, unless you know for sure you have a true socks proxy, or else all traffic will be routed through that, so if its not a true socks-level proxy none of the others will work. The proxy for Hudson is not a socks proxy, just HTTP, HTTPS, and FTP protocols. Also note, a full "Eclipse Platform", where native providers are provided, such as on Windows, will automatically detect and fill-in the system proxies. But, this auto-detection won't work if you have a "bare" p2-director app or something like the platform's "basebuilder"; in which case you must provide them with a script such as the following. See bug 401964 for some discussion of these issues, among many other network and infrastructure issues.

The easiest way to obtain this file is to configure an Eclipse installation on your own computer (in Eclipse, go to Window > Preferences > General > Network Connections), and then copy the configuration that Eclipse wrote in <eclipse>/configuration/.settings/org.eclipse.core.net.prefs.

Configuring for a proxy on Windows OS

On the Windows operating system, proxy settings (and exceptions to using the proxy) can be set in "Internet Options". These are "detected" by Eclipse and set in "native" values of proxy preferences, but, apparently, from searching eclipse bugs for "proxies", some functions in Eclipse use these preferences and others do not. In any case, you might HAVE to set the Windows Internet Options proxy exceptions and in some cases it might make things easier. (For one case of details/history, see bug bug 372880.

Additional Troubleshooting Tips

Buckminster CVS materializing: proxy error: Forbidden

To finish this thread. Michael Wenz pointed me to a change made in the cdo build (to solve this issue), a snippet from his email to me:

But I saw that the CDO build is green again and they still do an Ant call from Hudson that again triggers Buckminster. Previously that build failed with the same exception as ours did or do.

Not sure what these guys changed, but I saw that they added something in their build.xml that seems to fix this. I found 2 snippets that appear to be in connection with this: ... <condition property="no.proxy" value="${env.no_proxy}, dev.eclipse.org" else="dev.eclipse.org"> <isset property="env.no_proxy" /> </condition> ...

Hudson for Committer Project-level Administration

Normally "project level" administration is defined for a Hudson job. This allows for only one or a few committers to have "full access" to the job, to do builds, change the configuration, or even delete the job. To give access to everyone, say to "read" the builds, you can add the user "anonymous" and mark the "read" check box. Typically, it is desired to have some "in between" access to all the committers of a project, for example, to maybe any committer can kick off a build, but only the project-level administrator can change the configuration. If this is desired, there is a "role" groups that can be used instead of listing all committers by name. The "role" name is formed by perpending "ROLE_" to the upper case version of the Linux group that defines the committers. For example, EPP committers are authorized using the Linux group technology.packaging, so their Hudson group would be ROLE_TECHNOLOGY.PACKAGING. So, as an example, the project level authorization might look like the following, from the Hudson "configure project" page: