<!-- The content of this document is licensed under the CC-BY-SA license -->

33

<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->

29

<license/>

34

<license/>

30

35

31

<version>1.2.3</version>

36

<version>1.18</version>

32

<date>October 11, 2003</date>

37

<date>2006-12-24</date>

33

34

<abstract>This document serves as a HOWTO for using distcc with Gentoo.</abstract>

35

38

36

<chapter>

39

<chapter>

37

<title>Introduction</title>

40

<title>Introduction</title>

38

<section>

41

<section>

39

<title>What is distcc?</title>

42

<title>What is distcc?</title>

40

<body>

43

<body>

41

<p>Distcc is a program designed to distribute compiling tasks across a network to participating hosts. It is comprised of a server, <c>distccd</c> and a client program, <c>distcc</c>. Distcc can work transparently with <uri link="http://ccache.samba.org">ccache</uri> and Portage with a little set up.</p>

44

45

<p>

46

Distcc is a program designed to distribute compiling tasks across a network to

47

participating hosts. It is comprised of a server, <c>distccd</c>, and a client

48

program, <c>distcc</c>. Distcc can work transparently with <uri

49

link="http://ccache.samba.org">ccache</uri>, Portage, and Automake with a

50

little setup.

51

</p>

52

42

</body>

53

</body>

43

</section>

54

</section>

55

<section>

56

<title>Using distcc to bootstrap</title>

57

<body>

58

59

<p>

60

If you are planning on using distcc to help you bootstrap a Gentoo

61

installation, make sure you read the section <uri link="#bootstrapping">Using

62

distcc to Bootstrap</uri>, which is situated further down in this document.

63

</p>

64

65

</body>

66

</section>

44

</chapter>

67

</chapter>

45

68

46

<chapter>

69

<chapter>

47

<title>Setup</title>

70

<title>Setup</title>

48

<section>

71

<section>

49

<title>Dependencies</title>

72

<title>Dependencies</title>

50

<body>

73

<body>

51

<pre caption="Distcc dependencies (2.8 through 2.10)">

74

52

&gt;=sys-apps/portage-2.0.46-r11

75

<p>

53

<codenote>Distcc-2.11 and on require &gt;=sys-apps/portage-2.0.49-r6</codenote>

76

In order to use Distcc, all of the computers on your network need to have the

54

&gt;=sys-devel/gcc-config-1.3.1

77

same GCC versions. For example, mixing 3.3.x (where the x varies) is okay, but

55

sys-apps/shadow

78

mixing 3.3.x with 3.2.x <b>may</b> result in compilation errors or runtime

56

<codenote>(As of version 2.8 and up until 2.11) And the following optional dependencies when you have <i>gtk</i> in your <i>USE</i> flags</codenote>

79

errors.

57

&gt;=x11-libs/gtk+-2.2.1

58

</pre>

80

</p>

59

<pre caption="Distcc Dependencies (post 2.11.1)">

81

60

&gt;=sys-apps/portage-2.0.49-r6

61

&gt;=sys-devel/gcc-config-1.3.1

62

sys-apps/shadow

63

<codenote>In this revision you may chose between a Gnome and GTK GUI monitor, they have the following added dependencies</codenote>

64

<codenote>For GTK:</codenote>

65

&gt;=x11-libs/gtk+-2.0.0

66

&gt;=gnome-base/libglade-2.0.0

67

x11-libs/pango

68

<codenote>For Gnome:</codenote>

69

&gt;=x11-libs/gtk+-2.0.0

70

&gt;=gnome-base/libglade-2.0.0

71

x11-libs/pango

72

&gt;=gnome-base/libgnomeui-2.0.0.0

73

&gt;=gnome-base/libgnome-2.0.0

74

</pre>

75

</body>

76

</section>

77

<section>

78

<title>Installing Distcc</title>

79

<body>

80

<p>Installing Distcc is very easy. Simply set your USE flags and <i>emerge distcc</i>. But, there's a couple of options you should know about.</p>

81

<p>Distcc ships with a graphical monitor to monitor tasks that your computer is sending away for compilation. If you use Gnome then put 'gnome' in your USE flags. However, if you don't use Gnome and would still like to have the monitor then you should put 'gtk' in your USE flags.</p>

82

</body>

82

</body>

83

</section>

83

</section>

84

<section>

84

<section>

85

<title>Installing Distcc</title>

86

<body>

87

88

<p>

89

There are a couple of options you should be aware of before you start

90

installing distcc.

91

</p>

92

93

<p>

94

Distcc ships with a graphical monitor to monitor tasks that your computer is

95

sending away for compilation. If you use Gnome then put 'gnome' in your USE

96

flags. However, if you don't use Gnome and would still like to have the

97

monitor then you should put 'gtk' in your USE flags.

98

</p>

99

100

<pre caption="Installing distcc">

101

# <i>emerge distcc</i>

102

</pre>

103

104

<impo>

105

Remember, you must be sure to install distcc on all of your participating

106

machines.

107

</impo>

108

109

</body>

110

</section>

111

<section>

85

<title>Setting up Portage to use Distcc</title>

112

<title>Setting up Portage to use Distcc</title>

86

<body>

113

<body>

87

<p>Setting up distcc is very easy to do with Portage. Follow these simple steps on each computer you want to use distcc on:</p>

114

115

<p>

116

Setting up Portage to use distcc is easy. Execute the following steps on

117

each system that should participate in the distributed compiling:

118

</p>

119

88

<pre caption="Integrating Distcc and Portage">

120

<pre caption="Integrating Distcc and Portage">

89

# <i>emerge distcc</i>

121

# <i>emerge distcc</i>

90

# <i>nano -w /etc/make.conf</i>

122

# <i>nano -w /etc/make.conf</i>

123

<comment>(Set N to a suitable number for your particular setup)</comment>

124

<comment>(A common strategy is setting N as twice the number of total CPUs + 1

125

available)</comment>

126

MAKEOPTS="-jN"

127

<comment>(Add distcc to your FEATURES)</comment>

128

FEATURES="distcc"

129

</pre>

130

131

</body>

132

</section>

133

<section>

134

<title>Specifying Participating Hosts</title>

135

<body>

136

137

<p>

138

Use the <c>distcc-config</c> command to set the list of hosts. Here is an

139

example of some hosts that might be in your list:

91

</pre>

140

</p>

92

<impo>If you use distcc-2.11.1 or greater with &gt;=portage-2.0.49-r6, then all you need to do as far as setup is to set your hosts (see next code block), modify MAKEOPTS in <path>/etc/make.conf</path>, and add 'distcc' to FEATURES in <path>/etc/make.conf</path>.</impo>

93

<note>Edit your FEATURES flags to include &quot;distcc&quot;</note>

94

<note>You should also uncomment the DISTCC_TMPDIR line in <path>/etc/make.conf</path>.</note>

95

<note>Set <i>DISTCC_DIR=${PORTAGE_TMPDIR}/portage/.distcc</i> near the end of the file (last line is okay)</note>

96

141

97

<p>Next you have to specify what hosts you want to use. To do this you can use the <c>distcc-config</c> command to set the list of hosts. Here is an example of some hosts that might be in your list:</p>

98

<pre caption="Examples of host definitions">

142

<pre caption="Examples of host definitions">

99

192.168.0.1 192.168.0.2 192.168.0.3

143

192.168.0.1 192.168.0.2 192.168.0.3

100

192.168.0.1/2 192.168.0.2 192.168.0.3/10

144

192.168.0.1/2 192.168.0.2 192.168.0.3/10

101

192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4

145

192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4

102

@192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3

146

@192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3

147

<comment>(There are also several other methods of setting up hosts. See the

148

distcc manpage for more details.)</comment>

149

<comment>If you wish to compile on the local machine you should put 'localhost'

150

in the hosts list. Conversely if you do not wish to use the local machine to

151

compile (which is often the case) omit it from the hosts list. On a slow

152

machine using localhost may actually slow things down. Make sure to test your

153

settings for performance.</comment>

154

</pre>

155

156

<p>

157

It may all look complicated, but in most cases a variant of line 1 or 2 will

158

work.

103

</pre>

159

</p>

104

<p>It may all look complicated, but in most cases a variant of line 1 or 2 will work. An explanation of each line is: Line 1 is just a space-delimited list of hosts that will use default everything. Line 2 is a list of hosts that specifies the maximum number of jobs (by use of the /N) to send that host at any given time (specified with the <c>/n</c>). Since most people won't be using lines 3 or 4, I'll <uri link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc docs for more information.</p>

<p>The final step to integrating distcc into Portage is to re-open your <path>/etc/make.conf</path> and edit <c>MAKEOPTS</c> to include <c>-jN</c> (where N is an integer). Typically you will want to set this to the total number of processors in your network plus one.</p>

175

<p>

111

<pre caption="Final steps in make.conf">

176

Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the

177

<c>--allow</c> directive to allow only hosts you trust. For added security,

178

you should also use the <c>--listen</c> directive to tell the distcc daemon

179

what IP to listen on (for multi-homed systems). More information on distcc

<p>This is in some cases easier than the Portage setup. What you have to do is update your <c>PATH</c> environmental variable to include <path>/usr/lib/distcc/bin</path> before where gcc (<path>/usr/bin</path>). However, there is a caveat. If you use ccache you have to put distcc after the ccache part. Like so:</p>

<p>Then, as you would normally type <c>make</c>, you would type <c>make -jN</c> (where N is an integer). A safe number to use for N varies on your network and the types of computers you are using to compile. For me, I have a dual processor P3 and a K6-2/400 that compile everything for my desktop (a 1200MHz Athlon Thunderbird) and I use -j5. Test your own settings to find the right number.</p>

278

126

</body>

279

</body>

127

</section>

280

</section>

281

<section>

282

<title>Step 2: Getting Distcc</title>

283

<body>

284

285

<p>

286

Before the installation of distcc, a user called distcc must be added to the

Distcc is now set up to bootstrap! Continue with the official installation

328

instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge

329

system</c>. This is to make sure that all of the dependencies you want are

330

installed as well.

331

</p>

332

333

<note>

334

During bootstrap and <c>emerge system</c> distcc may not appear to be used.

335

This is expected as some ebuilds do not work well with distcc, so they

336

intentionally disable it.

337

</note>

338

339

</body>

340

</section>

128

</chapter>

341

</chapter>

129

342

130

<chapter>

343

<chapter>

131

<title>Cross-Compiling</title>

344

<title>Troubleshooting</title>

132

<section>

345

<section>

133

<title>A Note on Cross-Compiling</title>

346

<title>Some Packages Don't Use Distcc</title>

134

<body>

347

<body>

135

<p>Cross-compiling is using one architecture to build programs for another architecture. This can be as simple as using an Athlon (i686) to build a program for a K6-2 (i586), or using a Sparc to build a program for a ppc.</p>

348

349

<p>

350

As you emerge various packages, you'll notice that some of them aren't being

351

distributed (and aren't being built in parallel). This may happen because the

352

package's Makefile doesn't support parallel operations or the maintainer of the

353

ebuild has explicitly disabled parallel operations due to a known problem.

354

</p>

355

356

<p>

357

Sometimes distcc might cause a package to fail to compile. If this happens

358

for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.

359

</p>

360

136

</body>

361

</body>

137

</section>

362

</section>

138

<section>

363

<section>

139

<title>A Personal Note on Cross-compiling</title>

364

<title>Mixed GCC Versions</title>

140

<body>

365

<body>

141

<p>I'd love to be able to help out on the Gentoo Cross-compiling effort, but I don't have any non-x86 machines. I can create things that work <e>in theory</e>, but I have to rely on other people to test what I write. That works to an extent, but it's tough. ;-)</p>

366

142

<p>In the near future I hope to acquire a Sparc of some type to put Gentoo on so I can play around in-house.</p>

367

<p>

368

If you have different GCC versions on your hosts, there will likely be very

369

weird problems. The solution is to make certain all hosts have the same GCC

370

version.

371

</p>

372

373

<p>

374

Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of

375

<c>gcc</c>. This means that if you're mixing i686 machines with other types

376

(i386, i586) you will run into problems. A workaround for this may be to

377

<c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>.

378

</p>

379

380

<impo>

381

Doing this explicitly redefines some behaviour of Portage and may have some

382

weird results in the future. Only do this if you're mixing CHOSTs.

383

</impo>

384

385

143

</body>

386

</body>

144

</section>

387

</section>

145

</chapter>

388

</chapter>

146

389

147

<chapter>

390

<chapter>

148

<title>Using Distcc to Bootstrap</title>

149

<section>

150

<title>Step 1: Setup on New Box</title>

151

<body>

152

<p>Next, you have to boot your new box with a Gentoo Linux LiveCD and follow all of the steps up until the bootstrapping. Then proceed with a little preliminary setup on the new box.</p>

153

<pre caption="Preliminary Setup">

154

<codenote>Here we add distcc to FEATURES</codenote>

155

# <i>nano -w /etc/make.conf</i>

156

<codenote>Add &quot;distcc&quot; to FEATURES</codenote>

157

<codenote>Modify MAKEOPTS in <path>/etc/make.conf to include -jN (as described above)</path></codenote>

<p>distcc should be set up now to bootstrap! Continue the official install guide, and don't forget to re-emerge distcc after <c>emerge system</c>.</p>

184

</body>

185

</section>

186

</chapter>

187

188

<chapter>

189

<title>Troubleshooting</title>

190

<section>

191

<title>Mozilla and Xfree</title>

192

<body>

193

<p>As you emerge various packages, you'll notice that they aren't being distributed (and indeed aren't being built in parallel). This is because the developers of Mozilla and Xfree .ebuilds disabled parallel building because it is known to cause problems. This isn't necessarily a distcc problem.</p>

194

<p>That isn't to say that sometimes distcc will cause a package to fail to compile.</p>

195

</body>

196

</section>

197

<section>

198

<title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>

199

<body>

200

<p>With such a long title any explanation here is almost irrelevent. However, if you plan to use distcc across hosts that have the <uri link="http://www.gentoo.org/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do not, you will run into problems.</p>

201

<p>The solution requires a little foresight on your part; you have to run <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of which are a good thing to do since for the most part the protections offered by both packages is a good thing and is transparent to the user.</p>

202

</body>

203

</section>

204

<section>

205

<title>Mixed GCC Versions</title>

206

<body>

207

<p>If you have differing versions of GCC on your hosts, there will likely be very weird problems. The solution is to make certain all hosts have the same GCC version.</p>

208

</body>

209

</section>

210

</chapter>

211

212

<chapter>

213

<title>Distcc Extras</title>

391

<title>Distcc Extras</title>

214

<section>

392

<section>

215

<title>Distcc Monitors</title>

393

<title>Distcc Monitors</title>

216

<body>

394

<body>

217

<p>Distcc ships with two monitors. The text-based one is always built, it is called <c>distccmon-text</c>. Running it for the first time can be a bit confusing, but it is really quite easy to use. If you run the program with no parameter it will run once. However, if you pass it a number it will update every N seconds, where N is the argument you passed.</p>

395

218

<p>The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c> in your <c>USE</c> flags. This one is GTK+ based and runs in an X environment, and it is quite lovely.</p>

396

<p>

219

<p>There is a caveat to using these programs. If you want to monitor any emerge, you have to start the monitor like so:</p>

397

Distcc ships with two monitors. The text-based one is always built and is

398

called <c>distccmon-text</c>. Running it for the first time can be a bit

399

confusing, but it is really quite easy to use. If you run the program with no

400

parameter it will run once. However, if you pass it a number it will update

401

every N seconds, where N is the argument you passed.

402

</p>

403

404

<p>

405

The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>

406

in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment

407

and it is quite lovely. For Gentoo the GUI monitor has been called

408

<c>distccmon-gui</c> for less confusion. Elsewhere it may be referred to as

409

<c>distccmon-gnome</c>.

410

</p>

411

220

<pre caption="Starting the monitors">

412

<pre caption="Starting the monitors">

413

# <i>distccmon-text N</i>

414

<comment>(Or)</comment>

415

# <i>distccmon-gui</i>

416

<comment>To monitor Portage's distcc usage you can use:</comment>

221

# <i>DISTCC_DIR=/path/to/distccdir distccmon-text N</i>

417

# <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>

222

<codenote>Or you can start the graphical monitor...</codenote>

418

<comment>(Or)</comment>

223

# <i>DISTCC_DIR=/path/to/distccdir distccmon-gnome</i>

419

# <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gui</i>

224

</pre>

420

</pre>

225

<impo>If you are using Portage and used the above example, this will be <c>DISTCC_DIR=/var/tmp/portage/.distcc</c>.</impo>

421

226

<note>If you aren't using Portage you do not need to specify DISTCC_DIR (i.e., just type distccmon-text N).</note>

422

<impo>

423

If your distcc directory is elsewhere, change the DISTCC_DIR variable

424

accordingly.

425

</impo>

426

227

</body>

427

</body>

228

</section>

428

</section>

229

</chapter>

230

231

<chapter>

429

</chapter>

232

<title>Future Plans for Distcc and Gentoo</title>

233

<section>

234

<title>distcc-subnetscan</title>

235

<body>

236

<p><c>distcc-subnetscan</c> is a perl program in development that will scan a subnet for hosts that have a participating distcc daemon. Even this will be enhanced to test if the daemon on a remote host will conform to a specific <c>CHOST</c> setting to make cross-compiling easier.</p>

237

<p>The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/distcc/distcc-subnetscan/distcc-subnetscan.pl">right here</uri> until a more formal home can be found.</p>

238

</body>

239

</section>

240

<section>

241

<title>distcc-config</title>

242

<body>

243

<p><c>distcc-config</c>, the userland configuration tool for distcc is out of date. Soon it will be rewritten to be brought up the current version of distcc.</p>