Installation

a. If you are installing from the DVD, check to be sure that your operating system allows programs to be
executed from the DVD. For example, some Linux man pages for mount suggest setting the properties for the CD or DVD
drive in /etc/fstab to "/dev/cdrom /cd iso9660 ro,user,noauto,unhide", which is notably
missing the property "exec". Add exec to that list in /etc/fstab, or add it to your mount
command. Notice that the sample Linux mount command in install-guide-unix.html does include exec.

Perhaps install.sh lacks permission to run because you tried to copy all the files from the
DVD, in order to move them to another system. If so, please don't do that. There's an easier way. See the next
question.

Install.02q. The DVD drive is on system A, but I want to install on system B. What do I do?

Install.03
q. Why did the installation fail with Error occurred while processing: C:\Documents?

I was installing on Windows using the tar file. It said:

C:\Documents and Settings\John\cpu2006> install
The environment variable SPEC should point to the source of the
SPEC distribution as an absolute path. I will now try to set
the variable for you...
SPEC is set to C:\Documents and Settings\John\cpu2006
If this is NOT what you want, press control-C
Press any key to continue . . .
Installing from "C:\Documents and Settings\John\cpu2006"
Checking the integrity of your source tree...
Depending on the amount of memory in your system, and the speed of your
destination disk, this may take more than 10 minutes.
Please be patient.
The system cannot find the file specified.
Error occurred while processing: C:\Documents.
The system cannot find the file specified.
Error occurred while processing: and.
The system cannot find the path specified.
C:\Documents and Settings\John\cpu2006\tools\bin\windows-i386\specmd5sum: MANIFEST.tmp: no properly formatted MD5 checksum lines found
Package integrity check failed!
Installation NOT completed!

a. The problem is that the SPEC tools do NOT support spaces in path names. Sorry. This is a limitation
of the SPEC toolset and there are currently no plans to change this requirement. Please use a path that does not contain
spaces.

Install.04q. How do I uninstall? This script uninstall.sh doesn't seem to be it.

a. You are correct that uninstall.sh does not remove the whole product; it only removes the
SPEC tool set, and does not affect the benchmarks (which consume the bulk of the disk space). At this time, SPEC does not
provide an uninstall utility for the suite as a whole. But it's easy to do: on Unix systems, use rm
-Rf on the directory where you installed the suite, for example:

rm -Rf /home/cs3000/saturos/spec/cpu2006

On Windows systems, select the top directory in Windows Explorer and delete it.

If you have been using the output_root feature,
you will have to track those down separately. See the suggested commands in the appendix about uninstalling, in the install guides.

Note: instead of deleting the entire directory tree, some users find it useful to keep the
config and result subdirectories, while deleting everything else.

Install.05 Do I need to be root?

Occasionally, users of Unix systems have asked whether it is necessary to elevate privileges, or to
become 'root', prior to installing or running SPEC CPU2006.

a. SPEC recommends (*) that you do not become root,
because: (1) To the best of SPEC's knowledge, no component of SPEC CPU needs to modify system directories, nor does
any component need to call privileged system interfaces. (2) Therefore, if you find that it appears that there is
some reason why you need to be root, the cause is likely to be outside the SPEC toolset - for example, disk
protections, or quota limits. (3) For safe benchmarking, it is better to avoid being
root, for the same reason that it is a good idea to wear seat belts in a car: accidents happen, humans make
mistakes. For example, if you accidentally type:

kill 1

when you meant to say:

kill %1

then you will very grateful if you are not privileged at that moment.

(*) This is only a recommendation, not a requirement nor a rule.

runspec

runspec.01q. When I say runspec, why does it say
Can't locate strict.pm? For example:

a. You can't use runspec if its path is not set correctly. On Unix, Linux, or Mac OS X, you should
source shrc or cshrc, as described in runspec.html section
2.4. For Windows, please edit shrc.bat and make the adjustments described in the comments. Then, execute that
file, as described in runspec.html section 2.5.

runspec.03
q. Why does runspec fail with The feature 'http://xml.org/sax/features/namespaces' is not
recognized by XML::SAX::ExpatXS?

a. The problem is that the ExpatXS module as built on SuSE 10.1 requires a type of code relocation that
SELinux does not want to allow. For SPEC CPU2006 V1.1, several workarounds were suggested: enable relocations for the
offending object, or turn off SELinux, or try a different toolset. For CPU2006 V1.2, the problem is not expected to
occur.

runspec.04
q. Why did I get the message Error: Start tag for undeclared element platform_settings?

I installed SPEC CPU2006 V1.2, and now my runspec command exits immediately, with this
message:

ERROR: An error was encountered while parsing the flag description file
at /cpu2006/V1.2/config/Oracle-Solaris-Studio12.2.xml
Output from the XML validator follows:
-----------
Error: Start tag for undeclared element platform_settings
in unnamed entity at line 91 char 19

a. The problem above is that although the named file
(/cpu2006/V1.2/config/Oracle-Solaris-Studio12.2.xml) has useful information about the tested platform, the
information needs to be updated and put into new locations. For SPEC CPU2006 V1.2, instead of using a platform element, you need to use a platform file. The message above is telling you that
there is no element named "platform_settings" (as of V1.2).

An example of how to fix SPEC CPU2006 V1.1 flags files to match the SPEC CPU2006 V1.2 format is
available in flag-description.html.

You must fix this problem in order to proceed, because runspec exits immediately if flags files
are improperfly formatted. (Or, you could do your run with no flags files at all, and then add them back in afterwards
with rawformat --flagsurl. This tactic will allow your run to
proceed, but the results wll be marked invalid until you apply the flags.)

runspec.05 Do I need to be root?

Regarding the root account, the answer for runspec is the same as the answer for installation question #5, above.

Building benchmarks

Build.01q. I'm using Windows and I can't build any
benchmarks. What does CreateProcess((null), ifort ...) failed mean? For example:

a. This CreateProcess failure occurs on Windows when specmake cannot find your compiler.
(Recall from system-requirements.html that the benchmarks are supplied
in source code form, and that they must be compiled.)

The reason that it cannot find your compiler is, most likely, because:

You have not edited shrc.bat to correctly reference your compiler, or

You have used a config file that references a compiler that does not exist.

To fix your problem, investigate and address both items.

In shrc.bat, you need to either:

Reference a vendor-supplied file that sets the path.

The supplied shrc.bat in V1.2 mentions sample .bat files that are often
provided by compilers to set your environment. If you want to use this option but you can't find the right file,
check your compiler documentation, or you might try searching your hard drive for *vars*bat.

The file names change frequently; check your compiler docs.

For example, for Visual Studio 10, you might use something like call "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools\vsvars32.bat" but for
older compilers, you might need to look for vcvars32.bat, not vsvars32.bat.

For example, when using the (mid-2011) Intel compiler, to select 32 or
64-bit mode, you must remember
to add the appropriate argument after the call. This statement requests 32-bit mode: call
"C:\Program Files (x86)\Intel\ComposerXE-2011\bin\compilervars.bat" ia32

For example, when using the PGI 11.7 compiler, the path varies depending
on whether you are using 32 or 64-bit mode:
call "c:\Program Files\PGI\win64\11.7\pgi_env.bat"or
call "c:\Program Files (x86)\PGI\win32\11.7\pgi_env.bat"

Summary: your call statement will vary, by compiler, by version, and sometimes, by
features. Check your compiler's documentation, to find the right call.

Or, edit the path yourself.

You can set the path yourself if you like, with commands such as:

set PATH=%PATH%;"c:\program files\path\to\my\compiler\bin"

If you want to use option b, but you can't figure out what path to set, try looking in the
documentation for your compiler under topics such as "setting the path", or "command line", or "environment
variables". The documentation should mention whether any other environment variables (such as LIB or INCLUDE) are
required, in addition to PATH.

You must also use a config file that is appropriate for your compiler. Microsoft C++ users: please see below.

Build.02q. The file make.clean.err is mentioned,
but it does not exist. Why not?

a. Yes, but they are incomplete. In your directory %SPEC%\config\ there are
config files that can be used to compile all but one of the C/C++ benchmarks:

%SPEC%\config\Example-windows-ia32-visualstudio.cfg

or

%SPEC%\config\Example-windows-amd64-visualstudio.cfg

but (as of July 2011) these will not compile 462.libquantum. Please read the comments carefully in the above config files, to understand their limitations.

Users of Microsoft Visual C++ should also please note that the config files from
results posted with the Intel C++ compiler are not appropriate for you, nor is the file %SPEC%\config\Example-windows-ia32-icl.cfg on your kit. The Intel compiler has additional switches
that the Microsoft compiler does not recognize, and it spells the compiler name differently ("icl" vs.
"cl").

./libutil/prim_type.h", line 65.25: 1506-334 (S)
Identifier int8 has already been defined on line 618 of "/usr/include/sys/inttypes.h

a. The above symptom may occur if your compiler assumes that char variables are unsigned.
The source code for 482.sphinx3 assumes that they are signed. Please see the section About
Signed Char, below.

Setting up

Setup.01 q. What does hash doesn't match after copy mean?

I got this strange, difficult to reproduce message:
hash doesn't match after copy ... in copy_file (1 try total)! Sleeping 2 seconds...followed by several more tries and sleeps. Why?

a. During benchmark setup, certain files are checked. If they don't match what they are expected to,
you might see this message. Check:

Are you out of disk space?

Does the error go away if a different disk is used? Perhaps you have a bad disk, or an unreliable disk controller.

If the condition persists, try turning up the verbosity level. Look at the files with other tools; do they exist? Can you see differences?
Try a different disk and controller. And, check for the specific instance of this message described in the next question.

Setup.02q. Why does it say ERROR: Copying executable failed?

I got this strange, difficult to reproduce message:
ERROR: Copying executable to run directory FAILEDor
ERROR: Copying executable from build dir to exe dir FAILED!along with the bit about hashes not matching from the previous question. Why?

a. Perhaps you have attempted to build the same benchmark twice in two simultaneous jobs.

On most operating systems, the SPEC tools don't mind concurrent jobs. They use your operating system's
locking facilities to write the correct outputs to the correct files, even if you fire off many runspec commands
at the same time.

But there's one case of simultaneous building that is difficult for the tools to defend against: please
don't try to build the very same executable from two different jobs at the same time. Notice that if you say something
like this:

a. Several of the benchmarks use a common random number generator. During development of CPU2006, it
was often useful to have the random number generator as a separate pseudo-benchmark, to help with problem diagnosis.
("You miscompared on benchmark mumble? Hmmm. Do you also miscompare on specrand?")

For the released version of the suite, SPEC decided to retain specrand, in case it comes in
useful for later problem diagnosis. It is run with both the integer and floating point suites, but its time is not
reported, and its performance does not contribute to the bottom line metrics.

Run.02
q. Why does this benchmark suite take so long to run?

a. Please understand that the suite has been designed to be useful for at least 5 years. Benchmarks that
seem slow today probably will not seem slow at the end of life of the suite. You can see a bit more on this topic at question
17 of readme1st.html.

Run.03 q. Why was there this cryptic message from the operating system?

a. If you are getting cryptic, hard-to-reproduce, unpredictable error messages from your system, one possible
reason may be that the benchmarks consume substantial resources, of several types. If an OS runs out of some resource - for
example, pagefile space, or process heap space - it might not give you a very clear message. Instead, you might see only a
very brief message, or a diaolog box with a hex error code in it. Please see the hints and suggestions in the section about
resources in system-requirements.html.

It was reported that under Windows7 with SP1, a user saw SPEC CPU2006 benchmark failures
and this dialog box: which, after pressing the button, was followed by:

The problem was difficult to diagnose, because the problem sometimes moved around (showing up at
different stages of the run), with slight variations in the symptoms (e.g. which dialog boxes appeared). Eventually, it was
noticed that an older (pre-Windows 7) version of cygwin was present in the %PATH%; removing
cygwin from the %PATH% removed the failures.

Because the version of cygwin was obsolete, it would be unfair to "blame" it for the
failures!

Nevertheless, it may be fair to point out that providing a Unix-like environment on Windows poses
difficult problems. Historically there have been various approaches, with differing (incompatible) assumptions about how
to mask or bridge differences between Windows and Unix. The SPEC CPU toolset has its own approach and its own set of
assumptions, and this is not the first report of a difficult-to-diagnose error when a Windows/Unix compatibility product is
present on the path.

Therefore, SPEC recommends that, in general, you should remove items from your path that are not needed,
and, in particular, that you should remove Windows/Unix compatibility products from the %PATH% prior to invoking
runspec.

If you see odd error messages such as the above, please remove any such products from the PATH prior to
requesting support from SPEC.

New with CPU2006 V1.2, the utility printpath is documented, with an example of using it for the purpose of studying the path and
then removing cygwin.

Run.05 464.h264ref loops until killed using certain optimizations with a GCC
snapshot. 416.gamess also exhibits the problem.

Certain pre-release snapshots of GCC V4.8, when used to compile 464.h264ref with certain optimizations, will create an
executable that loops indefinitely.

For example GCC snapshot 4.8-20130106 with:

'-O2 -fno-strict-aliasing'

will create the problem.

UPDATE 5-Feb-2013 it has been noted that similar problems may occur with 416.gamess.

a. Analysis: 464.h264ref reference beyond end of array

The looping behavior is believed to be due to 464.h264ref file mv-search.c line 1093, where there is a code pattern that appears to read
beyond the end of an array, in the line

for (dd=d[k=0]; k<16; dd=d[++k])

No value is written outside the bounds of 'd'.

No Source Change Planned

It has been suggested that SPEC should change the code in the benchmark. At this time, SPEC does not intend to do so.

Although the code pattern does not follow the C99 standard (See section 6.5.6/8), it is at minimum arguable that the code
fragment is compliant under the C89 standard. The original code as provided by the International Telecommunications
Union contained the pattern.

Because the SPEC CPU benchmarks are drawn from the compute intensive portion of real applications, some of them use
popular practices that compilers must commonly cater for, even if those practices are nonstandard. In particular, some of
the programs (and, therefore, all of base) may have to be compiled with settings that do not exploit all optimization
possibilities that would be possible for programs with perfect standards compliance.

There are non-standard coding practices in at least 400.perlbench and 416.gamess. SPEC understands that these are present, but has
included the benchmarks in the suite to represent common code in use.

During development of new benchmark suites, SPEC welcomes participation by interested parties from academia, industry, and
vendors. During the development process, prior to release of the benchmark, SPEC always prefers to resolve portability
problems by improving the standards compliance of candidate benchmarks.

Nevertheless, it must be recognized that SPEC CPU suites are not intended as a language standard compliance test; and, once a
benchmark has been released, SPEC's greater concern is that changes not be introduced which might affect performance
neutrality.

Workaround

When compiling 464.h264ref, do not use optimization switches
that cause it to loop indefinitely. SPEC does not have a specific recommendation as to which switches to use; you may wish
to consult your compiler documentation. Reminder: in base, all programs of a given language must be compiled with the same
switches, therefore choices made for 464.h264ref may affect the performance of other programs in base.

UPDATE 5-Feb-2013 it has been suggested in the bugzilla
reports (5308653073)
at gnu.org that the switch -fno-aggressive-loop-optimizations may work around the issue.

Miscompares

Miscompare.01 I got a message about a miscompare. The tools said something
like:

a. We don't know. Many things can cause a benchmark to miscompare, so we
really can't tell you exactly what's wrong based only on the fact that a miscompare occurred.

But don't panic.

Please notice, if you read the message carefully, that there's a suggestion of a very specific file to look in. It may be a little hard to read if you have a narrow terminal window, as in the example above, but if you look carefully you'll see that it says:

*** Miscompare of rand.234923.out, see
/spec/cpu2006/benchspec/CPU2006/999.specrand/run/run_base_ref_oct09a.0000/rand.234923.out.mis

Now's the time to look inside that file. Simply doing so may provide a clue as to the nature of your
problem.

On Unix systems, change your current directory to the run directory using the path mentioned in the
message, for example:

Then, have a look at the file that was mentioned, using your favorite text editor. If the file does not
exist, then check your paths, and check to see whether you have run out of disk space.

Miscompare.02 The benchmark ran, but it took less than 1 second and there was
a miscompare. Help!

a. If the benchmark took less than 1 second to execute, it didn't execute properly. There should be one
or more .err files in the run directory which will contain some clues about why the benchmark failed to run.
Common causes include libraries that were used for compilation but not available during the run, executables that crash
with access violations or other exceptions, and permissions problems. See also the suggestions in the next question.

Miscompare.03 I looked in the .mis file and it said something
like:

'rand.234923.out' short

What does "short" mean?

a. If a line like the above is the only line in the .mis file, it means that the benchmark
failed to produce any output. In this case, the corresponding error file (look for files with .err extensions in
the run directory) may have a clue. In this case, it was Segmentation Fault - core dumped. For problems like
this, the first things to examine are the portability flags used to build the benchmark.

Have a look at the sample config files in $SPEC/config or, on Windows, %SPEC%\config. If you constructed your own config file based on one of those, maybe you picked a
starting point that was not really appropriate (e.g. you picked a 32-bit config file but are using 64-bit compilation options).
Have a look at other samples in that directory. Check at www.spec.org/cpu2006 to see if there have been any result submissions using the
platform that you are trying to test. If so, compare your portability flags to the ones in the the config files for those
results.

If the portability flags are okay, your compiler may be generating bad code.

Miscompare.04 My compiler is generating bad code! Help!

a. Try reducing the optimization that the compiler is doing. Instructions for doing this will vary from
compiler to compiler, so it's best to ask your compiler vendor for advice if you can't figure out how to do it for
yourself.

Miscompare.05 My compiler is generating bad code with low or no optimization! Help!

a. If you're using a beta compiler, try dropping down to the last released version, or get a newer copy
of the beta. If you're using a version of GCC that shipped with your OS, you may want to try getting a "vanilla" (no
patches) version and building it yourself.

Miscompare.06 I looked in the .mis file and it was
just full of a bunch of numbers.

a. In this case, the benchmark is probably running, but it's not generating answers that are within the
tolerances set. See the suggestions for how to deal with compilers that generate bad code in the previous two questions.
In particular, you might see if there is a way to encourage your compiler to be careful about optimization of floating
point expressions.

Miscompare.07 q. Why does 464.h264ref fail when it is run, printing *** Miscompare of
foreman_test_baseline_encodelog.out?

When I run the benchmark 464.h264ref, I see messages like these:

*** Miscompare of foreman_ref_baseline_encodelog.out; for details see
*** Miscompare of foreman_test_baseline_encodelog.out; for details see
*** Miscompare of foreman_train_baseline_encodelog.out; for details see

followed by a pointer to a file that contains a list of miscompares. The first line to miscompare is
typically line 15:

a. The above symptom may occur if your compiler assumes that char variables are unsigned. The
source code for 464.h264ref assumes that they are signed.

About Signed Char: If you are experiencing the symptoms listed under
Miscompare.07,
Miscompare.08, or
Build.05,
please check your compiler documentation for a compiler switch that enables signed char, such as:

-signed
-fsigned-char
-qchars=signed

Once you have found your compiler's switch for signed char, try adding that switch to your
PORTABILITY flags for the affected benchmark, to see if it solves the problem.

If you wish to generate reportable results and use them in public, then you should check whether SPEC has already approved
use of the flag. You can use the results search tool, www.spec.org/cgi-bin/osgresults, to look for published results from
hardware/software platforms that are similar to your platform.

a. The above symptom may occur if your compiler assumes that char variables are unsigned.
The source code for 482.sphinx3 assumes that they are signed. Please see the section About
Signed Char, above.

Results reporting

Results.01
q. Where did the reference times go?

I was reading an HTML (or PDF or PS) report and see that in the table SPEC now prints all three
observations as both seconds and ratio, with the bold underlined item for the median. Although this is useful, CPU2000
results reports included the reference times in their own column, and I miss them. Yes, I understand that the ratio
is defined as reference time / observed time, so I could calculate the reference times by
simple multiplication. Nevertheless, it was nice having them handy in the previous report format.

a. They are still fairly easy to locate. The text format still prints them on every line (just like
CPU2000), so if you go up to the top of your browser and change the .pdf (or .whichever) to
.txt, you'll find the reference times.

Results.02
q. It's hard to cut/paste into my spreadsheet

a. Please don't do that. With CPU2006, there's a handy .csv format file right next to the
other result formats on the index page. Or, you can go up to the top of your browser and change the .pdf (or
.whichever) to .csv

Results.03q. What is a "flags file"? What does the message Unknown Flags mean in a report?

a. SPEC CPU2006 provides benchmarks in source code form, which are compiled under control of SPEC's
toolset. Compilation flags (such as -O5 or -unroll) are detected and reported by the tools with the
help of flag description files. Therefore, to do a complete run, you need to (1) point to an existing flags file (easy)
or (2) modify an existing flags file (slightly harder) or (3) write one from scratch (definitely harder).

Find an existing flags file by noticing the address of the .xml file
at the bottom of any result published at www.spec.org/cpu2006. You
can use the --flagsurl switch to point your own runspec command
at that file, or you can reference it from your config file with the flagsurl option. For example: runspec --config=myconfig --flagsurl=http://www.spec.org/cpu2006/flags/sun-studio.xml int

You can download the .xml flags file referenced at the bottom of any
published result at www.spec.org/cpu2006. Warning: clicking on the .xml link may just confuse your web browser; it's probably better to use
whatever methods your browser provides to download a file without viewing it - for example, control-click in Safari, right
click in Internet Explorer. Then, look at it with a text editor.

format: Submission Check -> FAILED. Found the following errors:
- The "hw_memory" field is invalid.
It must contain leading digits, followed by a space,
and a standard unit abbreviation. Acceptable
abbreviations are KB, MB, GB, and TB.
The current value is "20480 Megabytes".

a. A complete, reportable result has various information filled in for readers. These fields are listed in
the table of contents for config.html. If you wish to submit a result to
SPEC for publication at www.spec.org/cpu2006, these fields not only
have to be filled in; they also have to follow certain formats. Although you are not required to submit your result to SPEC,
for convenience the tools try to tell you as much as they can about how the result should be improved if you were to submit it.
In the above example, the tools would stop complaining if the field hw_memory said something like "20 GB" instead of
"20480 Megabytes".

Notice that you can repair minor formatting problems such as these without doing a re-run of your tests. You
are allowed to edit the rawfile, as described in utility.html.