A previous method that I tried was treating the helped and hurt data
as samples from separate populations, and these were compared using a
T-test. Since we're applying a common change to "both" sample sets, I
don't think this is a valid analysis.
Instead, I think it is more valid to look at the entire change set as a
sample of a single population and compare the mean of that sample to
zero. Only the changed samples are examined because the vast majority
of the sample in unaffected. If the mean of the entire sample was used,
the mean confidence interval would always include zero. It would be
more valid, I believe, include shaders that were affected but had no
change in instruction or cycle count. I don't know of a way to
determine this using the existing shader-db infrastructure.
These two different methods communicate two different things. The first
tries to determine whether the shaders hurt are affected more or less
than the shaders helped. This doesn't capture any information about the
number of shaders affected. There might be 1,000 shaders helped and 3
hurt, and the conclusion could still be negative. The second methods
trieds to determine whether the sample set is overall helped or hurt.
This allows the magnitued of hurt (or help) to be overwhelmed by the
number of helped (or hurt) shaders. There could be 1,000 shaders helped
by 1 instruction and 3 shaders hurt by 50 instructions, and the
conclusion would be positive.
Comparing the declared result with the mean and median, I feel like the
second method matches my intuitive interpretation of the data. Here is
a result of the T-test:
total cycles in shared programs: 559379982 -> 559342256 (<.01%)
cycles in affected programs: 10791218 -> 10753492 (-0.35%)
helped: 1952
HURT: 908
helped stats (abs) min: 1 max: 5762 x̄: 37.71 x̃: 16
helped stats (rel) min: <.01% max: 28.57% x̄: 3.54% x̃: 2.09%
HURT stats (abs) min: 1 max: 573 x̄: 39.51 x̃: 10
HURT stats (rel) min: <.01% max: 27.78% x̄: 1.93% x̃: 0.66%
abs t: -0.34, p: 73.70%
rel t: 9.88, p: <.01%
Inconclusive result (cannot disprove both null hypothoses).
And here is the result of the mean confidence interval tests on the
same data:
total cycles in shared programs: 559378112 -> 559340386 (<.01%)
cycles in affected programs: 10791218 -> 10753492 (-0.35%)
helped: 1952
HURT: 908
helped stats (abs) min: 1 max: 5762 x̄: 37.71 x̃: 16
helped stats (rel) min: <.01% max: 28.57% x̄: 3.54% x̃: 2.09%
HURT stats (abs) min: 1 max: 573 x̄: 39.51 x̃: 10
HURT stats (rel) min: <.01% max: 27.78% x̄: 1.93% x̃: 0.66%
95% mean confidence interval for cycles value: -18.27 -8.11
95% mean confidence interval for cycles %-change: -1.98% -1.63%
Cycles are helped.
Since the confidence interval is calculated based on the sample mean and
the sample standard deviation, it can include values out side the sample
minimum and maximum. This can lead to unexpected conclusions. In this
case all of the affected shaders were helped, but the result is
inconclusive.
total instructions in shared programs: 7886959 -> 7886925 (<.01%)
instructions in affected programs: 1340 -> 1306 (-2.54%)
helped: 4
HURT: 0
helped stats (abs) min: 2 max: 15 x̄: 8.50 x̃: 8
helped stats (rel) min: 0.63% max: 4.30% x̄: 2.45% x̃: 2.43%
95% mean confidence interval for instructions value: -20.44 3.44
95% mean confidence interval for instructions %-change: -5.78% 0.89%
Inconclusive result (value mean confidence interval includes 0).
v2: Don't log statistics for spill or fills. Simplify T-test logging.
v3: Use confidence interval instead.
Acked-by: Jason Ekstrand jason@jlekstrand.net

Compiler wasn't happy about strncpy() depending on the source's length and not
the target's, but this whole thing can be simplified by simply using asprintf()
(which is already used in this file, so no #include necessary).
run.c: In function ‘main._omp_fn.0’:
run.c:964:21: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=]
strncpy(out_filename, current_shader_name,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
strlen(current_shader_name) + 1);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>

Modern kernels support I915_PARAM_HAS_CONTEXT_ISOLATION, which allows
us to whack certain context registers with impunity. One of those is
INSTPM/CS_DEBUG_MODE2, which controls whether Constant Buffer 0 is
relative to dynamic state base address, or a general GPU address.
If it's relative (the default), we can only push 3 ranges of UBOs.
If it's absolute, we can push 4. So this affects our shader output.
Follow the behavior of modern kernels and allow us to push all 4 ranges.
Tested-by: Ian Romanick <ian.d.romanick@intel.com>

This does two things:
1. Allows cross shader optimisations to be preformed on SSO
programs that contain more than one stage (seems like I
don't have any of these in my shader-db collection).
2. Allows us to write compilation errors to stderr. With this
change I discovered a Deus Ex: MD shader was failing to
compile (I've reported it to Feral).
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

Optional binding of variables can be processed before linking shader
objects for creating shader program. It is activated by adding lines
with a keyword "BindAttribLoc" followed by name and index as,
"BindAttribLoc name_str1 <index1>"
For example,
[require]
......
BindAttrbLoc vertex 1
BindAttrbLoc coord 2
BindAttrbLoc col 3
This makes the shader-db run
glBindAttribLocation(p, 1, "vertex");
glBindAttribLocation(p, 2, "coord");
glBindAttribLocation(p, 3, "col");
before glLinkProgram() to include these binding info in binary shader
program.
v2: get_shaders returns its own head of list for binding variables
instead of using a global head to support parallel processing of
multiple shader_tests
Signed-off-by: Dongwon Kim <dongwon.kim@intel.com>

I don't think anybody is using this script with any other driver.
Also take a $DEBUGGER variable so you can do:
$ DEBUGGER='gdb -q --args' ./intel_run shaders
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>

With option '-b', shader-db now generates a shader program binary file
using GetProgramBinary(). This shader program binary can be loaded via
ProgramBinary() to be executed by an application later.
v2: 1. define MAX_LOG_LEN and use it as the size of gl log
2. define MAX_PROG_SIZE and use it as the max size of extracted
shader_program
3. out_file is now pointer allocated by strdup for the file name
v3: 1. automatically using original shader test file's name + ".bin"
as a filename for program binary - better way to cover the case
with batch compilation of many shader test files in the same
directory
2. remove --out=<file name> since it is now unnecessary (due to v3-1.)
to provide custom file name. Instead, option, "--bin", which is
basically a flag that enables getting program binary as a file.
3. Now it tries to get the length of binary by reading program's
GL_PROGRAM_BINARY_LENGTH_OES parameter
v4: 1. '--bin' -> '-b'
2. stop generating binary program when failing to retrieve the binary
size
3. error checking after malloc for binary program
4. changed some of variable names
5. several consecutive fprintfs are consolidated
6. removed MAX_LOG_LEN and MAX_PROG_SIZE
v5: bug fix: +1 to the length of the output file to cover '\0'
Signed-off-by: Dongwon Kim <dongwon.kim@intel.com>

When using sha as part of the captured shader name that name can get so
long that part of it is cut off and not visible anymore (although this
also happens when using long directory names etc.) So this is a general
improvement.
Signed-off-by: Marek Olšák <marek.olsak@amd.com>

Since we need to create contexts on both the main thread and omp
threads, combine things into a single helper. This reduces some
duplication of logic in the following patches.
Reviewed-by: Matt Turner <mattst88@gmail.com>

If a field was not present, its value will default to 0.
This lets the code run on outputs that did not include the amount of
shared memory used, for example.
Signed-off-by: Pierre Moreau <pierre.morrow@free.fr>

memmem() does not attribute what the character after the searched string
is. Thus it will flag even when haystack is "foobar" while we're looking
for "foo".
Pull a small helper (from piglit) that correctly handles this and use
it.
v2: Drop unintentional whitespace changes (Eric)
Reviewed-by: Eric Engestrom <eric.engestrom@imgtec.com>
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>

The same declaration was being used for two distinct things - a short
EGL and the core GL extensions.
In the former it is a const string, while the latter is a dynamically
managed list of extensions.
Define distinct variables within the specific scope.
v2: use {egl,gl}_extension_string (Eric)
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>

These shaders have been generated by Dolphin 9649494f67 on Mesa
8c26b52349 for an HD4000 GPU.
They include a lot of uniform branches, mostly on integers, as well as
switch statements branching on small and bounded integers.
Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>

The upstream version of Orbital Explorer doesn't run on Mesa drivers,
as it (unnecessarily) requests a compatibility profile on non-Apple
OSes, and suffers from GLEW bugs. I sent a pull request two years ago
to port it to libepoxy and get it running on Mesa drivers, but it was
never accepted.
The author eventually emailed me and said that he considers it a
"finished experiment" and said the rendering method (geometry shader
based approach) is inefficient, and he intends to fully rewrite it
someday.
Since this is not representative of typical geometry shaders, and
the project is effectively dead, we may as well remove it.
Reviewed-by: Chad Versace <chadversary@chromium.org>

When GALLIUM_THREAD is already set in the environment, addenv()
will end up by setting GALLIUM_THREAD="0,0" which mesa doesn't
understand for some obvious reasons. Instead, use setenv()
because we want to force this behaviour in all situations.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>