Hooks for interactive use

Programs using 3bgl-shader can add a function to
3bgl-shaders::*modified-function-hook*, which will be called when
shader functions are redefined. It will be passed a list of names of
updated functions. For example in the shaders above, if the
(defconstant +scale+ 2.0 :float) form were recompiled in slime with
C-c C-c, the hook functions would be passed the list
(SHADER::INVERTED-FRAGMENT SHADER::INVERT-AND-SCALE) since
invert-and-scale depends on the constant, and inverted-fragment
depends on the function invert-and-scale. The hook function could
then see one of the fragment shaders it is using had been modified,
and arrange for a running program to try to recompile the shader
program next frame.

Current status

The compiler and type inference mostly work, including some fairly
complicated shaders.

Built-in functions/types/variables from glsl version 4.50 are
available, and older versions might work to the extent they are
compatible with the subset of 4.50 used in a particular shader. The
type inference doesn't distinguish between versions, so might allow
casts that wouldn't be valid in an older version.

CL style type declarations are allowed and should mostly be respected.

API isn't completely finished, so some parts may change (in particular
the base types like :vec3 may be renamed to 3bgl-glsl:vec3 at some
point.)

The external API needs more work, in particular some way to query
uniforms, inputs, outputs, etc.

Currently no way to translate line/column numbers from glsl error
messages back to source.

Performance is acceptable for shaders I've tested it on, but not sure
how it scales. It currently WARNs if it takes more than 2000 passes
for type inference, which may need adjusted or disabled for larger shaders.

Currently all functions that depend on a function/global will be
recompiled when things they depend on are recompiled, which can make
changing function signatures or types difficult if they aren't
compatible with the uses.

Recompilation may be more aggressive than it needs to be, for
example if the value of a constant is changed, it shouldn't need to
re-run type inference of functions that use that constant if the type
didn't change.

Dependencies on uniforms are sometimes missed, dumping a bare
reference to it in main function is a simple workaround.

Misc notes

Concrete types

GLSL types are currently named with keywords (though that may change
in the future), like :vec2, :vec3, :vec4, :mat2x4,
:sampler-2d-array-shadow etc. see the
source
for details for now, though most are fairly obvious.

Component swizzles

Components of GLSL vector types like :vec4 can be accessed with
'swizzle' functions like .xyz, so for example glsl someVec.rraa
would be (.rraa some-vec). Type inference should correctly use the
swizzle to determine minimum size of the vector if not specified.

Structure/interface slots

(@ var slot-name) is a shortcut for (slot-value var 'slot-name),
and either will compile to var.slot. GLSL doesn't allow specifying a
slot through a variable, so slot name must be a quoted compile-time
literal.

RETURN

Functions are required to use RETURN to return values, they will not
return the value of the last form as in CL. A function without a
RETURN will have a void return type. (return (values)) can also
be used to force a void return type, and for early exit from a
void function.

Overloaded functions

If a function doesn't have a specific derived or specified type, it
can be used with any compatible types, and the generated GLSL will
have a version for each type.

specifies that foo returns a float, the first argument is also
specified to be float, while the second isn't explicitly
restricted. The local variable A is specified to be a vec2, which
implicitly restricts Y to also be something that casts to vec2.

(declare (values)) can be used to explicitly specify void return
type for a function.

Uniforms, input, output, interface

Uniforms are specified with (UNIFORM name type &key stage location layout qualifiers).:stage specifies in which shader stages (:vertex,:fragment etc)
the uniform is visible (by default the uniform is visible in all
stages, though will only be included in generated GLSL for stages in
which it is referenced).:location N is a shortcut for specifying the location layout qualifier.:layout (...) allows specifying arbitrary layout qualifiers, argument is a plist containing qualifier and value (specify value = t for qualifiers that don't take arguments)
:qualifiers (...) allows specifying other qualifiers like restrict, argument is a list of qualifiers.

Inputs and outputs are specified with (INPUT name type &key stage location)
and (OUTPUT name type &key stage location)
where stage specifies in which shader stages (:vertex,:fragment
etc) the input is visible, and location is an integer which will be
output as layout(location = 1) in GLSL.

Interfaces between stages are specified as (INTERFACE name (&key in out uniform) &body slots). slots is a list of (slot-name type). in, out and uniform specify how the interface will be
visible, and are either T to make it visible to all stages as
name, or a plist of stage names and names to use for the interface in that stage.

For example (interface varyings (:out (:vertex outs) :in (:fragment ins :geometry (ins "ins" :*))) ...) will be visible as an output
named out in the vertex shader, as an input array named ins in the
geometry shader, and as an input named ins in the fragment shader.

name and slot-name in uniform/input/output/interface can either be
a symbol which will be automatically converted from lisp-style to
glslStyle, or it can be a list of (lisp-name "glslName") to
provide an explicit translation.

Running the example programs

Example program uses GLUT and GLU, and expects GLSL version 330.
Most lisp dependencies should be available in quicklisp, aside from possibly mathkit.

Load 3bgl-shader-example.asd through ASDF or Quicklisp, then run
(3bgl-shader-example:run-example). That should create a window with
a spinning teapot, hit 0-5 keys to try the various example
shaders.

If that is working, you can open example-shaders.lisp in emacs and
edit them and recompile as usual from slime (C-c C-c etc).

Getting names of uniforms/vertex attributes

In addition to generated GLSL source, GENERATE-STAGE returns a list
of uniforms as 2nd value, and attributes in 3rd value. Both are in
form (lisp-name "glslName" TYPE) for each entry. There isn't
currently any dead-code elimination, so listed names may not actually
be active in the final shader program.

Macros

DEFMACRO and MACROLET work as in CL code, and expansion runs on
host so can use arbitrary CL.

Array variables

There is partial support for arrays, though type inference doesn't
work completely correctly on them and local array variables can't be
initialized when bound.

Currently, array types are specified as (<base-type> <size>). (CL
style array/vector types may be supported at some point in the future)

Compute Shaders

Compute shaders work pretty much like other stages, except you can't
specify inputs/outputs, and must specify the workgroup size for
kernel invocations. The workgroup sizes are specified with the
layout declaration on the main kernel entrypoint. Compute shaders
also expose a number of constants describing an individual
invocation's relationship to the entire run: gl-local-invocation-id, gl-global-invocation-id, gl-work-group-id, gl-num-work-groups, and gl-work-group-size, all :uvec3, and gl-local-invocation-index, an :int.

5.1.1 Special variables

list of functions to call when shader functions are
modified. Passed a list of names of functions that have been
modified. May be called multiple times for same function if a whole
file using the 3bgl-glsl:defun macro is recompiled, so probably should
store names and only update shader programs at next frame rather
than updating programs directly from hook function.

5.1.3 Functions

Function: compile-formFORM

Run first passes of compilation on specified form. (Wrap with PROGN
to process multiple forms). Calls functions in
*MODIFIED-FUNCTION-HOOK* with names of any functions whose definitions
are possibly affected by compiling FORM (for example functions that
call a function defined/updated by FORM, and the (re)defined function
itself).

Generate GLSL shader for specified STAGE, using function named by
MAIN as glsl ’main’ function. ROOT and all functions/variables/etc it
depends on should already have been successfully compiled with
COMPILE-FORM. STAGE is :VERTEX, :FRAGMENT, :GEOMETRY, :TESS-EVAL,
:TESS-CONTROL, or :COMPUTE. VERSION specifies the value of the version
pragma in generated shader, but doesn’t otherwise affect generated
code currently. Returns a list of active uniforms in the
form (LISP-NAME "glslName" type . PROPERTIES) as second value, and a
list of active attributes in same format as third value. (GL shader
types like :VERTEX-SHADER are also accepted for STAGE)

For uniforms, PROPERTIES is a plist containing 0 or more of:

:COMPONENTS : (when EXPAND-UNIFORMS is true) for composite
uniforms (structs, etc), a list containing a list of uniform name and
slot names or array indices for each leaf uniform represented by the
type, for example a struct uniform containing an array of structs
might have entries that look like (foo bar 1 baz) corresponding to the
uniform "foo.bar[1].baz".