Cycling 74 » texturehttps://cycling74.com
Makers of Max Visual Programming Software for Artists, Musicians & PerformersTue, 31 Mar 2015 17:52:40 +0000en-UShourly1http://wordpress.org/?v=3.8.1Jitter Recipes: Book 2, Recipes 14-25https://cycling74.com/2006/02/14/jitter-recipes-book-2/
https://cycling74.com/2006/02/14/jitter-recipes-book-2/#commentsTue, 14 Feb 2006 19:17:52 +0000http://veryhandso.me/?p=1397The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses.

]]>The second installment of Jitter Recipes is a collection of simple examples that began as weekly posts to the MaxMSP mailing list. Here you will find some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing. The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses. I encourage you to take them all apart, add in your own touches and make these your own.

Recipe 14: JitBuffer Exposed

General Principles

Commentary

This example shows a couple of techniques that take advantage of the matrix interface of jit.buffer~.

Ingredients

jit.buffer~

jit.fill

jit.slide

jit.matrix

Technique

On the left half we are recording a live signal into the upper buffer. This buffer is then output as a matrix, and fed through the jit.slide object to slowly fade between buffer states. This faded buffer-matrix is then loaded into the second jit.buffer~, which is what we will use for playback

On the right half, we use a multislider and jit.fill to provide an easy drawing interface for waveshaping. Inside of the “use exprfill” subpatch, we’ve provided several common expressions that can be used to fill a jit.matrix with a windowing curve. This is particularly useful for doing granular synthesis.

Recipe 15: GestureRecord

General Principles

Recording gestural parameter data over time

Commentary

When recording high resolution videos from Jitter, it is often very difficult to get acceptable results from real-time manipulation of parameters. For this reason, it is often helpful to record real-time parameter changes on a frame-by-frame basis, and then play back these parameters at a rate that jitter can keep up with.

Ingredients

pattrstorage

autopattr

jit.coerce

jit.qt.movie

jit.qt.record

Technique

On the left half, we are using the pattrsorage object to save the state of all the named UI objects for each “frame”. The benefit of this implementation is the ease and robustness of the pattr-family of objects as well as the ability to save the state of objects such as matrixctrl.

The right half uses a matrix to store our parameter values, and then records frames into a jit.qt.record object.

Note that we’re using jit.coerce to force our float32 matrix into an acceptable format for QuickTime (char).

Recipe 16: VideoSynth2

General Principles

Using Jitter to create generative video

Using Matrix feedback and distortion to create movement

Commentary

This example is functionally very similar to the VideoSynth1 patch, except that we’re using jit.repos instead of jit.rota to do the image distortion.

Ingredients

jit.poke~

jit.matrix

jit.dimmap

jit.glue

jit.xfade

Technique

We are using jit.poke~ to perform automatic drawing on a named matrix (see VideoSynth1)

This matrix is then distorted using jit.repos and then fed back to be drawn again using our named matrix.

To generate a constantly shifting repos-matrix, we’ve implemented a similar tactic as that used for the drawing (see bot2 subpatch). This matrix is then mixed with a matrix of normalized coordinates generated by the jit.expr object. This has the effect of gently repositioning the matrix at each frame, thus giving us smooth movement.

The output of jit.repos is then mirrored by sending it to jit.glue and inverting the x-dimension of the right input using jit.dimmap.

Recipe 17: TwineBall

General Principles

Using the stroke messages to jit.gl.sketch to render curves in OpenGL

Using a matrix to generate sketch commands for procedural drawing

Commentary

With the introduction of JSUI, “stroke” commands were exposed for generating spline-curves. While this is pretty well-documented in the JavaScriptinMax PDF, many people aren’t aware that you can also use these commands with jit.gl.sketch to do curvy line drawings.

Ingredients

jit.gl.sketch

jit.matrix

jit.iter

jit.op

Technique

The movement of the points is generated by adding a matrix of motion-vectors (see ParticleRave-a-delic)

This matrix of point-locations is then sent to jit.iter to be broken down into 3-part lists.

For every matrix that is sent, we reset our sketch object and then begin our stroke using the “beginstroke line” message, and specifying the order (or smoothness of our curve) as well as color and width.

Each point of the matrix is then sent using the “strokepoint….”

Recipe 18: Manipulator

General Principles

Using expressions to generate values across a matrix

Texture Mapping

Reshaping a NURBS by setting control-point coordinates

Commentary

Sometimes, when learning the ins and outs of OpenGL, it helps to be able to see just how this matrix of vertices is affecting our geometry. This example uses jit.cellblock to provide a visual interface to our matrix data. This allows us to perform precise distortions of our surface, which can be very useful for generating interesting graphics.

Ingredients

jit.cellblock

jit.matrix

jit.gl.slab

jit.pack

jit.gl.nurbs

jit.qt.movie

Technique

Using the “exprfill” message, we load our first two planes with signed-normalized values across the x and y dimension respectively. This gives us values between -1. and 1. By filling these matrices in such a way, we create a plane across the x and y coordinates of our GL-world.

The z-plane is left empty for us to fill in as desired to create spatial distortion.

All of these matrices are then packed into one 3-plane matrix and sent on as the “ctlmatrix” for jit.gl.nurbs

We use jit.gl.slab to map a movie onto our jit.gl.nurbs shape. We could also load a shader program into this object to do processing on our texture.

Recipe 19: Tannenbaum

General Principles

Creating one’s own particle system using matrix operations

Rendering multiple copies of the same geometry

Manipulating text using matrix operations

Holiday visuals

Commentary

This patch was published for Xmas 2005, which should explain its holiday aesthetic. A particle matrix is generated which is then rendered as floating text-strings as well as vertices of a jit.gl.mesh. The text is also messed with a bit using some matrix processing.

Ingredients

jit.str.fromsymbol

jit.gl.text2d

jit.gl.mesh

jit.matrix

jit.poke~

jit.op

jit.expr

jit.map

Technique

The motion of our particle system is generated by our “motion” subpatch, which manipulates the values of our motion-vector matrix “season”. This matrix is then scaled down using jit.map, and then added to our position-matrix “cheer” (see Particle-Rave-a-delic). The jit.expr object is used to kill off old particles and reset them to 0. using the fourth plane of “cheer”.

The text sprites are generated by using jit.str.fromsymbol to convert the text to a matrix, which is then fed into a jit.op object to get randomized every three seconds. This matrix is then sent on to the jit.gl.text2d object to be rendered as bitmapped text on an OpenGL plane. This is repeated for every cell of the “cheer” matrix (see gLFO).

The white strip is generated by simply passing “cheer” to our jit.gl.mesh object with @draw_mode set to tri_strip.

Recipe 20: Squelcher

General Principles

Using video to create synthesis parameters

Creating audio-rate sequencing of parameters

Commentary

This example is the result of some after-hours patchery around the Cycling ’74 office. If you still haven’t figured out why you’d want to bother with a jit.buffer~, this patch should be all the convincing you’ll need. The easy interface between video and audio makes for some pretty unpredictable and intriguing effects.

Ingredients

jit.buffer~

jit.altern

jit.brcosa

jit.rgb2luma

jit.qt.grab

jit.matrix

jit.fill

jit.scissors

jit.pack

sync~,rate~, and all sorts of MSP objects

Technique

Using jit.qt.grab, we take the input of our camera, which is downsampled to 64×4 pixels, as this is all the data we will need. This video matrix is then sent into our “buffer2″ subpatch for processing.

Using jit.altern to create periodic breaks and jit.brcosa to manipulate the brightness and contrast, we manipulate our video data to give us more desirable results. This matrix is then converted to a 1-plane luminance matrix,broken up into 4 rows using jit.scissors, and then packed into a 4-plane matrix. This is then used to create a 4-channel jit.buffer~

A second jit.buffer~ is filled using the familiar multislider-jit.fill combo (see JitBufferEx).

Rather than playing these buffers back as waveforms, we instead use them as CV-like controls for our synthesizer. These controls can be assigned using the matrixctrl UI.

Inside the “synth” subpatch we find all of our MSP processing, which I leave to your own experimentation.

Recipe 21: SceneProcess

General Principles

Capturing OpenGL geometry to a texture

Using GLSL shaders to perform efficient video processing

Adding video effects to a live OpenGL patch

Commentary

One of the downfalls of using OpenGL can be its overly sharp look, which is difficult to combat without doing some video processing. Unfortunately, rasterizing OpenGL back to the CPU can blast your FPS. Luckily, with Jitter 1.5 we have the ability to do processing on the GPU using GLSL shader programs. By capturing our OpenGL geometry to a texture to a texture, we are able to apply video effects to our scene without losing FPS.

Ingredients

jit.gl.nurbs

jit.gl.texture

jit.gl.slab

jit.gl.videoplane

jit.expr

Technique

Our geometry is generated using jit.expr with an internal instance of jit.noise to create randomness. Note that we must turn off the @cache in order for our expression to be evaluated, since there is no input.

By using the @capture super, we tell jit.gl.nurbs to render to our jit.gl.texture named “super” instead of rendering to our render context.

Once we have the rasterized geometry in texture form, we can then use jit.gl.slab to process our scene.

Recipe 22: CatchNurbs

General Principles

Using audio to control OpenGL geometry

Commentary

This patch is a good demonstration of taking data in one form, and forcing it into a completely different form for the purpose of driving other processes. When doing work with transcoding audio into video, it helps to have several tricks like this.

Ingredients

jit.catch~

jit.matrix

jit.scissors

jit.pack

jit.op

jit.gl.nurbs

jit.slide

Technique

Our audio is converted to 1-D matrices by the jit.catch~ object and then downsampled and passed along to a matrix using “dstdim” messages (see TimeScrubber). The resulting matrix is then downsampled further, sliced into 3 columns and packed into a 3-plane named matrix.

From there, we use jit.slide to smooth the movement, and jit.op to scale the values to the desired range. This is then given to the jit.gl.nurbs object as a “ctlmatrix”.

Recipe 23: ElapseGraph

General Principles

Creating visual display of Audio data

Procedural drawing using OpenGL commands

Commentary

Often, when working with audio data, it helps a great deal to be able to visualize the data that is coming through your patch before deciding what to do with it. This example shows a way to visualize the envelope, or amplitude data of an audio signal, using jit.gl.sketch.

Ingredients

jit.poke~

jit.matrix

jit.iter

jit.gl.sketch

Technique

The absolute-value audio signal is drawn into the second(y) plane of our matrix using jit.poke~. The first(x) plane is generated using the “snorm[0]” expression.

For each matrix that we send out, we step through our drawing procedure. The trigger object is very helpful for this.

Our drawing prodedure works as follows:

reset the jit.gl.sketch

draw the background rectangle

draw the blue-line

draw the start-point at (-1.,-0.5,0.)

draw a line segment and point for each cell of the matrix

draw the end-point at (1.,-0.5,0.)

Note that we are using jit.gl.render with orthogonal rendering turned on. This allows us to do flat 2D drawing without perspective distortions.

Recipe 24: GLFeedBack

General Principles

Using the GPU to perform typical video effects

Capturing OpenGL scenes to textures

Commentary

With the SceneProcess example, we saw a way to render geometry to a texture using the capture attribute. This example demonstrates a way to set up the “to_texture” message for doing video feedback effects on an entire OpenGL scene.

Ingredients

jit.gl.videoplane

jit.gl.render

jit.gl.texture

jit.qt.movie

Technique

As is typically the case in Jitter, order of operations is very important here. We must make sure that everything is rendered to our scene before rasterizing it to a texture. Note that the “to_texture” message comes last.

We are using two different jit.gl.videoplane objects. One of these simply displays the output of our jit.qt.movie, with additive blending turned on. The other one will display the texture generated by capturing our scene.

You will notice that we must always render to a texture that has the same dimensions as our rendering context. To work around this, we have two different textures to switch between for fullscreen mode.

Recipe 25: RagingSwirl

General Principles

Performing manipulations of a 3-dimensional matrix

Using Audio signals to generate video

Matrix feedback

Visualizing 3-D data as a geometrical figure

Commentary

The mechanics of this patch are nearly identical to the VideoSynth1 patch applied to a 3-D matrix, with a very different effect. Through clever use of jit.dimmap and jit.rota, we are able to create complex motion in three dimensions. The resulting shape constantly shifts, swirls and mutates.

Ingredients

jit.gl.isosurf

jit.poke~

jit.rota

jit.dimmap

jit.matrix

Technique

The jit.poke~ drawing subpatch should be familiar from previous recipes (see AsteroidGrowths,VideoSynth1)

Because jit.rota will only perform 2D rotations, we need to use jit.dimmap to reassign our dimensions and then perform a second rotation.

We then need to replace our dimensions using jit.dimmap again. This matrix is then fed back to be drawn over in the next iteration.

After we perform our rotations, we do quad mirroring by using jit.rota in boundmode 4 (fold), and then set the jit.rota to double the x and y dimensions. We could also double the z dimension as well if we wanted to do octal mirroring.

Once our matrix has been prodded and twisted sufficiently, we send it off to be rendered using jit.gl.isosurf.

]]>So, you’ve finished the tutorials, you understand the basics of digital audio, and you can imagine using a jitter matrix for something. Perhaps you are looking for a couple of new recipes to expand your repertoire…

The following is a collection of simple examples that began as weekly posts to the MaxMSP mailing list. Here you will find some clever solutions, advanced trans-coding techniques, groovy audio/visual toys, and basic building blocks for more complex processing. The majority of these recipes are specific implementations of a more general patching concept. As with any collection of recipes, you will want to take these basic techniques and personalize them for your own uses. I encourage you to take them all apart, add in your own touches and make these your own.

Recipe OO: gLFO

General Principles

Controlling OpenGL with audio signals.

Rendering multiple copies of geometry with a single object.

Commentary

One of the most common questions with new jitter users is, “How do I bring audio over into the video world?” You’ll notice that many of the following examples deal with this question in one way or another. Using the OpenGL features of Jitter is a great way to create data visualizations of all kinds, and can be integrated easily with other video processes. In this weeks example, we will use a couple of low-frequency cycle~ objects to dynamically reposition spheres in 3-D space.

Ingredients

gl.render context

jit.poke~

jit.gl.gridshape

jit.iter

jit.matrix (named)

jit.slide

Technique

We need to provide a named matrix for our coordinates to be stored and altered. As we need (x,y,z) coordinates in floating point values, we use a 3 plane float32 matrix. In the video world, we are familiar with using red,green, and blue planes to convey color information. In openGL, these same planes refer to x, y, and z coordinates respectively.

We use jit.poke~ objects to load our named matrix with values.

jit.slide is used to smooth the movement of the balls between positions.

We use jit.iter to break our matrix down into 3-item lists, which are then converted to position messages. We are also banging our jit.gl.gridshape with each message. This creates a copy of our sphere for each cell of our matrix. You will see this technique whenever we need to create multiple copies of a jit.gl-object.

We need to make sure that the jit.gl.gridshape has @automatic 0, so that it will be rendered only when it is banged.

Recipe 01: Time Scrubber

General Principles

Recording vectors of audio to a jitter matrix.

Randomly resequencing these audio vectors for playback.

Commentary

When Jitter 1.5 shipped, I immediately set to work imagining all the things I could do with the new jitter audio features. This patch is the first answer I came up with. This example could be considered a basic recipe, in that it may not accomplish anything exciting on its own, but it outlines a general technique that can be applied to all sorts of different processes.

Ingredients

jit.catch~

jit.release~

jit.matrix(named)

counter,random objects

Technique

The jit.catch~ object grabs a vector of audio data and sends it out as a 1-D float32 matrix. By using “dstdim” messages to a jit.matrix object, we propagate these horizontal rows over the length of the matrix. This technique is the same as that used to create “slitscan” graphics.

Once we have this matrix full of audio vectors, we could do any number of things, such as running this through other jitter processing, interpolation, etc. Here we are using it as a buffer to randomly select audio vectors in our playback engine. To do this we select individual rows to send to the jit.release~ object.

The jit.release~ object is set to “varispeed” mode to allow pitch control with different metro speeds.

Recipe 02: VideoPillows

General Principles

Loading a 3D matrix with multiple frames of video.

Visualizing translucent volumes using jit.gl.volume.

Creating dynamic alpha masks for transparency.

Commentary

This patch loads frames from a downsampled Quicktime video into a 3-D matrix, and then uses jit.gl.volume to visualize this matrix. You may find that this will slow down upon switching to fullscreen mode. This is because jit.gl.volume must do per-pixel rendering, which can be a pretty heavy operation.

Ingredients

jit.unpack,jit.pack

jit.matrix(3D)

jit.qt.movie

jit.gl.volume

jit.op

Technique

Once we load a video into jit.qt.movie, we pass it through the “alpha-channel” subpatch. This unpacks the 4 char matrix and replaces the alpha channel with a mask generated by performing a luminance gate (sort-of like a noise-gate for video). This is accomplished by using the jit.op @op >.

Using “srcdim” messages to the jit.matrix object, we are able to continuously replace frame-slices in a random order. A similar scheme can be used to create your own video buffer like jit.matrixset. In our case, we are merely looking for a source of data to fill a cube of pixels.

This downsampled video buffer is then sent to jit.gl.volume to be rendered as a nice soft collection of colored blocks.

Recipe 03: Crumbling Fortresses

General Principles

Loading a 3D matrix with multiple frames of video.

Visualizing 3D matrix values as OpenGL geometry

Creating dynamic alpha masks for transparent textures.

Mapping OpenGL geometry with a texture.

Commentary

This is a good example of applying one basic recipe in a variety of situations. Here we use the same basic technique as the Video Pillows example, using the jit.gl.isosurf object for a very different effect.

Ingredients:

jit.unpack,jit.pack

jit.matrix(3D)

jit.qt.movie

jit.gl.volume

jit.op

Technique

Because jit.gl.isosurf doesn’t use the color of the 3-D matrix for its surface color, we must generate a texture to be mapped onto the resulting geometry. We have the same alpha-channel patch that we saw before, only this time, we are outputting the matrix after the jit.rgb2luma to generate the geometry. This gives us a one-plane matrix that we can use to look up isolevels.

We then use the luminance-gated video as a texture that is applied to our geometry. Once the texture is applied, the transparency mask makes it appear as if parts of the geometry have been removed, leaving jagged edges. This is a good technique for creating seemingly complex surfaces without requiring too many vertices.

Recipe 04: Asteroid Growths

General Principles

Controlling OpenGL with audio signals.

Using Amplitude Modulation to create more interesting response.

Using 3-D matrix for volumetric data.

Using matrix feedback

Commentary

This example continues to be one of my favorites, as it combines audio modulation techniques with basic matrix processing in a way that generates some very organic motion. At first glance this patch will probably appear to be vastly complex, but the mechanics of it are really pretty simple.

Ingredients

jit.poke~

jit.gl.isosurf

jit.matrix (3D, named)

Various MSP objects(rand~,*~,etc.)

Technique

Because we are using a single audio signal to create our geometry, and we need to generate multiple dimensions of movement, we will use our audio signal to modulate other signals. I use the rand~ object to generate randomly ramping signals at various rates. These signals are then multiplied to fall within the dimensions of our matrix (50x50x50). Because we only need one plane of data, we use our input signal to modulate another rand~ signal.

As if it weren’t enough to have this matrix full of modulated audio data, we’ve also added a “growth” matrix. By slightly downsampling the matrix and then mixing with the original, the pixels appear to grow outward. This also creates a low-pass filter which smooths the movement over time.

The resulting matrix is finally fed into the jit.gl.isosurf object to be converted into OpenGL geometry.

Recipe 05: Shifty

General Principles

Visualization of Audio

Image Distortion and Feedback

Commentary

This example shows a technique for using the graphic representation of sound provided by jit.graph to render more complex and interesting visuals. A similar technique can be used to create your own Visualizer.

Ingredients

jit.catch~

jit.graph

jit.matrix

jit.rota

Technique

We are using jit.catch~ to convert audio vectors into matrix data, which is then passed off to jit.graph to plot the floating point values of each cell in a waveform shape.

The output of jit.graph by itself looks like a simple oscilloscope representation of our audio. As this on its own is pretty simple, we then pass it on for further processing.

The output of jit.graph is then passed on to the jit.rota object to be slightly rotated and then looped back to mix with the original using a lumakey object. The lumakey object renders all black pixels transparent so that the fed-back matrix will still be visible.

For added visual pleasure, try hooking up more video processing on the end of this chain, such as jit.robcross or jit.ameba.

Recipe 06: Dancing Sprites

General Principles

Using Audio to control OpenGL

Rendering multiple copies of the same object

Particle Systems

Using Movie Textures

Creating Alpha Masks for texture blending

Commentary

This patch generates numerous copies of the same videoplane that dance around through space in response to the audio file. This example is one of the more complex patches, using techniques from a couple of other patches we have already seen. If we were to break down the individual components of this patch, you would most likely recognize most of the processing stages.

Ingredients

jit.catch~

jit.gl.videoplane

jit.iter

jit.matrix(named)

jit.scissors

jit.unpack,jit.pack

jit.op

Technique

We are loading a matrix using the same technique as the TimeScrubber patch. This matrix is then downsampled and chopped up into 3 columns using jit.scissors. Each of these columns is then packed in as different planes for a 3-plane matrix. This allows us to generate (x,y,z) values from a single audio input.

On the right side of the patch is all of our rendering process. You’ll recognize the “make-alpha” subpatch from the VideoPillows example.

Inside of the “sprite-generation” subpatch you will find the same setup we used with the gLFO patch to render multiple instances in various positions. We use a scale message to resize the “sprites”.

Take note that the viewalign attribute of our jit.gl.render is turned on. This enforces “billboarding” of our sprites so that they always face the camera.

Recipe 07: FftCollector

General Principles

Using a Matrix to store Audio data.

Using Jitter to manipulate FFT frames

Commentary

This patch was originally posted as a response to a fellow who was looking for a way to do time-lapse audio recording. While this may not be a complete solution to that problem, it does illustrate the ability of Jitter objects to be of great assistance in working with frequency-domain audio.

Ingredients

fft~,ifft~

jit.catch~

jit.matrix

jit.peek~

Technique

The whole process here should be pretty clear if you have looked at the previous jit.catch~ examples. The only trick is that we are using 512 sample FFT frames instead of time-domain audio signals. By fixing the framesize of jit.catch~, we make sure that we don’t get frame offsets in our matrix. Note that we are capturing both amplitude and phase output from the FFT.

The “output” subpatch allows us to read back our captured FFT-buffer at varying speeds. One interesting experiment would be to upsample or downsample our matrix with interpolation and see what sorts of effects it has on playback.

Recipe 08: VideoBlobs

General Principles

Generating organically moving geometry

Using Quicktime Movies as Texture

Visualizing 3D Matrix as Geometry

Commentary

This patch displays a movie as the backdrop of the rendering context using the “draw_pixels” message, and then overlays a blob-like 3D object mapped with the movie as a texture. The combined effect creates an interesting distortion of the moving image that continuously changes shape.

Ingredients

jit.noise

jit.gl.isosurf

jit.gl.texture

jit.gl.render

jit.matrix

jit.slide

route,prepend

jit.qt.movie

jit.xfade

Technique

To generate our geometry, we simply use jit.noise to generate a 5x5x5 matrix of random values. This is then upsampled with interpolation to create smooth shapes. This matrix is then fed through jit.slide for time-interpolation, and then sent through a feedback loop for additional manipulation (see AsteroidGrowths). This is then sent to jit.gl.isosurf for rendering.

To place our movie in the background of our rendering context, we use route to remove the “jit_matrix” prefix, and then prepend the “draw_pixels” message. This message is then sent to jit.gl.render. Note that this doesn’t provide any means of interpolation, so the movie will look pretty jagged at fullscreen. If you need a smoother look, a jit.gl.videoplane would be a better option.

We then hook up a jit.gl.texture object to our jit.gl.isosurf object and connect our jit.qt.movie.

Experiment with different attributes for a variety of effects.

Recipe 09: ParticleRave-a-delic

General Principles

Basic synthesis of random video sprite textures

Particle Systems

Rendering multiple copies of geometry

Simple matrix operations

Commentary

The most important thing to take away from this example is that you can easily generate your own particle physics using simple matrix operations. After posting this to the MaxMSP mailing list, one enthusiastic individual claimed to have used this patch in his VJ set at a club. As you could imagine, any of the parameters can be controlled by other things, such as audio volume, gestural controllers, etc.

Ingredients

jit.op

jit.noise

jit.charmap

jit.gl.gridshape

jit.gl.texture

jit.slide

jit.matrix

Technique

For the sake of brevity, I’ll omit topics already covered in other examples.

To generate our particle image, we simply create a single-plane 6×6 matrix of noise and and then upsample to a larger 4-plane matrix for smooth gradients. This grayscale image is then sent through a jit.charmap object to be colorized in random and exciting ways. We then use the “mask1″ matrix to soften the edges of our alpha channel, which gives us nice fuzzy-edged sprites.

To perform our particle physics, we start with a named matrix with all values set to 0. We then use jit.noise to generate a set of (x,y,z) velocities. To constraint these values, we use a couple of jit.op objects to bring them into the desired range (-0.05,0.05). To prevent jerky changes in direction, we use jit.slide to make a rounded transition.

This velocity matrix is then added (jit.op @op +) to our named matrix to iterate one frame of motion. The result of this calculation then gets sent through a multiplier (gravity), and then back out to our named matrix. This creates a feedback loop.

We then send our matrix out to jit.iter to be broken down into position messages (see gLFO).

This is a relatively simple particle model, which doesn’t include environmental forces. These aspects could be added by using other jit.op or jit.expr operations.

Recipe 10: BufferOps

General Principles

Non-realtime Audio operations

Using Matrix Processing to manipulate Audio

Commentary

This basic recipe demonstrates the use of jit.buffer~ to manipulate the buffer as a jitter matrix. This allows you to do several operations that would be much more difficult using standard MSP processing, such as recursive processes and instant sample-mangling.

Ingredients

jit.buffer~

jit.op

Technique

The upper section of this patch allows us to load two buffers with different audio clips. The output message is used to send the the buffer data out as a matrix. At this point the left matrix is operated on by the second matrix depending on which operation we choose.

The second stage of the patch shows how to apply scalar operations in place. The output of the jit.op object is simply sent back to the jit.buffer~ to replace its contents.

Recipe 11: ScanMusic

General Principles

Converting Video to Audio

Commentary

As this patch shows, doing direct conversion between audio and video can be a pretty straightforward process. Whether or not this will provide you with useful or pleasing output is another question altogether. This is intended as a basic module that can be processed further or included in a more complex system.

Ingredients

jit.qt.movie

jit.matrix

jit.release~

Technique

Once we have a video loaded, we convert it to grayscale using jit.rgb2luma.

Jit.release~ expects a 1-D horizontal matrix, so we use “srcdim” messages to jit.matrix to select a single scan-line from our video.

This is then sent on to jit.release~ to be converted into audio.

Recipe 12: TextureDistorter

General Principles

Image/Texture distortion using geometry manipulation

Image/Texture distortion using texture coordinates

Using mathematical expressions with Jitter Matrix

Using matrix to create OpenGL geometry

Commentary

This patch presents two methods of manipulating and distorting your video image with OpenGL processing. You will notice that I make use of the jit.expr object a great deal in this example. With a little practice, this object will quickly become your Swiss Army Knife of matrix operation. The jit.gl.mesh object provides a very handy interface to openGL drawing primitives.

Ingredients

jit.gl.mesh

jit.gl.gridshape

jit.expr

jit.matrix

jit.qt.movie

jit.noise

jit.op

jit.gl.slab

Technique

The first step is generating the plane that we will map the texture onto. We do this by outputting the matrixoutput of a jit.gl.gridshape object (you can try other shapes as well). This matrix of vertex coordinates is then modulated by another matrix (see ParticleRave).

The modulation matrix is generated by creating 3-plane noise and then scaling the values to fit within our range. The jit.expr object does this quite efficiently, as we can give it an algebraic expression that would require three jit.op objects to accomplish.

In addition to spatial coordinates, we must generate texture coordinates as well, since we will be mapping an image onto this shape. To do this I merely send a 2-plane matrix (x,y) through a jit.expr object, which generates normalized values (0.-1.) across each dimension.

Notice that we’re also using a second jit.expr object with a very different expression(1/2 duty-cycle sine). By fading between these two matrices we control how much distortion is generated by our second expression.

We then use the jit.gl.slab object to convert our video into a texture that is bound to our jit.gl.mesh object.

Hopefully this patch will get you started imagining all sorts of transformations that you could perform on your textures.