Thinking about fog. Starting with my code that fades out the foreground based on the depth data, I modified the script so that there’s some randomness to the transparency. I want the edges of the object to shift in and out, as if in a fogbank. But the effect needs work, and I didn’t get any good images today.

I spend a lot of time remembering. I scribble lists of what to try next, I keep track of my hours, I write blog posts about what I’m working on. My code is strewn with comments and reminders. I save drafts and sketches, I backup my work. The computer is a machine for remembering.

It’s easy to forget how useful forgetting can be. If I’m stuck on something, especially when writing code, most of the time I just need to get away from it for a while. In the movie True Stories, David Byrne’s character says something that’s stuck with me for years:

I really enjoy forgetting. When I first come to a place, I notice all the little details. I notice the way the sky looks. The color of white paper. The way people walk. Doorknobs. Everything. Then I get used to the place and I don’t notice those things anymore. So only by forgetting can I see the place again as it really is.

Forgetting, not only as a creative strategy, but as a pleasure. A way to recapture the joy of experiencing something for the first time. This came up again in a book I just finished, Michael Pollan’s The Botany of Desire:

Memory is the enemy of wonder, which abides nowhere else but in the present. This is why, unless you are a child, wonder depends on forgetting – on a process, that is, of subtraction.

When I want to wonder, I read anything by Italo Calvino. His final, unfinished book, Six Memos for the Next Millennium, is a series of lectures about various qualities he valued in literature. He starts with “lightness”:

My working method has more often than not involved the subtraction of weight. […] Above all I have tried to remove weight from the structure of stories and from language. […] Maybe I was only then becoming aware of the weight, the inertia, the opacity of the world – qualities that stick to writing from the start, unless one finds some way of evading them.

To erase the opacity of the world, what an admirable idea! I want my little fogbank script to attempt this. By erasing information from the video stream, by subtracting opacity, I can introduce a kind of lightness. An opportunity to see something as if for the first time.

Feet are good tools for forgetting. When I need a new perspective, I walk up Signal Hill, often along the North Head trail. I was up there once last summer, soon after I first moved here, and there was an incredibly dense fogbank hovering just off the edge of the cliff. I sat down and watched it for a bit. A kind of brilliant backlit haze, and peering into it felt almost as if my eyes were closed, but white instead of dark. After a while, I realized I could see something shifting around in the haze. It took me a minute to figure out what I was looking at, and I’m still not entirely sure if it was my contact lenses, or the inside of my eyes. But my vision was full of subtle little squiggles and inconsistencies, and if I moved my head they would swish from side to side. Entoptic phenomena.

The idea of seeing things with new eyes is present right in the title of Lawrence Weschler’s Seeing Is Forgetting The Name of The Thing One Sees. Near the end of the book Robert Irwin rants a bit about getting away from screens, something else I should try not to forget:

The point is to get people to peel those visors off their faces, to remove the goggles, to abandon the screens. Those screens whose very purpose is to screen the actual world out. Who cares about virtuality when there’s all this reality – this incredible, inexhaustible, insatiable, astonishing reality – present all around!

I took the code that visualizes the Kinect depth data, and the code that knocks out the background, and combined them. So now, as things get further from the camera, they fade into transparency. I can insert any background here, and it doesn’t have to be the same colour, but for now I’ve put the blue recycling bin to work again.

There was a lazy susan in the studio, luckily enough, and it came in handy for spinning the chair (you can see the edge of it showing up in the clip above). Here’s the setup with the Kinect sensor in the background.

There’s something eerie about seeing the chair slip out of view, as if the scene is lit by candlelight. Fading the object into the background introduces a kind of fog, and the chair starts to seem larger, a building looming out of mist. Its four legs begin to read as the corners of a house. I want to use this effect to make a small landscape model feel like an island.

I’m also wondering if I could connect this to a higher-quality camera. The Kinect depth data wouldn’t sync perfectly, but it might work well enough. I’d love to try some photography or video outdoors using this effect. Funny to think about introducing virtual fog to the streets of St. John’s.

Here’s a part of a wonderful poem about embracing fog and flaws.

I tell you it has taken me all my life
to arrive at the vision of gas lamps as angels,
to soften and blur and finally banish
the edges you regret I don’t see,
to learn that the line I called the horizon
does not exist and sky and water,
so long apart, are the same state of being.
—Lisel Mueller, from Monet Refuses the Operation

I’m working on a little world, or at least a window into one. One thing I want to try is digitally merging a live video stream with other elements, such as computer-generated imagery or prerecorded video. This way I can create something that looks like a video, but that reacts in real-time to movement or sound. I like the idea of making something that appears at first to be a video loop, but is actually a computer simulation with endless variation. This is sort of how procedural generation works in game design.

For A House By The Water I want to build a small model of an island or coastline, but I’m still figuring out what materials to use, and how big it needs to be. For now, a rock will do. Today I walked to the studio along a slightly different route, past a field and a construction site, where I found a large white stone.

I spent most of the afternoon putting together some code to knock out the background from a webcam stream. A blue recycling bin made a pretty good “green screen”, and I was able to superimpose live video of the rock on top of another video (a couple of seagulls). I made a little clip of this process while I moved a light around, so you can see what’s happening.

I like having the rock here. My studio desk is full of books and equipment, and the rock brings a different sort of presence and purpose. I often work with things that only exist on a screen, and it’s a reminder to stay grounded in the real world. I’ll keep it around so it can keep an eye on me.

I arrived at The Rooms to find a team of employees evicting my roommates, the tall sculptures left behind by the previous artist in residence. So I have the place to myself now. I feel like capturing the space in some way. I’ve brought my camera, but I set up the Kinect sensor instead. It captures video and also depth information, which I can visualize using Processing. I start with Daniel Shiffman’s Point Cloud demo and spend some time modifying the code. I take it for a spin.

Thirty times a second, the Kinect takes two pictures, compares them, and spits back a bunch of numbers representing the distance of each pixel from the sensor. The Point Cloud script reinterprets these numbers and projects each pixel in a simulated three-dimensional space. Watching the demo in real-time, it feels strange to see yourself whirl around while the camera stays still. The script rotates the scene around an imaginary point in the virtual space.

I want to try something else. I place the sensor on top of a ladder and aim it at the ceiling, which is full of lights, pipes, ventilation and a dangling extension cord. I increase the density of the depth image, and adjust the colours used for the projection. When I end up with something I like, I export screenshots of each frame and stitch them together into a gif animation.

I like this curious architectural snapshot. The virtual rotation creates a strange illusion where it feels as if the camera arcs through the ceiling and looks down through it, into the room. The depth projection is interrupted by the shadows of objects closer to the camera (the dark holes in the orange surface). This is due a limitation of the stereo vision system, but I enjoy the theatrical quality it adds to the scene, like a stage set only meant to be seen from one angle. Or as if the world has unfolded from flatness, like a pop-up book.

One of the books I brought to the studio is In Praise of Shadows by Jun’ichirō Tanizaki. Written in 1933, it’s a wonderfully ruminative essay on Japanese aesthetics, the beauty of impermanence, and shadows in architecture:

An empty space is marked off with plain wood and plain walls, so that the light drawn into it forms dim shadows within emptiness. There is nothing more. And yet, when we gaze into the darkness that gathers behind the crossbeam, around the flower vase, beneath the shelves, though we know perfectly well it is mere shadow, we are overcome with the feeling that in this small corner of the atmosphere there reigns complete and utter silence; that here in the darkness immutable tranquility holds sway.