Links

Wednesday, May 23, 2007

audio vs video

The Redmonkers have been starting to web publish video interviews along with their usual audio interviews.
Coté seems to be doing most (all?) of the work, and you can catch these as he releases them on
his blog.

I like to see people experimenting with new technology, and 'upgrading' from audio to video sounds
like a fun experiment (pardon the pun). But it doesn't work for me.

My issues:

There really isn't that much 'extra' in a video interview, over just the audio.
You get to see faces. You get to see some body language. Maybe a picture or two.

The idea of watching an interview means I have to have two senses trained
on the media: eyes and ears. You're killing my continuous partial attention!

I can't listen to it on my video-challenged iPod.

The reason I don't have a video-capable iPod is that the situations in
which I listen to my iPod don't lend themselves to allowing me to watch
something on the device as well: driving, mowing the lawn, washing the dishes, etc.

I fully admit to being an old fuddy-duddy; even Rush Limbaugh does video
of his show. Good luck guys, and, if you can, also make the audio portion
of your videos available as an MP3.
I'm not alone in this wish.

But let me change the direction here. Let's look at an environment that is
high on video interaction, and absolutely bereft of audio interaction. Your
programming environment. Your IDE, be it Eclipse, NetBeans, IntelliJ, Visual Studio,
XCode, Emacs, or a text editor and a command-line. How many of these programs use
audio to help you develop code? None. Well, I might be lying, I'm not
familiar with all of these environments, but I don't recall any of these
environments making use of audio like they do visuals.

When we're programming on all cylinders, we're in 'the zone'. Continuous
full attention. Eyes reading and scanning, fingers typing and clicking and
moving mice. Where's the audio? It ain't there.

I would be remiss in not pointing out here that audio feedback like this is only
useful to those of us lucky enough to have decent audio hardware and software in our
heads. Those of us without such luck
wouldn't be able to take advantage of audio feedback. On
the other hand, folks who lack decent video hardware and software in their heads
would most likely appreciate more
emphasis on a sense they are more dependant on.

The most obvious use case for audio in a development environment is with debugging.
There are a lot of cases while
debugging when you just want to know if you hit a certain point in your program. The
typical way you'd do this is to set a breakpoint, and when the location is hit, the
debugger stops, you see where you are, and hit continue. Breaking your attention and
demanding your input. What if, instead, you could set an audio breakpoint, that would
play a sound when the location was hit? So your attention wasn't broken. And you didn't
have to press the Continue button to proceed.

With regard to audio debugging, I know this has been experimented with many
times in the past. I've done it as well, a decade ago, when I was using a
programming environment that I was able to easily reprogram: Smalltalk.

But audio usage in development environments is not yet mainstream.
There's lots of research to be done here:

What are the best sound palettes to use: audio clips, midi tones,
short midi sequences, percussion vs tones?

How should we take advantage of other audio aspects like
volume and pitch and three dimensional positioning,
especially regarding continuously changing quantities
like memory or i/o usage by an application?

How do we deal with 'focus' if I'm also listening to
Radio Paradise while I'm working?

Does text-to-speech make sense in some cases?

How do we arrange multiple audio feedback to be presented
to us in a way that's not unpleasing to listen to? Not just a
cacophony of random sounds.