inferno programmer's notebook

Thursday, December 27, 2012

WAV file format contains audio sample data and optionally meta-data that describe the offsets of sample loops and cue points. The loop offsets are used by sampler software to generate a continuous sound, and the cue points mark the point in the sample data where the sound fades away after the note has been released.

A WAV file "smpl" chunk will identify the byte sample offset of the start and end of the loop in the sound data. Using wavplay.b as a starting point I tried to loop a sampled sound.

My first test was simply to treat the sample as-is and loop the sound using the given offsets. This did not give good results with a notable noise as the data from the end of the sample joined with the beginning. I realized nearing the end of writing this post that the mistake I made was treating the offsets as counts of bytes instead of samples. The documentation I was using said they were bytes. But the early mistake caused me to develop the following, which turned out useful. You can try out my first experiment yourself with this lab's code and the included sample.

Audacity has a feature where you can move the selection boundary to the nearest zero in the waveform. In the picture above that would be the beginning, middle, or end of the waveform. Using this feature creates a more seamless transition between the end and start of the loop. However, the sound samples can be any point along the curve and there are very few actual zeroes in the sample data. Instead I search for the crossing point of the y axis from negative to positive. The start and end samples must cross the zero-line going in the same direction. In the picture, the end of the graph is crossing the boundary in the upward direction and would join perfectly with the start or the curve forming a perfect loop. Play the sample without the -n flag to hear the difference.

% wavloop -s 045-A.wav

This works in most cases, but because the sample is in stereo the left and right channel might not be heading in the same direction at the same time.

For the release cuepoint I follow a similar technique. I search for the zero crossing point where the data is rising. That will be where I jump to. But where I jump from could be anywhere in the sample data. So when the release event happens I also need to look for the nearest crossing point from where I am. This gives us the state transitions in the code for generating the sound data: START, LOOPING, RELEASING, RELEASE, and DONE.

For the final experiment I tried to use the begin and end markers in the wav file as sample counts instead of bytes. In a 16bit, 2 channel WAV file there are 4 bytes for each stereo sample. This ended up making the best sounding loops.

% bind -a '#A' /dev
% wavloop -n 045-A.wav

This is a lot of explanation for a little bit of code. I was hoping writing it out would help me figure out how to fix the looping to sound seamless in all cases, which it did, since I eventually tried the experiment of treating offsets as sample counts. But finding the nearest zero is still useful for jumping to the release cue point. The other way of making that join is by using a cross-fade, which I didn't try.

The next step will be to load in a complete sample set, at least for one windchest, and implement midiplay's Instrument interface to plugin to midiplay.

The plan is to have a copy of this data to give to each of my children. My Dad recently scanned and sent me all his photographs of me and my siblings growing up; he also included pictures of himself and my Mother when they met in Africa. With technology today each generation can build a digital library of family history to hand on to the next generation. In the past a family album may have been passed on to only one person. The accumulation of digital data still presents problems. It requires discipline to store files that are open and not locked into devices or proprietary formats.

With digital preservation in mind I've tried to use file formats recommended for long term archiving. WAV files for audio, DV for video, JPG and PNG for pictures, PDF for documents, and plain text.

Today the storage media is a 1TB external hard-disk. In my current computing environment I'm plugging that disk via USB into computers running Windows 7, Ubuntu, MacOSX, RiscOS, and Raspian. Ideally I'd be able to launch an application from the hard-disk that'd be able to playback the archived media on any of these host systems. This is where Inferno enters the picture.

Based on the criteria for selecting the file types, (non-proprietary, well documented, wide support) all the host systems should support the files natively, or a download can be easily found that will. However, the challenge is to get Inferno to do it nearly as well and work with a single set of tools everywhere. The tools are then preserved with the media on the disk.

So here's the plan. I've created a clone of inferno-os (caerwynj-inferno-ae) and I will try to support the media files that are stored in a digital archive. The target functionality is the following:

Display pictures (JPG, PNG, GIF)

Playback audio (WAV)

Playback video (AVI-DV, MPEG-2)

View ebooks (ePub3.0)

View documents (Plain text, HTML4.0, PDF, DjVu?)

Playback MIDI (built-in synethsizer)

Mount disk images, archives, and file systems

Compress or decompress files (gzip, bzip2)

Another important aspect of this is to have a collection of EMU's that run on all the target platforms. I started this project in the inferno-bin repository a while ago. But it needs updating and the discipline to keep updating it. Ultimately the system interface of EMU needs to be locked down to give the freedom to run dis well into the future without the need to recompile.

Here's where we are with support inside Inferno:

File type

Inferno support

JPG, PNG

Supported by wm/view

WAV

Supported by wavplay

MIDI

Limited support by midiplay

ePub

Support for older OEB versions in ebook/ebook. I've started updating to support ePub standard.

Plain text

Supported

HTML4.0

Limited support in Charon

PDF

Unsupported, but MJL's PDF library are the beginnings of limited support

AVI-DV

No support

MPEG-2

Some support for an MPEG device. Could try to use Raspberry-Pi device support for MPEG-2

There has been a three and a half year gap in my posts to this blog. In that time I hadn't done any Limbo programming. I've used Acme as my editor everyday, but I was drifting towards using Notepad++ more often. In the past couple of months I've had the time to contemplate doing some hacking projects. I wanted to explore what I could do with Inferno for multimedia file types. This lab was the first thing I tackled in using Inferno again. I had to open up the Limbo paper to remember even some basic syntax.

It bothered me that wm/view only displayed images using the Inferno 256 color map. Charon didn't have this limitation and I thought it had something to do with their respective image libraries. They don't use the same code. I extracted Charon's img.b code out into another view tool only to realize once I'd finished that the difference was not in the handling of JPEGs or PNGs but in the remap of the raw image to an Inferno image after the image was loaded.

I changed wm/view to remap the graphic to 24bit color. Here are the results.

There are a few other files in this lab.

view.b

The modified wm/view to display JPEG and PNG in true color.

img.b

The img library extracted from charon

pics.b

A view tool similar to the Plan 9 view command. It uses the charon img.b library. It also used the rioclient, riolib, and menuhit files included in this lab to give the appearance of a Rio window. You can see it in the upper left of the screenshot. Using this you can zoom in, pan, and fit to window. The zoom now works much better with true color than with the colormap.

riolib.b

An alternative to wmlib.b for non-TK window applications

rioclient.b

Use with riolib.b for non-TK window applications

menuhit.b

Menus for non-TK windows

clock.b

A non-TK clock demonstration from Plan 9

picsar.b

Similar to pics.b but doesn't use the charon img library. Also doesn't do the remap to 24bit color.

matador.jpg

Test JPEG

test.png

Test PNG

FILES

wavplay plays a WAV file. It is merely a combination of the wav2iaf and the auplay commands already in Inferno. I have no audio in IAF format, but I am putting together 100s of GBs of wav files as I'm ripping my CD collection.

% bind '#A' /dev
% wavplay track.wav

FILES

Thursday, December 20, 2012

NAME

lab 107 - midiplay

NOTES

Midiplay plays back MIDI files. It uses the synthesizer I described in lab 62 and the MIDI module from lab 73. The command takes only one argument, the path to the midi file. I've included one in the lab to demonstrate. Bind the audio device before using it.

% bind -a '#A' /dev
% midiplay invent15.mid

The synthesizer has 16 note polyphony. It uses three oscillators, one at the pitch, one at half pitch, one at double pitch. There is also a filter, two delays and a vibrato.

The sample rate is 8000hz and there is one mono channel (MIDI channel events are ignored). It performs well enough to work on my laptop without JIT turned on. All the synthesizer parameters can only be tweaked from inside the code at the moment.

FILES

Monday, December 06, 2010

NAME

lab 106 - UNIX RUDP Support

NOTES

A simple port of the Inferno native RUDP support to a standalone UNIX environment (with a client and server example). This particular example is single threaded and specifically intended for use with synchronous RPC mechanisms (in this particular instance a synchronous 9P client and server).

It could probably be greatly improved in several dimensions, but this example should provide a minimal implementation which more complicated incarnations can be built off of. Enjoy.

FILES

Thursday, June 03, 2010

This blog is now located at http://ipn.caerwyn.com/.
You will be automatically redirected in 30 seconds, or you may click here.
For feed subscribers, please update your feed subscriptions to
http://ipn.caerwyn.com/feeds/posts/default.

Wednesday, November 04, 2009

NAME

lab 105 - automount

NOTES

A small modification to mntgen yields an automounter which automatically mounts a server based on path name. This is a fairly crude proof of concept with hardwired port numbers and top-level mount path -- but it would be fairly easy to make it a bit more robust.
Essentially, I just added a mount command to the code which dynamically adds the directory node to the mntgen tree on reference. Then I made the file system handling call its own thread so that the mount could reference the synthetic without locking up the original single threaded synthetic file server. One problem is that unknown servers take some time to respond with file note found, which is less than desirable. Fixing this and other annoyances is left as an exercise for the reader.

FILES

Thursday, August 06, 2009

NAME

lab 104 - ducts: bi-directional mount channels

NOTES

Hi there, I want to talk to you about ducts. Are your ducts old fashion, out of date?

In this lab, I'll walk through a little facility I built to allow me to have bi-directional mounts over a single file descriptor channel. The nature of the way I access my Blue Gene environment requires me to do an awful lot over a single communications channel. I wanted the ability to both access the target node's file system while simultaneously exporting mine, while only using a single file descriptor.

To facilitate this, I wrote a little 9P-knowledgeable multiplexor which directs T-msgs to an export thread and R-msgs to a mount thread while multiplexing the responses from those threads back onto the same file descriptor.

The initial implementation of the base mechanism was relatively simple until I had to start considering how to clean things up when I started unmounting directories on one side or both. The liberal use of sys->pipe throughout the code meant it was somewhat difficult to detect when things were finished. I ended up adding a signaling packet to the protocol to notify the remote Duct when I had closed my mount, and then implemented a channel based monitor infrastructure so I could track what was still running and appropriately shut down the various pieces as things unmounted (it is possible to unmount one side without unmounting the other).

The interface is fairly simple, if using the module interface you spawn an uplink, specifying an input file descriptor and an output file descriptor for the connection (which may be the same FD), the path you wish to export and the mount point for where you want to place the imported file hierarchy.

If you are running it from the command line, it uses stdin and stdout for the input and output FDs and you only need specify the export path and mount point.