2009-02-12 12:13 pm

When NWC imports a MIDI file it does not faithfully represent triplets; correcting them manually is a tedious business so I have written a User Tool to do this.

It depends on correctly recognising the approximations that NWC generates during the import process. My current understanding of these is based on examples that I have exported and re-imported. I invite those who know NWC from the inside to help me to do better.

// NWC's efforts in terms of representing a triplet come from examples. There may be more to come.// When adding to this array, always insert sub-array elements in descending size$validLengthGroup = array (array(6,5,5),array(3,3,2),array(8,5,3), // a very poor and problematic representation of a triplet, but it does occur e.g. // with triplet (crotchet, crotchet rest, crotchet)=(8,3,5). However, in the quaver version of this // example the rest has disappeared altogether (3,NULL,1). so we have no chance!array(11,5),array(5,3));

// then sort the array descending to match the convention in definition of validLengthGroup rsort($lengthSet);

// now we are ready to check for a valid group of lengths foreach ($validLengthGroup as $thisgroup) { $result = false; // in case the last group has different length to the array under test if (sizeof($lengthSet) != sizeof($thisgroup)) continue; $result = true; // assume a match until a mis-match is found foreach($lengthSet as $i=>$v) if ($lengthSet[$i] != $thisgroup[$i]) $result=false; if ($result) break; // found the winner, no need to go on searching } return $result;}

// there is at least one candidate in the queue, record length and check for triplet if($tied_note_pending) { // add its length to that stored for the previous note$last_element = sizeof($lengthSet)-1;$lengthSet[$last_element] += $length; } else {array_push($lengthSet, $length); }

if ($tied_note_pending = $is_tied) continue; // yes, really not "=="! this is ready for the next lap

// check if we have a triplet while (true) { // start a loop so we can retest having discarded one noteif (is_valid_group($lengthSet)) { //we have a triplet, output the notes in modified form $numConvertedTriplets++; $length = array_sum($lengthSet)/2; // this is the un-triplet-ised length of a single element// $dur = array_search($tickLength, $length); WHY DOESN'T THIS WORK? use alternative code foreach($tickLength as $key => $value) if ($value == $length) {$dur = $key; break; }// $durxtwo = array_search($tickLength, $length*2); WHY DOESN'T THIS WORK?

foreach($tickLength as $key => $value) if ($value == $length *2) {$durxtwo = $key; break; } // Output the triplet, being two or three notes/rests/chords$output_tied_note_pending = false; $length_index = 0;foreach($TripletQ as $this) { if ($output_tied_note_pending) {$output_tied_note_pending = isTiedNote($this);continue; // dumping this note and processing the next item in the queue } if (isset($this->Opts["Opts"]["Beam"])) unset($this->Opts["Opts"]["Beam"]); if (isset($this->Opts["Opts"]["Stem"])) unset($this->Opts["Opts"]["Stem"]); // triplet stems don't need to be aligned // can't get rid of Opts altogether - a rest might have Opts["VertOffset"] set - not likely though

$length_index++;}$TripletQ = array(); $lengthSet = array(); $tied_note_pending=false;break; // out of the while loop} elseif (sizeof($lengthSet)<3) { // not a triplet yet but there is still time so store data continue 2; // breaking out of the while loop to get a new item from the clip} else { // not a triplet so output the first note, drop it from the stored queue and retest$output_tied_note_pending = true;while ($output_tied_note_pending) { $this = array_shift($TripletQ); $output_tied_note_pending = isTiedNote($this); echo $this->ReconstructClipText()."\n";}$this = array_shift($lengthSet); // dump its length too// Now we must retest because the remaining notes, if any, could be a (2 note) tripletif ($TripletQ) continue; // repeat the while loop} // end if triplet, maybe triplet or not triplet } // end while(true)loop} else { // not a note, sequence is spoiled so output everything in the queue, plus this non-note item and start afresh if ($TripletQ) foreach($TripletQ as $this) {echo $this->ReconstructClipText()."\n"; } echo $o->ReconstructClipText()."\n"; $TripletQ = array(); $lengthSet = array(); $tied_note_pending=false;} // end if is_note else} // end for each clip

G'day Brian,just did a quick test with a file I originally created to test Bryan Creer's Swing tool. Took out the Swing staff, exported to MIDI and re-imported. Unfortunately your tool didn't recognise any triplets.

The answer lies in the MIDI import parameters you have used: "16th note resolution and quarter rest resolution". At this resolution your example triplets are very poorly represented, the note length ratios being (1,2,1). I could add this to my array of valid 'triplet signatures' but I fear that would lead to too many false positives (though when I tried this, there are none coming from your test file).

I generally use 32nd note resolution, which I assume is the default, and then your triplets come out as (3,3,2). With this change, the triplets in bars 5 and 7 are correctly recognised but in bars 13 and 15 a triplet is recognised one note too early; this is due to my accepting the triplet ratios in any order - so maybe I shouldn't do that.

This can easily be changed by commenting out the sort at line 57, though it is then necessary to modify the comment at line 28 and the subsequent triplet signature array so that note length ratios are always presented in the order they have been observed to occur, rather than descending size:

// When adding to this array, always insert sub-array elements in the order they have been observed$validLengthGroup = array (array(6,5,5),array(3,3,2),array(8,3,5), // a very poor and problematic representation of a triplet, but it does occur e.g. // with triplet (crotchet, crotchet rest, crotchet)=(8,3,5). However, in the quaver version of this // example the rest has disappeared altogether (3,NULL,1). so we have no chance!array(1,2,1), // another poor one, included to cope with Lawrie's test filearray(11,5),array(3,5));

Here I have also added a triplet signature (1,2,1) which is needed to cope with your re-import of your test file but I do think this is a bad idea for the reasons stated!

As I said in the original post, the tool is programmed to recognise the sequence of notes that NWC generates as an approximation to a triplet, based on the ratio of their lengths, but of course I have only put in the 'triplet signatures' that I have seen as a result of exporting and re-importing various test triplets. How much better it would be if I had the inside information that would enable me to predict what NWC will do in any import situation!

In getting to the bottom of why your test file was not ideally handled, I now understand my own comment in the code snippet above, so I need to change that too. With a rest resolution of a quarter, NWC doesn't have any chance to represent a quaver triplet rest!

Finally, NWC itself is in the best position to recognise triplets when importing MIDI files because at that stage the full resolution is available. Why doesn't it attempt this?

-- Brian

PS I had intended to attach a NWC file to this response as you did in your post; how does one do that?

Thanks for your input. This has raised issues about the range of MIDI import resolutions that should be accommodated, also the need to deal with concatenated rests. I can see a way to deal with these and will post a new version when I have had the time to test it out thoroughly.

I have now revised my User Tool to deal with approximations to triplets imported by NWC from a MIDI file; the new version appears below (brm_triplify).

The main change is to recognise that not everybody will import their MIDI file using the default parameters, so the tool certainly has to cope with finer resolutions. Coarser resolutions are problematic because as the approximation to a perfect triplet gets worse, the risk of falsely identifying a sequence of notes as a triplet becomes greater and there comes a point where it is best to skip these.

The other change is to handle concatenated rests. The need for this really only becomes apparent at high resolution and it is tricky: adjacent rests behave like tied notes, except that there is no indication which ones are notionally tied together. It gets worse: NWC will happily generate, for example, a dotted 4th rest, where one component - the 4th rest - does not form part of a triplet but the dotted bit - an 8th rest - does. I hope it doesn't churn out double-dotted rests - I could cater for them but I haven't.

I attach my test file which contains, on the top stave, a variety of triplets and, on the other staves, the result of exporting and re-importing at different resolutions. No parameters are required when invoking the tool; it will work with what it is given and convert to triplets any sequence that can safely be identified.

Lawrie drew my attention to Andrew Purdam's "tripletise" user tool but this requires very specific steering; for example the first triplet in my test file, at 64th note resolution (on the bottom stave) requires steering parameters "4 8t d32 16t 64t 16t 64 8t d32". It took longer to work that out than to do the edit manually, especially since it dealt with only one other triplet in the clip. I wanted something that will automatically unscramble as much as possible of the mess that NWC makes of importing triplets, with as little intervention as possible by the user.

I have no doubt that there are other triplet 'signatures' that I should have included but haven't; that is what the Newsgroup is for!

Finally, since it hasn't evoked a reponse yet, I repeat the question: why doesn't NWC make a better job of importing triplets in the first place, when exact information is available?

<?php/*******************************************************************************brm_Triplify Version 1.01

Seeks the triplet approximation sequences that NWC has created when importing a MIDI fileand converts those sequences into normal triplets.

History:[2009-02-14] Version 1.01 - Recognition of triplets from a wider range of MIDI import resolutions and handling of concatenated rests[2009-02-12] Version 1.00 - Initial version*******************************************************************************/require_once("lib/nwc2clips.inc");

// NWC's efforts in terms of representing a triplet come from examples. There may be more to come.// When adding to this array, always insert sub-array elements in the order they have been observed. Also beware that// the further these ratios stray from (1,1,1),(2,1) or (1,2) the greater is the risk of false positive triplet identification$validLengthGroup = array (array(6,5,5),array(5,6,5),array(3,3,2),array(8,3,5), // a very poor and potentially problematic representation of a triplet, but it does occur e.g. // with triplet (crotchet, crotchet rest, crotchet)=(8,3,5) if the import resolution is a bit lowarray(11,10,11),array(19,24,21),array(21,11),array(11,21),array(11,5),array(5,11),array(3,5));

// now we are ready to check for a valid group of lengths foreach ($validLengthGroup as $thisgroup) { $result = false; // just in case the last group has different length to the array under testif (sizeof($lengthSet) != sizeof($thisgroup)) continue;$result = true; // assume a match until a mis-match is foundforeach($lengthSet as $i=>$v) if ($lengthSet[$i] != $thisgroup[$i]) $result=false;if ($result) break; // found the winner, no need to go on searching } return $result;}

if ($tied_note_pending = $is_tied) continue; // yes, really not "=="! this is ready for the next lap

// check if we have a tripletwhile (true) { // start a loop so we can retest having discarded one note if (isValidGroup($lengthSet)) { //we have a triplet, output the notes in modified form$numConvertedTriplets++;$length = array_sum($lengthSet)/2; // this is the un-triplet-ised length of a single element//$dur = array_search($tickLength, $length); WHY DOESN'T THIS WORK? use alternative codeforeach($tickLength as $key => $value) if ($value == $length) {$dur = $key; break; }//$durxtwo = array_search($tickLength, $length*2); WHY DOESN'T THIS WORK?foreach($tickLength as $key => $value) if ($value == $length *2) {$durxtwo = $key; break; }

// Output the triplet, being two or three notes/rests/chords$output_tied_note_pending = false; $length_index = 0; $last_output_note_was_rest = false;foreach($TripletQ as $this) { if ($output_tied_note_pending) { $output_tied_note_pending = isTiedNote($this);continue; // dumping this note and processing the next item in the queue } if ($last_output_note_was_rest && ($this->GetObjType() == "Rest")) {// $last_output_note_was_rest is set anywaycontinue; // dumping this partial rest and processing the next item }

if (isset($this->Opts["Opts"]["Beam"])) unset($this->Opts["Opts"]["Beam"]); if (isset($this->Opts["Opts"]["Stem"])) unset($this->Opts["Opts"]["Stem"]); // triplet stems don't need to be aligned // can't get rid of Opts altogether - a rest might have Opts["VertOffset"] set - not likely though

$length_index++; } $TripletQ = array(); $lengthSet = array(); $tied_note_pending=false; $last_output_note_was_rest=false; break; // out of the while loop} elseif ((sizeof($lengthSet)<3) || ($is_rest)) { // not a triplet yet but there is still time $last_note_was_rest = $is_rest; continue 2; // breaking out of the while loop to get a new item from the clip} else { // not a triplet so output the first note, drop it from the stored queue and retest $output_tied_note_pending = true; while ($output_tied_note_pending) { $this = array_shift($TripletQ);if (($this->GetObjType() == "Rest") && (note_length($this) < $lengthSet[0])) { // now it gets complicated because if the partial rest we have just removed from the queue was dotted // we should only strip off 2/3 of it and put the rest back in the queue! // NB ignore the double-dotted case - I have yet to see one of these! $this_opts = $this->GetOpts(); $this_is_dotted = isset($this->Opts["Dur"]["Dotted"]); if ($this_is_dotted) { // the rest to be output is of this duration but not dotted unset($this->Opts["Dur"]["Dotted"]);echo $this->ReconstructClipText()."\n";$lengthSet[0] -= note_length($this);$this_length = note_length($this);//$this_dur = array_search($tickLength, $length); WHY DOESN'T THIS WORK? use alternative codeforeach($tickLength as $key => $value) if ($value == $this_length) {$this_dur = $key; break; }unset($this->Opts["Dur"][$this_dur]);$this_length /= 2;//$this_dur = array_search($tickLength, $length); WHY DOESN'T THIS WORK? use alternative codeforeach($tickLength as $key => $value) if ($value == $this_length) {$this_dur = $key; break; }$this->Opts["Dur"][$this_dur]="";$new_size = array_unshift($TripletQ,$this);continue 2; // break out of this while loop and retest for a triplet } else { // not dotted so it's much simplerecho $this->ReconstructClipText()."\n";$lengthSet[0] -= note_length($this);continue 2; // break out of this while loop and retest for a triplet }} else { // a note, chord or a single rest $output_tied_note_pending = isTiedNote($this); echo $this->ReconstructClipText()."\n";} } $this = array_shift($lengthSet); // dump its length too // Now we must retest because the remaining notes, if any, could be a (2 note) triplet if ($TripletQ) continue; // repeat the while loop} // end if triplet, maybe triplet or not triplet } // end while(true)loop $last_note_was_rest = $is_rest;} else { // not a note, sequence is spoiled so output everything in the queue, plus this non-note item and start afresh if ($TripletQ) foreach($TripletQ as $this) {echo $this->ReconstructClipText()."\n"; } echo $o->ReconstructClipText()."\n"; $TripletQ = array(); $lengthSet = array(); $tied_note_pending=false; $last_note_was_rest = false;} // end if is_note else unset($o); //???} // end for each clip

Finally, since it hasn't evoked a reponse yet, I repeat the question: why doesn't NWC make a better job of importing triplets in the first place, when exact information is available?

that would be 'cos we don't really know the answer. There is some surmise that it may be to restrict exposure to copyright violation... for myself, I don't know and in the time I've been on this forum Eric hasn't revealed his reasons for not improving those areas of MIDI import.

The left hand notes are arpeggiated, but NWC imports them as Chords. I'm not saying NWC is wrong here,

The left hand notes start almost at the same time and are shown as a chord but thereafter, as notes are added to the arpeggio, instead of adding notes to the chord, NWC cuts off the previous note/chord before starting the next. Maybe it would be too complicated to do anything else; it seems a reasonable import strategy.

Quote

just that this is yet another thing that anyone hoping to decypher MIDI import needs to know.

It is worth saying that any automated tool (even NWC itself) would struggle to deal with a source like your example, having a degree of randomness; presumably this one is was played in real time. It has, at least, shown me a small bug in the code at line 47, which should be

Brian may be using an old version of PHP - I think NWC used to use version 3 or so. He may not see this problem with "$this" until he upgrades to the version 5 PHP that the latest NWC now uses.

Brian, if you upgrade to PHP 5 [and I think it's reasonable to expect all users of your script to upgrade], within function isValidGroup, I think you could just use in_array in place of the last foreach loop, since PHP supports arrays as the "needle" as of version 4.2: return in_array($lengthSet, $validLengthGroup);

You could perhaps similarly avoid a foreach loop in function note_length, using array_intersect_key: return array_shift(array_intersect_key($tickLength, $opts["Dur"]));

Other miscellaneous comments:- The "if ($TripletQ) array_push..." could probably be simply "$TripletQ[] = $o;"- Your 2 array_search's didn't work because you have the 2 parameters reversed: some functions are needle/haystack, and others are haystack/needle :-(- Your first "continue 2" could probably simply be a "break"? - your last "continue" could be removed?

The NWC2 User Tool Starter Kit started with PHP4. The latest kit includes PHP5, which has a new object model which enforces a more restrictive use of a $this variable (which is predefined as an instance pointer in class object methods).