Edit:
New version of the sound driver is available: grab v2a here.
New version of SMPS2ASM is available: grab v0.2 here.
New version of the song pack is available: grab v4a here.

Changes:

driver: now with support for playing songs with 6 FM channels and no DAC channels;

driver: universal voice bank (UVB) is now included in every music bank; while this reduces the space in banks, this allows much more freedom for expanding the sound driver;

driver: UVB has been SMPS2ASM'ed;

driver: all fixes and changes are now always on;

driver: driver makes use of local labels (instead of nameless temporaries) and uses S2-style memory management for the z80 RAM (instead of hard-coding everything);

driver: several new coordination flags and fixes based on other drivers from ValleyBell's research pack; specifically:

meta CF $FF08 (smpsChanTempoDiv) which works like CF $E5 in S1/S2 drivers;

meta CF $FF09 (smpsChanFMCommand) that allows sending FM commands to the appropriate YM2612 part for the current track;

meta CF $FF0A (smpsNoteFill) that works like $E8 did in S1/S2 drivers;

meta CF $FF0B (smpsPitchSlide) that allows you to do pitch slides (use smpsAlterPitch to set slide speed);

meta CF $FF0C (smpsSetLFO) that lets you enable/disable the LFO and set AMS and FMS.

SMPS2ASM: can now dump z80 banks given an offset to a pointer table within the bank;

SMPS2ASM: simple dataflow analysis added to propagate key displacement changes throughout a track; this is used to fix the handling of the maximum-frequency PSG note (since it is different in S1/S2 and S3/S&K/S3D drivers);

song pack: redumped all songs with the new SMPS2ASM;

song pack: S3, S&K and S3D songs (and SFX for S&K) now have two versions: the 1:1 versions with unused data and the "optimized" versions without it; the optimized versions also have ALZ song fixed.

A warning: songs dumped with the new SMPS2ASM MUST be used with the "_smps2asm_inc.asm" file present on the sound driver file; earlier versions of the include file will fail. Moreover, I strongly recommend that you redump old songs so that they sound correct due to the PSG maximum-frequency note issue mentioned above.

To use the driver, you will need to either use the constants for zQueueVariables, zMusicNumber, zSFXNumber0 and zSFXNumber1 like what is descriped in Alriightyman's guide to use it in S2 or use the exported symbols for these constants to reference the correct z80 variables.
Edit: v3 is out. The changes are:

Due to its effects in some S3, S&K and S3D songs, I undid the following bugfix:

[bugfix] original S3/S&K/S3D driver could not correctly handle playing the sequence "<note>, <duration>, $80, <duration>, <duration>", and would play <note> again instead of a rest.

S1 and S2 songs that were affected by the previous change got a new version that explicitly puts a $80 before the duration of each rest;

The "_spms2asm_inc.asm" file now handles the differences in modulation flags that are explained in this post by ValleyBell;

I have made a release of the SMPS2ASM tool.

For the SMPS2ASM tool, I am providing a Windows binary; for those of you that don't use Windows, I am also providing the source code; it is bundled with the source code of my S2 Special Stage editor, which you can nab here.

Edit: v2 is out. Now includes S3's B2/B3 DAC samples and corrects the Sonic 3 Miniboss and Knuckles songs to use it. Also features the ASM'ed version of the credits medley from Sonic 3 Collection prototype 0517.

Original post follows:

As I have been saying elsewhere, I have written a tool that converts a binary music or SFX file (in SMPS format) to an 'universal' ASM format. I was planning on releasing this some two weeks ago; the delay comes about as a result of having to find and fix sound driver bugs, followed by illness; so sorry about that.

The original idea for this kind of tool is Cinossu's (see his utility); I just made a better one that handles songs and SFX from Sonic 1, Sonic 2, Sonic 3, Sonic & Knuckles and Sonic 3D Blast. I will not be releasing the tool just yet: when I started writing it, I had no idea that the SMPS formats (coordination flags, mostly, but also DAC samples) changed so much between Sonic 1/2 and the others; I hacked them in, but my intention is to rewrite the tool so support for any given game is simply a matter of pointing it to a text file containing definitions.

But there is no reason people should be denied the fruits of it; specifically: this file. (edit: updated link) The contents are:

All songs, SFX and DAC samples* from Sonic 1;

all songs, SFX and DAC samples* from Sonic 2;

all songs, SFX and DAC samples** and SFX from Sonic & Knuckles;

all songs and DAC samples** from Sonic 3D Blast;

all songs and DAC samples** from Sonic 3 that are not identical to songs from Sonic & Knuckles;

_smps2asm_inc.asm, a file that does the bulk of driver conversion;

'Z80 Sound Driver.asm', a modified and improved version of the Sonic & Knuckles driver;

all DAC samples from Sonic 2 (of which the Sonic 1 samples are a subset), Sonic & Knuckles and Sonic 3D Blast.

The included sound driver has tons of changes and fixes compared to the SVN version (some of which I will eventually optionally backport to SVN). They include:

[enhancement] You can select which DAC groups to use;

[enhancement] added a coordination flag for setting the tempo divider of a single track;

[enhancement] added a coordination flag for setting note fill in the same way it is done in Sonic 1/2 drivers;

[enhancement] added a coordination flag to send an FM command to the correct port (from Ristar);

[enhancement] added all PSG tones (actually volume flutter) from Sonic 1/2 (the Sonic 1 flutters are a subset of the Sonic 2 flutters);

[enhancement] removed unnecessary padding between songs;

[bugfix] original S3/S&K/S3D driver could not correctly handle a rest followed by a 'no attack' coordination flag, followed by a note;

[bugfix] original S3/S&K/S3D driver could not correctly handle playing the sequence "<note>, <duration>, $80, <duration>, <duration>", and would play <note> again instead of a rest.

As of this time, the driver is a drop-in replacement for the S&K disassembly; in the future, I will make edits to kram1024's tutorials to use this driver in Sonic 1 or Sonic 2 (unless kram1024 comes back and does it, as we were talking about before he disappeared).

The _smps2asm_inc.asm file is based off of one from Cinossu's tool; I revamped it and updated it to perform the bulk of the conversion. It handles different pointer formats, different IDs of DAC samples, handles the 12-semitone (1 octave) difference between PSG notes in S1/S2 versus S3/S&K/S3D, and does the bulk of coordination flag conversion. While this makes it trivial to convert songs and SFX in general, converting to Sonic 1 or Sonic 2 drivers is a lot harder (unless you are converting from Sonic 1 or Sonic 2) due to the lack of necessary coordination flags.

Please feel free to ask any questions that aren't clear after looking around in the driver (or which I failed to convey in this message...).

* = Sonic 1's DAC samples are a subset of Sonic 2's DAC samples.
** = Sonic 3's samples are a subset of those from Sonic & Knuckles. Sonic 3D Blast samples are composed of $1D out of the first $1E DAC samples from Sonic & Knuckles, plus two unique samples.

Glad to hear that you've finally released the modified S3&K z80 sound driver, and the ASM'd music! This should make music porting and hacking much easier . Now I'm just waiting for SMPS2ASM.

Quote

The _smps2asm_inc.asm file is based off of one from Cinossu's tool; I revamped it and updated it to perform the bulk of the conversion. It handles different pointer formats, different IDs of DAC samples, handles the 12-semitone (1 octave) difference between PSG notes in S1/S2 versus S3/S&K/S3D, and does the bulk of coordination flag conversion. While this makes it trivial to convert songs and SFX in general, converting to Sonic 1 or Sonic 2 drivers is a lot harder (unless you are converting from Sonic 1 or Sonic 2) due to the lack of necessary coordination flags.

If I'm reading this correctly, this means no longer do hackers have to use Music Pointer Fixer (MPF) or do manual pointer fixing to fix the pointers for music for z80 sound drivers? All I have to do now is just "include" ASM'd music files to the S3&K driver for my S3&K hack, and I'm ready to go? Awesome, I didn't think such a thing was possible! I'm going to try the new driver, and see if it improves the playback of some of my ports for my S3&K hack. (Some Ristar songs and S2F songs sounded wrong in the previous driver.) Thanks for the release.

So I tried replacing the "Z80 Sound Driver.asm" file in my S3K disassembly with the new one and building outright and I get a number of errors, ranging from "Too many operands" to "Generic error". I tried looking at the .asm file of both the new sound driver and the conversion tool and it doesn't seem to be formatted correctly; as in it seems illegible compared to other .asm files I have used.
I think I might just be missing something obvious, but maybe not...?

Marvellous. Looking forward to digging into this once I have some computers that aren't in pieces.

Since the tool isn't out yet, any chance of converting the Sonic 3C 0517 credits medley to add to the selection? I can send you a pre-extracted binary file if it helps.

Also, one thing:

flamewing, on 05 November 2011 - 01:39 PM, said:

** = Sonic 3's samples are a subset of those from Sonic & Knuckles.

There's an exception to this: S&K provides a different sample from S3 for B2/B3; this is used in the S3 miniboss and Knuckles themes and the S&K Knuckles theme. Is there anything in the driver to deal with this? If not, do you think it would be easy to add? This is the last thing in Sonic 3 Complete that's accomplished by 68K meddling (I fiddle the DAC bank pointers for those samples depending on the track selected) so it'd be nice to cross that one off.

For completeness, it's perhaps also worth mentioning that there are some very, very slight differences in channel setup in a few of S3's sound effects, but I can't imagine offhand why anyone would want the S3 versions - all it does in the existing game is cause some bad music cutout in DEZ3 and a couple of other places (I think SSZ, which may partly explain why that got a similar fiddle before S&K release).

If I'm reading this correctly, this means no longer do hackers have to use Music Pointer Fixer (MPF) or do manual pointer fixing to fix the pointers for music for z80 sound drivers? All I have to do now is just "include" ASM'd music files to the S3&K driver for my S3&K hack, and I'm ready to go?

Yes, you are reading this right. If you want to see it in action, I have included several Sonic 1 songs into Sonic 2: Heroes in the past few versions; while it (still) uses the Sonic 2 driver, I also only had to include the ASM'ed files plus the converter file.

D.A. Garden, on 06 November 2011 - 01:05 AM, said:

So I tried replacing the "Z80 Sound Driver.asm" file in my S3K disassembly with the new one and building outright and I get a number of errors, ranging from "Too many operands" to "Generic error". I tried looking at the .asm file of both the new sound driver and the conversion tool and it doesn't seem to be formatted correctly; as in it seems illegible compared to other .asm files I have used.
I think I might just be missing something obvious, but maybe not...?

It wasn't you, no; I forgot to convert the files to DOS line endings -- see, I use Linux. For me, it works because I run as under Wine, which handles all line endings. I have updated the first post with a link to the corrected versions; it is also here for convenience.

Tiddles, on 06 November 2011 - 05:08 AM, said:

Since the tool isn't out yet, any chance of converting the Sonic 3C 0517 credits medley to add to the selection? I can send you a pre-extracted binary file if it helps.

It is slightly better if you send the link to the ROM and the address of the song; but if the song is self-contained (that is, does not use a track or the voice bank of another song, like some S3/S&K songs do), there is no difference. My tool extracts all reachable music tracks and voices, even if they are not in sequence, as long as it can find them.

Tiddles, on 06 November 2011 - 05:08 AM, said:

There's an exception to this: S&K provides a different sample from S3 for B2/B3; this is used in the S3 miniboss and Knuckles themes and the S&K Knuckles theme. Is there anything in the driver to deal with this? If not, do you think it would be easy to add? This is the last thing in Sonic 3 Complete that's accomplished by 68K meddling (I fiddle the DAC bank pointers for those samples depending on the track selected) so it'd be nice to cross that one off.

I didn't know about that different sample, no; I should have double-checked. I can easily modify both the tool and the driver to compensate for this. Do you have the address/size/pitch of the different sample(s)? Bonus points if you also have them extracted :-)

Tiddles, on 06 November 2011 - 05:08 AM, said:

For completeness, it's perhaps also worth mentioning that there are some very, very slight differences in channel setup in a few of S3's sound effects, but I can't imagine offhand why anyone would want the S3 versions - all it does in the existing game is cause some bad music cutout in DEZ3 and a couple of other places (I think SSZ, which may partly explain why that got a similar fiddle before S&K release).

Yeah, I did extract the SFX and saw those slight differences. They were minor enough that I decided not to upload the S3 SFX in favor of the S&K ones.

It is slightly better if you send the link to the ROM and the address of the song; but if the song is self-contained (that is, does not use a track or the voice bank of another song, like some S3/S&K songs do), there is no difference. My tool extracts all reachable music tracks and voices, even if they are not in sequence, as long as it can find them.

I didn't know about that different sample, no; I should have double-checked. I can easily modify both the tool and the driver to compensate for this. Do you have the address/size/pitch of the different sample(s)? Bonus points if you also have them extracted :-)

Everything in the setup is identical to the S&K version (pitch $16 and $20 for B2 and B3 respectively) - the only thing that differs is the sample data, which is the same length as the S&K version too ($848). Both B2 and B3 share that sample data, which I've now uploaded. It can be found at $F3465 in S3A. $F015F and $F0164 are the headers.

The update seems to have fixed a few of the errors, but I am still getting a lot of "operands" based issues...
[snip]
I can't figure this one out. I also apologise for the wall.

Update from SVN. The fix for this issue has been there for over a month.

Tiddles, on 06 November 2011 - 12:30 PM, said:

OK then - it's at $C633A in Sonic 3C prototype 0517.
[snip]
Everything in the setup is identical to the S&K version (pitch $16 and $20 for B2 and B3 respectively) - the only thing that differs is the sample data, which is the same length as the S&K version too ($848). Both B2 and B3 share that sample data, which I've now uploaded. It can be found at $F3465 in S3A. $F015F and $F0164 are the headers.

The update seems to have fixed a few of the errors, but I am still getting a lot of "operands" based issues...
[snip]
I can't figure this one out. I also apologise for the wall.

Update from SVN. The fix for this issue has been there for over a month.

Yeah, that's a problem. I barely know how to use SVN as it is. Plus I've been working on the released disassembly of S3K for quite some time now so I don't know how that would work; updating stuff (and possibly losing work in the progress) just for an updated sound driver. It seems yet again this is something that is way over my head. I appreciate the release though; I'm sure someone can make use of it.

Yeah, that's a problem. I barely know how to use SVN as it is. Plus I've been working on the released disassembly of S3K for quite some time now so I don't know how that would work; updating stuff (and possibly losing work in the progress) just for an updated sound driver. It seems yet again this is something that is way over my head. I appreciate the release though; I'm sure someone can make use of it.

You can also copy over the sonic3k.macrosetup.asm file, as it is all it takes to fix the issue. This is slightly harder with WebSVN down; so here it is. You will also need to update Size_of_Snd_driver_guess to at least $1580 (in sonic3k.asm).

While I'm here, I may as well lay out the the issues I've had with the original S&K driver in general in case you have any insight on them or feel like making some optional tweaks for them, since I think some of this would be helpful to others too - if nothing else it's worth making everyone aware of this stuff I think. No worries if you don't want to make any changes for it.

Sometimes the sound driver "forgets" to apply a music track change that was queued during a 1-up. You can reproduce this in S3K by giving yourself 99 rings, then collecting another one for a 1up, immediately turning Hyper, and then breaking any shield or ring monitor while the 1up music is still playing. The transition to invincibility music is lost and the zone music resumes.
I think I've figured out why this is happening, too: the game code uses Play_Sound instead of Play_Sound_2 to play the sound effects for the items in those monitors, which is resulting in the sound effect being queued as the music to fire off when the 1up finishes. So really this is something that is probably best fixed in the game code itself, and it turns out it sort of ties in with point 4... unless there is ever a legitimate reason to play SFX through zMusicNumber instead of the zSFXNumbers?

Tempo changes always apply immediately, even during the 1up music, reverting to the previous tempo after its completion. In terms of Sonic games' actual use of the tempo feature, it would make more sense if tempo changes are queued to be applied after the 1up track finishes, like music changes. This would make reinstating the correct 1up sound during Sonic 3 special stages work sensibly, for instance, or collecting super sneakers while 1up music is playing.

It would be nice to have an easy way of defining multiple tracks to have 1up lock/fade-in properties - although this is probably pretty niche to S3C. At the moment I deal with this by just adding more conditionals for different track IDs which isn't too bad.

I've been thinking about whether it would be possible to operate music and control effects on a totally separate set of IDs from sound - so for example, you could have music tracks from $01-$40 when calling Play_Sound but still keep sound on $32-$DB when calling Play_Sound_2. This would give loads of room to add extra music and sound effects. Given that the 68K code uses different functions depending on which it's trying to play (and even calls the right one some of the time!) and puts them into different addresses for the Z80 to pick up, it seems like it might be possible. Having had a quick flip through the code today, I was thinking it might be possible to do this fairly simply by re-engineering zPlaySoundByIndex to look at the zUpdatingSFX flag. I'd be interested in your thoughts on whether this is a viable path and what other obstacles the driver might present if I tried this myself.

Thanks once again for all the work you've done here - I still haven't got around to integrating it but I really can't wait. (Except that I kind of have to wait. But hey, you know what I mean.)

While I'm here, I may as well lay out the the issues I've had with the original S&K driver in general in case you have any insight on them or feel like making some optional tweaks for them, since I think some of this would be helpful to others too - if nothing else it's worth making everyone aware of this stuff I think. No worries if you don't want to make any changes for it.

Sure, lets see:

Normally, there is no difference between the queue slots; during v-int, the z80 will copy "music" and "sound" entries into an internal FIFO queue and go through all entries in the same way. In general, it will be hard to trigger multiple stuff using Play_Sound in the same frame, but anything that does will override the previously queued one. However, when a 1-up is playing, things behave differently: first, the internal FIFO queue is not updated while the 1-up is playing (so entries in the queue stay there longer and have a bigger chance of being overridden); second, if any 1-up (or the S&K credits song) queued for play during this time (with Play_Sound), the entry is cleared instead; third, anything queued with Play_Sound_2 will have their entries cleared. So what happens is: trigger 1-up (fade out song/SFX); turn hyper (queue super song; does not go to internal FIFO queue on v-int); break shield or ring monitor (uses Play_Sound, overwrites the queued super song). For a fix, changing the monitor items to use Play_Sound_2 would do it. I actually noticed that something like this might happen while disassembling the driver, but I didn't think it would be possible to make it noticeable in the original (unmodified) games.

The way Sonic 3 (and up) do tempo is quite different from what Sonic 1 and 2 did: the latter had alternate main tempo values for each song which would be used during super sneakers (and hence songs could remain unaffected by it by just having the same main tempo as they had for the song itself), whereas Sonic 3+ play tracks twice in a frame every N frames (N can change -- for example, in special stages), so every music is affected the same way. The problem here is that the 68k itself sets the new tempo value; it gives the z80 no chance to do anything about it. I suppose that I could make that into a 'queue' that is updated on v-int and even only when 1-up song is not playing. That would still require another change so 1-up song does not play sped up.

That is the best way; it requires 2 new conditionals to work correctly. I suppose I could add comments on where it should be, or maybe check it by way of macros with multiple parameters.

I think it could work. You will have to look at the zFillSoundQueue and zCycleSoundQueue functions too, but by setting a flag after the first call to zCycleSoundQueue (and making sure the 68k calls the right functions!), it could work. You probably won't be needing the S1/S2 volume flutters, so I think you have enough space to do it too. Right off the bat, I can't see any more problems; but I am interested in helping do this.

I was thinking the same thing about point 1 - and if I'm going around fixing calls for point 4, it should take care of this too. Your description does raise another issue though, of collecting a second 1up while the first is playing, which sounds like it would also wipe a pending music change... I don't know if routing 1up through Play_Sound_2 would be a viable fix, since we want it to get lost in this case, but that would make separating music and sound functions more of a headache again.

The queue idea sounds very good for tempo - it sounds like quite an undertaking, but it really makes a lot more sense in the context of Sonic games, so I'd be very pleased if you did decide add such a thing. Thinking about it, it would also be nice to have a way to request a music track with an initial tempo change (e.g. collecting invincibility with sneakers active, restarting zone music from an event while sneakers are active, etc.) At the moment I do nothing about the former and for the latter, I create a wait object to run Call_Sound_Event a few frames later - and by "a few frames", I mean "some number of frames that seemed to work most of the time", so if I continue with that approach, do you have any ideas on a more scientific frame gap?

I'll stick to the old conditionals for the different 1up tracks for now then - it's good to have the approach validated.
A macro would be nice but I'm not sure who else would ever have a use for it, so I'm happy enough to tweak it myself.

Having had a look through today, with reference to your info here, I think I see a reasonable way to make the music/sound ID split work; I'll have to try it and see. I am concerned regarding its impact on the coordination flag FF/01 to play another music/sound effect by ID, though - without changing the actual flag format, I imagine it would end up being restricted to either music or sound only. Is this flag actually used by anything? Sound effects would seem a more sensible option to leave available, unless something relies on calling fade effects, but I could rig those up to work in both instances if necessary. Regarding the comment on flutters and space - are we running close to a space limit in the current version then?