Stupid question: do all the Oric disks format share the same structure? I mean same track and sectors format and definition?
Sedoric, OricDos, FT-Dos, XL-Dos, Stratsed, RanDos, Pinforic disks, BD-Dos, the DOS for the Pravetz, the disks made by FloppyBuilder, by WriteDisk, or whatever other disk format?

The goal would be to know if I can write some universal sector reading routine, to make the Read Track command more reliable.

I'm sure in one thing: Pravetz DOS has completely different structure.
Else, if the goal is "universal sector reading routine", you can find it in Dbug's source 'loader.asm'.
It can read sector directly from FDC (i.e. not using ROM code at all).
I didn't test it yet on real hardware, but it should work reliable.

iss wrote:I'm sure in one thing: Pravetz DOS has completely different structure.
Else, if the goal is "universal sector reading routine", you can find it in Dbug's source 'loader.asm'.
It can read sector directly from FDC (i.e. not using ROM code at all).

Thanks, actually that's the question I should have asked: are all the Oric drives based on a FDC 1793 controller?
If so, whatever the structure, yes reading should be possible on all formats with the right FDC calls.

Oh and BTW, do you know if there's a way for the FDC to know the amount of tracks / sectors on a disk? I mean, if I read the track 42 / sector 9 and get an error, will I be able to know if it's because it doesn't exist, or because the disk is faulty?

The disk geometry/structure is something totally arbitrary: On the Oric as well as on the Atari ST or PC, you could have mixes of number of sectors, sector size, track density on the same floppy: You can totally have a MSDOS compatible boot sector (512 bytes per sector) that loads something that will read the rest of the floppy with tracks formated with 256 bytes per sector, and you could have half of it in HD format and the rest in DD or SD.

On floppy protections, it was common in the ST early days to have the DOS structure info to pretend it was a 80 tracks/9 sectors per track floppy, have a non formated track 80, but a formated and fully usable track 81 used to contain protection required data: Normal copiers would just copy the first 80 tracks and not see the new one, more advanced copiers would create a 82 tracks floppy with a normally formated track 80 that the game could read (and thus detect as non original).

Ok then I'll have to focus to my initial goal, which is
- using a FDC 1793
- read old Oric disks to turn them into DSK files (which excludes all disks made by recent tools from already existing DSK files)

My main problem is now double:
1- how to make the READ TRACK command more reliable
2- how to know how many tracks there are on the disk without reading any data specific to the system

I think I need first to understand clearly how the whle thing works with Sedoric (does it use the Microdisc eprom routines to read a disk once Sedoric is operating or not, things like that).
And if things could be different using a Jasmin ROM/drive.
See you in 2018

The Jasmin is using a different drive controller, but as far as I know it should be 100% able to read and write the same thing as what the Microdisc controler does.
It's totally possible to read the same floppy with both a Microdisc and a Jasmin, Fabrice proved that with his Fantasmagoric boot sector, and in theory FloppyBuilder is compatible with both systems (that's why there are two boot sector files), but unfortunately I never finished the compatibility part of the loader (Fabrice method was to use the X register to index the base I/O registers to make the code work with both controllers, I have probably fucked up something because I did not manage to get it to load on the Jasmin).

A way to detect non formated or protected tracks is to try to load them multiple time and compare the values: Many Atari and Amiga protections were based on specially written bits that could not be reliably read, they would return 0 or 1 randomly, but of course if you copied the floppy the copy would always return either 0 or 1, which made it possible to detect that it was not the original disk.

Dbug wrote:The Jasmin is using a different drive controller, but as far as I know it should be 100% able to read and write the same thing as what the Microdisc controler does.
It's totally possible to read the same floppy with both a Microdisc and a Jasmin, Fabrice proved that with his Fantasmagoric boot sector, and in theory FloppyBuilder is compatible with both systems (that's why there are two boot sector files), but unfortunately I never finished the compatibility part of the loader (Fabrice method was to use the X register to index the base I/O registers to make the code work with both controllers, I have probably fucked up something because I did not manage to get it to load on the Jasmin).

A way to detect non formated or protected tracks is to try to load them multiple time and compare the values: Many Atari and Amiga protections were based on specially written bits that could not be reliably read, they would return 0 or 1 randomly, but of course if you copied the floppy the copy would always return either 0 or 1, which made it possible to detect that it was not the original disk.

It's a "different controller" but based on the same controller, the difference is really slight, and both use the same command set. Both are the same controller as the one which is the basis for IBM/PC controller.

I don't have the time right know, but I some point I will make sure to test this on my side.
My suspission for your track read problem is at some point the controler need to resync with the floppy, and it may add some dummy sync bytes. It's highly possible that NIBBLE just do some assumption on the number of sync bytes, and discard some if there are too much. I don't know.

It would be interesting to try the same thing on different compatible computer (like a PC with a FDC controller)

The surface of a floppy is just a bunch of oriented magnetic regions. A 1 bit is flagged anywhere that a region with magnetism running in one direction transitions into a region with magnetism pointing in the other. Different drives run at slightly different speeds, and there's an aerodynamics factor, so the controller uses a phase-locked loop to try properly to space the 1s amongst 0s.

The disk needs to be able to store an arbitrary number of 0s in a row because it needs to be able to store an arbitrary data stream but you can't just leave an arbitrary distance between flux inversions and expect the correct number of 0s to be picked up. A phase-locked loop with no input over a surface with a slightly variable rotation rate is likely to lose synchronisation. So an encoding scheme needs to be derived that places flux inversions sufficiently regularly to keep the PLL synchronised.

Single density/FM was the most naive scheme possible — insert a 1 as a clock bit between every data bit. So there's never more than one 0 in a row. But a disk is an analogue recording surface so if you try to place flux transitions too close together, they'll bleed into each other. So spending so much of the available bandwidth on clock bits gives a relatively low data rate.

MFM is the modified version. It's just a smarter solution to the keeping the PLL synchronised problem. Take the same size of bit windows as FM. Keep the same rule that there'll never be more than two windows in a row without a 1 in them. But increase the underlying clock rate so that you can determine whether the 1 is in the left half or the right half of the window. You can achieve that with a simple rule — write and read at double the rate, but insert a 1 clock bit only if both the bit on the left and the bit on the right are 0. If either is 1 then there's already something there for the PLL.

Further complications are that (i) different sectors may have been written by different drives. So may be at different data rates; and (ii) the read and write heads in a drive are placed slightly apart and the physical density of bits is constant by rotation so different by track, implying that the two heads are a different number of bit windows apart depending on the track. Which, all together, means that sector contents may be spliced in to the existing stream with an abrupt change in phase and slightly aside from where the previous sector lay.

IBM resolved this in two ways:

leave a large gap between sectors to ensure that even a track formatted on the fastest drive and then modified on the slowest doesn't lose data;

start each piece of independently written data with a sequence of 00 bytes — which encode as 01010101, priming the PLL — and then a synchronisation word, for proper framing of the bytes that follow.

They defined two different synchronisation words, both of which are intended to be illegal byte encodings. $4489 is used for most purposes, being $A1 with the clock bit that should be at $20 missing. Missing a single clock is unlikely to be enough to throw off a PLL. There's also $5224 which is used only for the index hole. That's $C2 with the clock bit at $80 omitted.

When the WD is normally scanning the disk, any time it spots either synchronisation value it updates its framing for all future bytes. When performing a read sector, it switches off its byte-frame synchronisation during the body of the sector. When performing a read track, it leaves it on.

Unfortunately, $5224 is not just $C2 with an omitted clock, it's also the 16 bits in the centre of the 18 that result when correctly encoding the 9-bit data stream 000101000. So it crops up all the time in the middle of sectors. And each time it crops up during a read track, the WD is going to throw out its previous idea about byte windowing and return a read C2.

So you're therefore almost always going to see additional $C2s, depending on the particular data in the particular sectors on the particular track. It's not faulty hardware and they're not some freak race condition. You should be able to reread the same track over and over and always get the same $C2s.

Clues that you can use:

if a $C2 is spurious it'll occur partway through an expected byte period. So it'll come early;

if it's within sector contents then you can spot whether it's spurious by comparing to a read sector of the same place.

So you can in theory spot the first spurious $C2 through timing, do a manual decode on the data returned to make a guess about the sector it was in, read the sector to figure out how the $C2 threw framing off, shift the rest of the previous read track, determine what the next sector would be, read that to patch in contents, etc.

I believe the WD17[7/9]X support sector sizes only up to 2048 bytes. So sadly you can't even do a destructive (i) read track to get the real first few bytes; (ii) start a write track and write just enough to begin a really big sector; (iii) read the track again. Which I think you could on an IBM PC-style 8272, which is a completely different chip. It's also found in the Amstrad CPC and ZX Spectrum +3 if you're looking for something with that chip and a 3" drive.

Euphoric is emulating incorrectly by not inserting the $C2s but I think it comes down to the MFM_DISK file format. It's a heavily reductive, simplified model of a disk surface that can't tell you the difference between a $C2 that is ordinarily encoded and a $C2 or $A1 that has a clock missing. The missing clock makes a big difference to read track, read sector and read address on a WD so an emulator has to make a choice. With no defined guidance, I suppose each makes its own guess — mine has a process of guessing which appear probably to be outside sector contents, which may or may not align with other people's guesses. Similarly it makes no provision for data that is not exactly aligned at a constant clock rate with the index hole across the entire track at a consistent data rate. Presumably Euphoric is pretending that there's no such thing as clock bits, that alignment is perfect, and that the missing clocks aren't actually the trigger. But I'm speculating.

Summary: spurious $C2s are part of the WDs intended design. Not a hardware fault. They relate to a part of the floppy disk mechanism that the most popular Oric file format doesn't have any comprehension of. As a result don't trust any of the emulators as guides, as each is inventing their own way of dealing with the file format.

EDIT: I've a full emulation implementation of a synchronising read track if it would be of any help to throw a DSK at it and confirm or deny the output for you.

Oh! Wait! A distant spark fired, and I remembered the smart solution to two-drive copying. Not originally mine, so no credit due, alas.

A common feature of the Oric disk interfaces is to use one controller chip to support multiple drives. But neither of the WD chips used by the Microdisc or the Jasmin supports multiple drives. So drive selection is powered by external electronics.

So, yeah, you've probably skipped ahead of me already: the trick is to change which drive is in use while the WD is mid-command.

If you've a disk you're reading from and one you want to write to, write a series of the longest sectors available to the one you're allowed to write to. Frustratingly that's only 1024 bytes on the WDs. But write as many of those as closely together as you can. On one track start exactly from the index hole, on another write, say, a 512-byte gap before writing all your sectors as closely together as possible.

Then you're going to:

select the disk you wrote to, request a read of the first sector you wrote;

upon the beginning of input bytes, switch the active drive to the one you want to read;

discard at least the next byte as a mishmash, possibly a few more because there'll definitely be a phase jump and if the data rates differ then the PLL may really be momentarily confused;

keep every byte until the end of the sector — those will be bytes from the reading disk's surface which don't attempt to synchronise with the sync marks;

ignore the inevitable CRC error;

switch back to the first drive and attempt to read the second sector;

repeat;

do the same thing with the other track you formatted;

combine the results.

I believe the Microdisc has only a single concept of enabling the motor, so all drives should always spin at once.

You'll actually want at least one sector that straddles the index hole, but you should get at least one so that's good.

You may be unlucky and switch from the writeable disk to the one you're reading with such timing that you end up being fed clock bits rather than data bits. If your drives have exactly the same speed of motor and both disks have exactly the same weight and aerodynamics then you might be in a bind. But they probably don't. And at worst you can ask the user to eject either disk, rotate it a little and reinsert it.

... then your limit is what you can write, not what you can read.

If you're on a PC with a real FDC then I think you can write sector lengths that are longer than the track, and can turn your data rate up such that you get both clock and data bits from an '80s density disk. Which is useful for capturing things like Apple ][ / 8D DOS images on PC hardware.

Wow, thanks for the reply. I'm quite sure the answer to my question is there, but I have to confess I read it twice so far and felt lost before the end each time due to lack of knowledge Thankfully I think it has all the keys to do some Google and improve!

Stupid question but I still don't find the information: does anyone know where (if?) the disk format informations are stored on a FTDOS (Jasmin) disk?
Maybe it's not stored at all... I seem to recall someone saying that it was always double sided, 41 tracks and 17 sectors, but I don't find back where it could be...