Low level hard disk programming

Low level hard disk programming

Author

Message

kkar#1 / 11

Low level hard disk programming

I looked for information about hard disk's low level format on various groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took me to posts like: "dont do it", "it's hardware vendor specific", "the new ide doesnt make a real low level format". Now, I will make a few questions that involves low level programming and I like to see professional answers, not the simple answers like the above mentioned. Then, my questions are: - How is implemented the Int13,function 5 actually for hard disks? - How is implemented the bad sector marking on a hard disk? (not the FFF7h cluster mark in the FAT), but the real table management that knows which physical sector is really bad, something like the "track address fields" for a floppy disk format. - How does the BIOS low level format work and which are the supported disks (it means, vendor, capacity, etc). - ... I have other doubts, but first of all I want to know if there is anybody that really can help me. Thanks!

>Now, I will make a few questions that involves low level programming >and I like to see professional answers, not the simple answers like >the above mentioned. >Then, my questions are: >- How is implemented the Int13,function 5 actually for hard disks?

Look, if you don't believe us, go disassemble the damn BIOS yourself. It isn't that hard. You'll find that the controller in an IDE drive does a lot of the work independently.

You could also go download a Linux kernel and read through the disk driver code.

Quote:

>- How is implemented the bad sector marking on a hard disk? (not the >FFF7h cluster mark in the FAT), but the real table management that >knows which physical sector is really bad, something like the "track >address fields" for a floppy disk format.

There are spare sectors on each track of an IDE disk. At the factory, they scan the disk looking for physical bad blocks. If bad blocks are found, there is a table on the track that tells it which sectors are bad, and the drive substitutes some of the extra sectors instead. The microcode that controls all of this is usually stored in a special track on the hard disk, and is loaded into the drive's controller upon spin-up.

Unless you rewrite the controlware, you can't dink with the physical bad block table. --

Providenza & Boekelheide, Inc.

Tue, 01 Feb 2005 13:56:02 GMT

Michael Brow#3 / 11

Low level hard disk programming

Quote:

> I looked for information about hard disk's low level format on various > groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took > me to posts like: "dont do it", "it's hardware vendor specific", "the > new ide doesnt make a real low level format".

> Now, I will make a few questions that involves low level programming > and I like to see professional answers, not the simple answers like > the above mentioned. > Then, my questions are: > - How is implemented the Int13,function 5 actually for hard disks?

Easy answer: disassemble your BIOS and see.

Slightly longer, and probably dated, answer: [] Do some stuff to make sure the drive isn't busy (and exists etc), like checking the status port [] Load port 1F2h with the number of sectors to format [] Load port 1F3h with the starting sector number [] Load port 1F4h with the low cylinder number [] Load port 1F5h with the high cyclinder number [] Set sector size and head vie port 1F6 [] Maybe set write precompensation (port 1F1h), maybe not if the controller doesn't need it [] Fire command 50h (format track) to port 1F7h [] Fire the sector data table (identical to the data passed to function 5 int 13h) to the controller via port 1F0h (in words)

This is a summary of what is done, there are various checks needed to make sure you can go onto the next stage, and that the data you are sending to your drive is correct. Get hold of a copy of The Undocumented PC for a much more complete description, or there's probably many sites on the net that will help. Or read the Linux kernel source.

I don't see whay people would say "don't do it" for this question ...

Quote:

> - How is implemented the bad sector marking on a hard disk? (not the > FFF7h cluster mark in the FAT), but the real table management that > knows which physical sector is really bad, something like the "track > address fields" for a floppy disk format.

There's two layers of this. One is done automatically by the HDD itself, before any data is actually put onto the cable. You control this by what you send to the controller in the sector data table. How this is implemented depends a lot on the disk (sorry). Maybe it's stored in flash RAM on the HDD, maybe it's put onto the disk in a special spot, maybe it's done completely differently. All that you can know is that you tell it to remap the sector, and somehow it magically does it. What happens from there is not known (nor important, unless you are into forensics or something).

The second is putting a "bad sector" mark on the sector. This does not fix it, but just writes a byte to the sector data that when read back tells the HDD to report it as a bad sector. This effectively does nothing apart from provide one bit of storage per sector, for you to use in whatever way you want (however it should be used for bad sector marking).

Quote:

> - How does the BIOS low level format work and which are the supported > disks (it means, vendor, capacity, etc).

The people who say that you cannot low-level format drives now are telling the truth. To actually rewrite the *real* low level stuff, like sector placements, etc, you will need to contact the manufacturer. These real low-level formats are vendor- (and often model-) specific. So you can't do a low-level format. The best you can do is a medium-level format, which is what the "low-level format" option in your BIOS screen does (if you have one). It is easy to get the drive information from the controller in a standard format (command ECh), and from this the BIOS can do a format as above. This is as low as you can get. Some manufactures do release real low-level format utilities but I have not got around to reversing them to see what they actually do. Feel free to try :)

Quote:

> - ... > I have other doubts, but first of all I want to know if there is > anybody that really can help me. > Thanks!

Probably the IDE/ATA spec could help you a lot here :)

-- Michael

Tue, 01 Feb 2005 14:55:56 GMT

kkar#4 / 11

Low level hard disk programming

Thanks Tim, is nice to see that there is somebody that can help me.

Quote:

> But you don't believe them? What would it take to convince you?

I believe with proofs or reference to the proofs (source code in this case, because I am supposing that my question is read by programmers, not by casual PC users).

Quote:

> Look, if you don't believe us, go disassemble the damn BIOS yourself. It > isn't that hard. You'll find that the controller in an IDE drive does a > lot of the work independently. > You could also go download a Linux kernel and read through the disk driver > code.

These are reference to the proofs, you must be sure that I will do it.

Quote:

> There are spare sectors on each track of an IDE disk. At the factory, they > scan the disk looking for physical bad blocks. If bad blocks are found, > there is a table on the track that tells it which sectors are bad, and the > drive substitutes some of the extra sectors instead. The microcode that > controls all of this is usually stored in a special track on the hard disk, > and is loaded into the drive's controller upon spin-up. > Unless you rewrite the controlware, you can't dink with the physical bad > block table.

All of this is very interesting. I thank if you or anybody else can give me more information about it. I would like to see more reference of this theme (documents,links,source code,etc).

Wed, 02 Feb 2005 06:06:40 GMT

Orlando Llane#5 / 11

Low level hard disk programming

Quote:

> I believe with proofs or reference to the proofs (source code in this

You want proof? Low level format your own hard drive :)

See ya! Orlando

Wed, 02 Feb 2005 07:56:14 GMT

kkar#6 / 11

Low level hard disk programming

Michael: thank you very much. This is the answer I was waiting for. I am searching info about all of this, but I want to begin to program something now. First of all I am interested in knowing how I can do the bad sector marking that you name in:

Quote:

> The second is putting a "bad sector" mark on the sector. This does not fix > it, but just writes a byte to the sector data that when read back tells the > HDD to report it as a bad sector. This effectively does nothing apart from > provide one bit of storage per sector, for you to use in whatever way you > want (however it should be used for bad sector marking).

Also, I want to know if I can mail you ocasionally for other questions of this type, because as you can see here there are not real programmers that can help me.

Sun, 06 Feb 2005 08:56:03 GMT

\#7 / 11

Low level hard disk programming

On Fri, 16 Aug 2002 06:55:56 +0000 (UTC), "Michael Brown"

<snip>

Quote:

>The second is putting a "bad sector" mark on the sector. This does not fix >it, but just writes a byte to the sector data that when read back tells the >HDD to report it as a bad sector. This effectively does nothing apart from >provide one bit of storage per sector, for you to use in whatever way you >want (however it should be used for bad sector marking).

I wouldn't have thought that this would work for any HDD later than an MFM or RLL drive, and possibily early IDE drives. I do remember doing this type of thing back about 20 years ago or so, but not for PC type drives.

> Michael: thank you very much. This is the answer I was waiting for. I > am searching info about all of this, but I want to begin to program > something now. First of all I am interested in knowing how I can do > the bad sector marking that you name in:

> > The second is putting a "bad sector" mark on the sector. This does not fix > > it, but just writes a byte to the sector data that when read back tells the > > HDD to report it as a bad sector. This effectively does nothing apart from > > provide one bit of storage per sector, for you to use in whatever way you > > want (however it should be used for bad sector marking).

This is done through the formatting as described before. For each sector you want a "bad" mark in, set the relevant byte of the sector data table to 80h, to mark it as "bad". Then, when reading the data back (ATA/IDE command 20h), check the value of port 1F1h (hard disk error register) for each sector to see if there was an error. If the "bad sector" bit has been set for the sector, then bit 7 of the error register will be set. It is good practice to read this register anyhow, as it reports stuff like ECC errors etc. I'm not sure how this all works when using DMA transfers however.

Quote:

> Also, I want to know if I can mail you ocasionally for other questions > of this type,

Sure no problem. I've updated my signature to let people know this now :)

Quote:

> because as you can see here there are not real > programmers that can help me.

There are a *lot* of good programmers on this group: Orlando Llanes and Randall Hyde spring immediately to mind as they are in my address list :) However, this is not the best place for low-level stuff like this, probably as OS development group might be able to help more.

-- Michael Brown My inbox is always open (remove the obvious):

Sun, 06 Feb 2005 15:56:00 GMT

Michael Brow#9 / 11

Low level hard disk programming

""Arargh (CircleAroundAnA NOSPAMarargh decimal NOSPAMcom)" "

Quote:

> On Fri, 16 Aug 2002 06:55:56 +0000 (UTC), "Michael Brown"

> <snip> > >The second is putting a "bad sector" mark on the sector. This does not fix > >it, but just writes a byte to the sector data that when read back tells the > >HDD to report it as a bad sector. This effectively does nothing apart from > >provide one bit of storage per sector, for you to use in whatever way you > >want (however it should be used for bad sector marking).

> I wouldn't have thought that this would work for any HDD later than an > MFM or RLL drive, and possibily early IDE drives. I do remember doing > this type of thing back about 20 years ago or so, but not for PC type > drives.

Yup, works fine on a 2 gig Quantum Fireball (yeah, I know, this is stone-age stuff now ... ). I haven't tried it on my 20 gig HDD because I have stuff on it I don't want to lose :)

Format Track was in the ATA-1 (aka IDE) standard, was marked as "optional/vendor specific" (along with the note "It is recommended that system implementations not utilize this command." in ATA-2 and ATA-3 (both also called EIDE), and was gone from ATA/ATAPI-4 (aka UDMA-33), in the form of being marked obsolete. This is all from the specs at http://www.t13.org/

Another good description of IDE here: http://www.gaby.de/gide/IDE-TCJ.txt

I would guess that most older (ATA-3 and before) drives would support it, quite possibly just emulating it though and not actually doing any formatting. I have no idea about ATA/ATAPI-4 and later.

-- Michael Brown My inbox is always open (remove the obvious):

Sun, 06 Feb 2005 16:55:58 GMT

Orlando Llane#10 / 11

Low level hard disk programming

Quote:

> There are a *lot* of good programmers on this group: Orlando Llanes and > Randall Hyde spring immediately to mind as they are in my address list :)

:)

kkarma's not worth any of our flaming time. According to him, if you don't know how to low level format a hard drive with assembly language then you don't know crepe. Pardon my French ;) For the annoyed, I promise that this is my final post on the subject.

See ya! Orlando

Mon, 07 Feb 2005 01:56:09 GMT

do..#11 / 11

Low level hard disk programming

Quote:

>I looked for information about hard disk's low level format on various >groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took >me to posts like: "don4t do it", "it's hardware vendor specific", "the >new ide doesn4t make a real low level format". >Now, I will make a few questions that involves low level programming >and I like to see professional answers, not the simple answers like >the above mentioned. >Then, my questions are: >- How is implemented the Int13,function 5 actually for hard disks?

This is not fully done on new systems because the IDE interface does not use this information anymore.

The modern drives of the IDE/ATA variety actually use the LBA absolute sector number for everything, No matter how you address the drive.

To do the format like you are talking about you must first tell the drive to use your CHS cylinder, head, sector values.

Then you just go ahead and find the appropriate commands in the list IDE/ATA low level stuff and send it.

Quote:

>- How is implemented the bad sector marking on a hard disk? (not the >FFF7h cluster mark in the FAT), but the real table management that >knows which physical sector is really bad, something like the "track >address fields" for a floppy disk format.

These are generically known as P&G tables. Permanent and gathered errors table or list.

This idea comes from SCSI which is where much of the modern improvments in IDE have come from. Approximatly 3% of the drive on a physical track by physical track basis is set aside to catch gathered errors as they occur.

Then you factor in the 6000Tpi of tracks per inch density. the track are the same as cylinders (concentric rings goinf outward.) You actually have about an inch of track width on a 3 1/2 drv. so you actually get something like.

6000 * 4 * 6804 is where you end up.

Now if it was me I would just use the 6144 (1024 * 6) as my SPT (Sectors per track) or about 3Megs per physical track on the media.

This illustrate the departure of the IDE/ATA technology from it's FM / MFM / RLL / ESDI roots. The sector address no longer represented the physical address of the sector. This now represented a virtual position and could be changed at will by the onboard electronics for data preservation. All commands to access the P&G tables are vendor specific.

Quote:

>- How does the BIOS low level format work and which are the supported >disks (it means, vendor, capacity, etc).

Please see the above then add this to it.

Modern drives use a two layer approach to a format. The sector boundary's are no longer variable.

Now there is an inner layer of hard coded sectors that the onboard electronics can only read nor write. Then there is the soft coded parts that contain sector specific data. This can be CHS placement Information in addition to LBA address & sector data.

Quote:

>- ... >I have other doubts, but first of all I want to know if there is >anybody that really can help me.

If you really want to get into the nitty gritty then I suggest you grab the BT168.zip off of my site and study it closely.

Messy though the code is, poorly documented it may be. How ever it runs spinrite. It also handles upto 128 Gig drives. in CHS upto 8-Gig and in LBA from there up.

Done myself from the ground up. When I get time I will be cleaning up the code and makeing some real documemtation for it.

Quote:

>Thanks!

walk.to/doors

-- Doors - Dont look at the future in a window. just walk to the door http://walk.to/doors - Open it and go there.

http://www.humboldt1.com/~doors/doors.htm OR SEARCH AT http://www.humboldt1.com/~doors/search.htm