If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register or Login
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Writing a custom Filesystem Format

There is one thing that I've always wondered about, and yet never seem to find when searching through google, tutorials, or even looking at programming textbooks.
The question seems easy enough, but the answer seems elusive.

How do you write your own Filesystem/Format?

I've always wanted to know how to write my own system for the learning experience.
I've worked on simple ones that read from a flatfile of a defined size, but never one that was on a fully formatted drive.

Re: Writing a custom Filesystem Format

You don't "write" a filesystem format, you define it. That's the "easy" part.

The hard part is getting the OS to recognize it and interface with it. This requires, either:
- A third party program that accesses the drives directly
- A driver for the OS
- Kernal inclusion

Needless to say, all of the above are far from trivial, even for professionals, and well outside the scope of plain old C++.

To put things into perspective, NTFS is still not fully supported on linux type systems. ext2 is supposedly supported on windows with thrid party drivers, but ext3 and ext4 aren't.

And I can tell you there is a LOT of pressure to get those working.

Is your question related to IO?
Read this C++ FAQ article at parashift by Marshall Cline. In particular points 1-6.
It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

Re: Writing a custom Filesystem Format

Actually the problem with NTFS is that the documentation on the spec was never supposed to become public. Bits and pieces leaked out over the years; whether or not Microsoft has finally just released the entire format is not something I've kept up to date on. I had to write code to pull the paging file off the disk (and what I found in there would make your skin crawl!!!).

Re: Writing a custom Filesystem Format

You don't "write" a filesystem format, you define it. That's the "easy" part.

The hard part is getting the OS to recognize it and interface with it. This requires, either:
- A third party program that accesses the drives directly
- A driver for the OS
- Kernal inclusion

Needless to say, all of the above are far from trivial, even for professionals, and well outside the scope of plain old C++.

To put things into perspective, NTFS is still not fully supported on linux type systems. ext2 is supposedly supported on windows with thrid party drivers, but ext3 and ext4 aren't.

And I can tell you there is a LOT of pressure to get those working.

I'm not referring to the likes of NTFS because, as CMalcheski said, it's a proprietary Microsoft filesystem that wasn't meant to become public.

I meant something more-or-less basic for testing. I'm just asking mainly about how to format a drive in a custom format.
I don't mean the OS drive either, just -- for example -- a USB key, or another partition.

I know it'd require different applications to access/alter/read/write information to it already.
I'm not saying to entirely incorporate it into the system and make it work with the basic file manager, I'm saying to build a separate one that will work with the new partition.

Re: Writing a custom Filesystem Format

Originally Posted by S_M_A

Go ahead and make something of you own mind but it doesn't matter, you still have to do what monarch_dodra says.

I'm not attempting to dispute what he's saying, just trying to understand it, which I actually get now.
At first I misunderstood what he meant, thinking that drivers would be to integrate the system into the OS's file manager -- but then I realized that's to recognize the actually drive/format, and detect how the file system is dealt with before data manipulation can even be achieved.

I realize that creation of one, in a whole, is not particularly easy by any means -- but it's far from unachievable. I'm just merely trying to learn, that's all. No disrespect intended.

Re: Writing a custom Filesystem Format

At Amazon search for a book Linux kernel development (3rd Edition ) , on page 261 page the book you will find virtual filing system , ( all filing systems under linux are implemented as such ) Study the interface and implementation ( language C ) after you are mastered this ( you can even submit some patches ) you are then in a position to develop your own. Writing a good filing system as hard as developing a kernel ( operating system ). Lot of things have to be mastered , like data structures in C, As you would Digg deep you will find only the best of developers get to work on these.

Re: Writing a custom Filesystem Format

Originally Posted by aamir121a

At Amazon search for a book Linux kernel development (3rd Edition ) , on page 261 page the book you will find virtual filing system , ( all filing systems under linux are implemented as such ) Study the interface and implementation ( language C ) after you are mastered this ( you can even submit some patches ) you are then in a position to develop your own. Writing a good filing system as hard as developing a kernel ( operating system ). Lot of things have to be mastered , like data structures in C, As you would Digg deep you will find only the best of developers get to work on these.

Thank you so much for that! Seriously, I wasn't aware of the book's existence, the only book I really have is Stroustrup's book the C++ Programming Language.

I love learning about kernal/file system development. I much prefer designing and working on the back end of systems

Also, if you want to get your hands dirty, as aamir121a said, the linux kernel development is a good place to look. As you have already discovered, there isn't many web resources for this topic, but if you download the linux kernel source and look at ../fs/ramfs you will see one of the smallest FSs implemented directly into the kernel.