We need to figure out how the video is stored inside the AV001_0AV001_00.MPG file. I was thinking that there would be some links like this:

record in TI000_0TI000_00.IFO -> somewhere in TM001_0TM001_00.MAP -> somewhere in AV001_0AV001_00.MPG

So far I haven't be able to find any of the links. I don't think you will be able to get a good clip off the drive without finding where the segments live. As I found that the ratio between the MPG_SEG_MPG_SEGM.DAT file and AV001_0AV001_00.MPG file are changing with disk size. The segment lengths should get larger on the larger disk drives.

Pete, if you want to try looking through the directory entries in the TI000_0TI000_000_00.IFO files. That might help us figure out the links. I don't think we will get very far with the video file extraction until we can figure out this relationship. Some of it is definitely index array type thing. I know the TI000_0TI000_000_00.IFO looks like it is arranged that way anyway.

Example have array A. A[1] is first record, so if you see a A[200] you have to do:
Location of A[200] = (size of one record of A) * 200 + (A[1] location)
The indexing come in where you have a result from one array feed another array.
Ex. I is an index for A
A[I[5]]
It does appear that the TI000_0TI000_00.IFO is arranged this way for storing the titles and chapter headers. More about that later.

iBoard has a nice feature in it. If you highlight up to for bytes in the view mode down is the hex area. It will display the little-endian and big-endian numbers, up to four bytes at a time. This file format the Magnavox is using the little-endian.

So, until now you have used successfully iBoard, and which other software?

iBoard, which runs on Windows, is what I've been using to see what is on the hard drive. It will copy blocks, write blocks as well as view and edit data right on the hard drive. It makes it easier to see what is on the drive. I would think there are other block hex editors for the Mac and Linux systems as well. I just haven't looked for any.

We are hoping to be able to figure out the file format used on the Magnavox units to be able to resize a file system for a drive upgrade. Also, other people would like to extract the video directly from hard disks.

We are hoping to be able to figure out the file format used on the Magnavox units to be able to resize a file system for a drive upgrade. Also, other people would like to extract the video directly from hard disks.

I did give isobuster a go with the image I pulled off my 500gb hard drive with iBored...unfortunately what it seemed to do is detect a bunch of MPGs and the ocassional other filetypes in between...and when I extracted the MPEGs they all seemed ok and played fine in windows media player, and were all found 'in order of recording', but some files were detected as being JPGs or other types, and when I renamed them as MPEG files, they played fine as well..so something different was being detected by isobuster...something to disrupt it from thinking it was one huge mpeg with pointers, or thinking it was another file type altogether..

One other thing I noticed, was that the mpegs that were extracted had some unusual time cuts... it wasn't like a half hour show would be one MPEG..the end of one file and the beginning of another might be in the middle of a show...and if you put the two pieces together in a video editor like Sony Vegas, it wasn't just a smooth transition from one clip to the next - there was usually a half second gap or something else noticeable...so the whole process is interesting, but not exactly a solution unless you really wanted just a particular clip, were trying to rescue footage, etc. I still seem better off just extracting that one huge file...but am still searching.

Thanks for that link on the Philips discussions...that's what got me trying tis solutoin...but I'm just not sure if the person that tested using isobuster with their extracted Philips DVR hard drive data noticed 'many small files that could all be reassembled' like me....and actually tried to do this and found what I did...or if they just saw a bunch of files and said 'oh here it is, i just have to assemble them sometime' assuming it could all cleanly be dumped into Pinnacle to reassemble the parts they wanted...

Copperblade, thanks for the tips. Maybe Sleuthkit will help. I hope to find some time to work on this. This is what I do know.

There is no partition table, sector 0 is all 0s.
The format is setup in a way to do one thing and that is to store and edit videos. Some of the files have copies, apparently for keeping a working copy and an edit copy.
Some of the data is linked by position. This big problem is still how to jump from the .MAP files to the video stored in the .MPG file. I think if we can figure that out we can easily get the mapping from the VBI_DATVBI_DATA.DAT to the .MAP file. I think the .MAP file is stored in to sector blocks with the first sector pointing into the .MPG file just based on it's position in the .MAP file. So the first two sector block points to the first block in the .MGP file and the second two sector block points to the second block in the .MPG file and so on. Just a theory right now, I have not confirmed this yet.

Yeah partition tables are pretty meaningless when you're using the entire disk. I routinely make filesystems without partitions for my storage devices.

I was thinking about whether or not they're using the space as just a big caching area, or have some kind of filesystem underneath. Either way, I don't think there is any way for the data to be sequential. Maybe large parts of it are, but it can't be guaranteed. I don't think it can even be guaranteed that a single title will be on the disk in one big sequential chunk because that means after storing and deleting multiple times you will get fragmentation with useless space that eats up the drive. (Also since it lets you record on the fly, there's no way for it to know how much space on the disk it will need to reserve.)

I think the best way to figure this out is to start with a fresh, unused drive in one of these devices and make a snapshot of the device. Then make a snapshot after each of:

1) Record one title
2) Record a second title
3) Delete one title
4) Record a third title

That would be the systematic way to get enough information about the underlying method of storing the data. Maybe to make the process easier, you could pick a small drive and just make sure the contents have been zeroed a few times.

Yeah partition tables are pretty meaningless when you're using the entire disk. I routinely make filesystems without partitions for my storage devices.

I have been doing some of my testing on a 40 Gig drive. The information at the beginning takes up about 1 Gig, only two files changes with the filesystem size and that is the MPG_SEGMPG_SEGM.DAT file and the AV001_0AV001_00.MPG file. I believe the MPG_SEGMPG_SEGM.DAT file is a used / free hash table for the AV001_0AV001_00.MPG file. I currently have an image that I was looking at with three short 1 minuet videos on it. I didn't try deleting the second file before recording the third though. Hopefully I can look at this some this summer.

I don't want to open a new thread, and since this one is related to HD's tests, will an SSD (Solid State Disk) work fine with a Magnavox? Maybe those devices help with the advance of this investigation.

I don't want to open a new thread, and since this one is related to HD's tests, will an SSD (Solid State Disk) work fine with a Magnavox? Maybe those devices help with the advance of this investigation.

I don't see why not. They do cost a lot more than a hard drive. But they are faster and lower power than even the notebook 2.5" drives. I put a 2.5" AV hard drive in my main 2160A.

On the video extraction front. I have been looking at a lightly use drive image and have been able to extract most of the video that was stored on it. I still can't figure out how to go from the directory listing found in TI000_0TI000_00.IFO to the video in the AV001_0AV001_00.MPG file. I know the information should be in the TM001_0TM001_00.MAP and the TC001_0TC001_00.CHP. But I can't seem to connect the dots. I did decode the time length part of the directory listing that you see while looking at the titles.

Maybe I can find sometime soon and post what I have so far and someone else could connect the dots.

Could you share that file? I'm realy interested to take this to the next level.

Thank you!

I'll try compressing the entire 40 Gig hard drive. Since most of it is blank it is currently doing about 1000:1 ratio. I am guessing that it will be about 100 Meg. I put a link in a post to where it will be stored. A friend of mine has suggested a few places where I could place a big file to share.

The file will take over 4 hours to compress so I'll try to post earlier part of this week.

In my post through the thread I have what I have figured out so far. So you have that to start with.

I posted the first 1.2 Gig of the hard drive to 242Meg zip file. This is the first time I used We Transfer. It is a neat service I hope they can keep it running. Anyway, the file will be deleted on September 13. Let me know if you need it posted after that date.

I believe the link from the title file TI000_0TI000_00.IFO, is in the TM001_0TM001_00.MAP file. I just can figure out the MAP file at all. Some with some knowledge of mpg file might be able to see what it is but I can't.

We Transfer does use Adobe Flash Player. So you will need a browser with that loaded in it to have this work.

Web browsers are web browsers, not downloaders. I only download with downloaders, so that all characteristics of the original that can be preserved are preserved, not the least of which is the timestamp indicating source's "creation" date instead of the download date, and so that recovery from drop point is easy and no prior bytes downloaded are wasted when a download connection breaks.

I keep 6 web browsers open most of the time. Only one is allowed Flash, and tabs with Flash in them that don't behave politely, which is most Flash using pages, get closed summarily.

Every HDD I throw at the Funai are compatible as long as the "Delete Volume>Create New Volume" procedure is used with a 512 Allocation unit size.
The three file formats; NTFS, exFAT, and Fat32 are also compatible.

With Fat32, the prog Fat32Formatter was used with Delete partition.

***None of the HDDs have Advanced format technology, so I'm not sure if they will work.
________________________________

The Bad,
The Funai does not "format" the drives, the existing file system is used with what appears to be a small DOS OS installed on the front end, so only the Funai can see/read the content on the HDD.

After "formatting" the drives in the Funai, then connecting them back to a Windows PC, the original file system and volume name are retained, and the only visible content is an empty recycle bin.

In Ubuntu, when a drive can be seen/mounted, different commands(from my very limited understanding) would result in a DOS warning and Permission Denied.

I have been blanking the hard drive will all 0s before starting so there wouldn't be any extra bits turn on that the DVR didn't do. The hard drive format they are using is their own and doesn't have a partition table or boot sector or any computer general file system on it. The drive is just for storing data.

I haven't pulled together the information into one spot but several of my post have information that I have decoded.

I'll try to download the data you posted mrmazda and see what I find. I have most of the file uses figured out but not the bits in them.

I think the TM001_0TM001_00.MAP and TM002_0TM002_00.MAP have time indexes on them. Most of the data looks like this because it just increments going down a list. I haven't been able to figure out what the first 16 bytes means at all. I think that is where the secret is for mapping from the directory listing in the TI000_0TI000_00.IFO through the TM001_0TM001_00.MAP to the AV001_0AV001_00.MPG that stores the video.