Ok, first try at auto-marking bad MBs based on ffmpeg log output and primitive analysis.

...

Edit: Here's the Java source if anyone else wants to play. http://pastebin.com/bKgzCRqD The mammoth inner class at the bottom is for writing the animated gif

Great stuff!!

One suggestion: when comparing a block with other block its best to only compare it with the one above and the one to the left.

This is because of how mpeg-4 works: the current block will directly affect the one below and the one to the right. So you are really comparing to yourself in a way. Also: if the current block is defect (or one until the one below is reached) it is very likely the block below is bit-shifted so you're not comparing the current block to what is supposed to be below it.

Keep that in mind.

Regards,

arnezami

Hmm ok so it is pointless to try manipulation in blocks past the first bad block? Basically we need to completely fix one bad block at a time?

I think you're okay fixing half-block errors that occur later in the image, but don't mess with full block or greater brightness/hue changes... they come and go freely as you fix earlier blocks. At least that's what I'm noticing on Frame 9... I'm having to revert some of my earlier edits.

Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe. Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

About the problem where the shift occurred: Frames are framed by large blocks of FFs with 47 1F FF 1X in them if they are large enough and X counting up. On Both sides of that IF7 they are already align. So the shift and reshift should have occurred between those. Maybe it is just enough to add a F at the end of the block before IF7 and remove one at the start of the block after IF7 (0x23cde9).

This detail may help suss out what the packet headers should be in the bit-shifted packet. For instance, we know that the PID field can't take certain values like 0x0004 through 0x000F, etc.

Lining up the null packets in front of the bit-shifted one in a 47-column hex editor it's easy to see bad bytes, because we know that the 184 bytes following a 471FFF header should all be FF, such as this from raw.ts:

This packet-aligned-column view also allows you to see where the good packets pick up after this mangling - at 0x23BFDC with a 0x4703E81A, followed by 0x4703E81B. This also tells us that we have the right number of bytes in the scrambled interim, because the columns still line up.

Moving up the rows, it's easy to see where there's corrupted headers:

0023BC30 C0D86101 should be 4703E815, or something with a bit set to indicate a sequence discontinuity.0023BCEC BE66AF8C should be 4703E8160023BDA8 4703E817 is correct0023BE64 77034818 should be 4703E8180023BF20 77EB35EE should be 4703E819

So it seems like it should be practical to write a tool that can walk through the raw.ts and use the rules of the packet header formatting and sequence numbers to reconstruct a more coherent transport stream. Even just getting all the errors out of the null packets by replacing everything with FF might make it easier to discern good patterns.

Edit: For example, in raw.ts there are bytes missing or added between 0x37ACC and 0x371EC - that's the first place where the columns shift in a 47-column hex display.

"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

iframe 6 (try1)Left leg is not fully deployed yet, but i'm perplexed by the tip of it. At least I think it's the tip because it doesn't look like a cloud or a foam spot on the sea. It kind of resembles the tip of the right leg on iframe 5, which shows the right leg in similar stage of deployment as left leg on this frame. The thing is it doesn't align. It's off by one block (half of macroblock). Is such an alignment error even possible?

Some macroblocks are highly suspect, they may be random noise that vaguely looks like information that I forced into the picture (18:12 and 25:12 are the main concerns).

Brace for mmbX:82804:80,X:134444:80,0:0:583,39:0:5496,43:0:-1,2:1:6129,21:1:8584,22:1:8796,6:3:18105,7:3:18276,9:3:29527,30:3:32359,31:3:-1,43:3:33591,2:6:44738,4:6:46341,5:6:48794,35:6:52255,38:6:52740,40:06:52938,42:06:53105,0:7:53564,2:7:-1,4:10:55390,1:11:-1,10:11:63669,27:11:-1,2:12:67622,17:12:70590,18:12:71241,19:12:71659,21:12:71834,23:12:-1,25:12:72549,26:12:72948,27:12:73733,29:12:75341,33:12:-1,1:13:76584,2:13:-1,05:13:77037,30:13:83673,33:13:84191,37:13:84690,14:14:87836,17:14:-1,19:14:88523,30:14:92989,8:15:95815,12:15:96507,15:15:96729,22:16:109491,25:18:131296,42:18:135673,05:22:-1,27:22:173820,0:24:-1,27:24:192142,4:27:-1,25:27:215436

If you zoom in on 6, it seems to me that 17:12 is sea foam, not the leg. The color and texture isn't right. (It'd be handy to have a magnifying glass on Slapbet for stuff like this.)

Then, I suspect 15:11 is actually 15:10. The bit offsets in rows 10 and 11 jump all over the place; there's a jump from 56129 to 63669 between 09:10 and 10:10, and by 00:11 we're at 68940. The actual end of row 10 to 11 probably lives/lived early in that range; or earlier.

If you zoom in on 6, it seems to me that 17:12 is sea foam, not the leg. The color and texture isn't right. (It'd be handy to have a magnifying glass on Slapbet for stuff like this.)

This was my first assumption too, but there is no good place I can put it other that the top of the leg. It's very bright and causes a propagating streak if placed in other places. The range it can be in is constrained and it's in a longish piece of good data so it can't be moved much without either impinging on fixed pieces or raising the question where is the tip of the leg then. The leg section on the other hand is really dark and wants a very light macroblock on top of it.

Can't be. 4:10 to 42:10 is a solid good piece of data anchored by the smudge on 26:10

Good point.

Hmm. If you switch back and forth quickly between 'common_dirt2' (frame 8 ) and 6, 6's left leg is too long, not too short. And the right leg is a little short. So maybe 15:11 is more like 16:12? Then everything else shifts down and to the left one. The lower part of the leg isn't set in stone, I don't think. Or 15:12/13? Not sure, though. Wish I was more nimble with the MPEG format.

Note, this is from try1.ts. It's fairly easy to port between coerced.ts and try1.ts frames though. Just need to add/subtract an offset to account for the extra sections of data in try1.ts that aren't in coerced.ts.

Note, this is from try1.ts. It's fairly easy to port between coerced.ts and try1.ts frames though. Just need to add/subtract an offset to account for the extra sections of data in try1.ts that aren't in coerced.ts.

I changed 11:1:5384 to 22:1:5384 to properly align the dark dirt in top center... I really should give them numbers And also 30:2:-1 to 3:0:-1Now a lot of dirt blobs start to align... Now I really gonna number them..

<snip>The following is how it should look like IMHO.There are 3 blocks of XXs which are different for each frame. The first one seems to contain a counter. The rest i dont know. But it should be mentioned in that ISO*.pdf how they should look like/what they should contain.

The next block of XXs is part of the Presentation Time Stamp PTShow to get the PTS is explained quite well here: http://dvd.sourceforge.net/dvdinfo/pes-hdr.htmlPTS: 21 01 xx xx xx. We can now decode the PTS of the good frames and define the PTS of the messed up ones.

And the right leg is a little short. So maybe 15:11 is more like 16:12? Then everything else shifts down and to the left one. The lower part of the leg isn't set in stone, I don't think. Or 15:12/13? Not sure, though. Wish I was more nimble with the MPEG format.

I can see no way that this thing can be in any other row than the one it's currently in. Leg macroblocks look in correct and unchangeable positions to me. The only place it could go is to the right. It may be possible to calculate an approximate position based on known bitpositions and average macroblock size. But try1 seems to contain large chunks of junk so that approach is questionable.

Here's what it would look like if it was shifted 1 block to the right. Pretty weird, but kind of fits. Plus this change would lighten the columns under it and darken the ones to the left. And the leg tips look pretty weird on iframe 5 as well. But again, I do not know if that such a misalignment of blocks in macroblocks is even theoretically possible.

Here's what it would look like if it was shifted 1 block to the right. Pretty weird, but kind of fits. Plus this change would lighten the columns under it and darken the ones to the left. And the leg tips look pretty weird on iframe 5 as well. But again, I do not know if that such a misalignment of blocks in macroblocks is even theoretically possible.

I do like the fit of that image, as you can make out the curve of the tip of the leg. Perhaps the bright white is another camera window artifact, like the missing part of the left side of the left leg - maybe it just caught the white border in just the right way in the light, and you wound up with a magnified or brightened reflection. There's a similar sort of oddity at the tip of the leg in the current rendition of frame 5 as you observed - perhaps nailing that down more fully would help figure this one out.

« Last Edit: 05/11/2014 01:05 AM by mvpel »

Logged

"Ugly programs are like ugly suspension bridges: they're much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code." - Eric S. Raymond

I changed 11:1:5384 to 22:1:5384 to properly align the dark dirt in top center... I really should give them numbers And also 30:2:-1 to 3:0:-1Now a lot of dirt blobs start to align... Now I really gonna number them..

Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe. Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

Yeah, been working on the raw and the corruption is much worse than I had assumed. Not only are there huge swaths of garbage, but also probable missing bits (like the 4-bit offset that was found for sure).

Now I see where the aeroquartet guy was coming from - it *should* be impossible to repair this video - but all the more reason why the work being done here is so amazing. At the current pace, we'll have silky smooth HD video before the next launch

I'm going to continue playing with michaelni's tsfix, but I don't see a way that this can be brute-force cleaned. Missing (and possibly added) bits expand the search space well beyond computational feasibility. We might be able to tidy up some of the intermediate frames some, but the masters will have to be done by hand. So keep up the awesome work bit flippers and block movers!

One note to the maintainer of slapbet - it would probably be very easy to integrate my ffmpeg log parsing code into the webapp to enable bad-block highlighting with a checkbox. See ~499 of http://pastebin.com/bKgzCRqD and following - just string and integer parsing in Java which is a trivial port to js.

Working on those headers I found that some errors are single bit flips, but there are also stripes of bytes which are simply garbage/randomly turned bits one after an other. In an image you cant repair this I think. Substitute MBs maybe. Headers can also be fixed, but if part of an frame MB itself is messed up like this you cant really hope to fixed it by flipping random bits.

Yeah, been working on the raw and the corruption is much worse than I had assumed. Not only are there huge swaths of garbage, but also probable missing bits (like the 4-bit offset that was found for sure).

Now I see where the aeroquartet guy was coming from - it *should* be impossible to repair this video - but all the more reason why the work being done here is so amazing. At the current pace, we'll have silky smooth HD video before the next launch

I'm going to continue playing with michaelni's tsfix, but I don't see a way that this can be brute-force cleaned. Missing (and possibly added) bits expand the search space well beyond computational feasibility. We might be able to tidy up some of the intermediate frames some, but the masters will have to be done by hand. So keep up the awesome work bit flippers and block movers!

One note to the maintainer of slapbet - it would probably be very easy to integrate my ffmpeg log parsing code into the webapp to enable bad-block highlighting with a checkbox. See ~499 of http://pastebin.com/bKgzCRqD and following - just string and integer parsing in Java which is a trivial port to js.

Edit: also, a mouseover showing the x:y would be nice

This is all incremental. Anything you get us thats better than we have right now is progress!

iframe 6 (try1)Left leg is not fully deployed yet, but i'm perplexed by the tip of it. At least I think it's the tip because it doesn't look like a cloud or a foam spot on the sea. It kind of resembles the tip of the right leg on iframe 5, which shows the right leg in similar stage of deployment as left leg on this frame. The thing is it doesn't align. It's off by one block (half of macroblock). Is such an alignment error even possible?

If I understand your question correctly: yes.

It is possible the data from one block (not macroblock) is used by another block within the same macroblock. This would for example happen if somehow a bad bit was interpreted as a "stop-bit" of the data of a block. This way the next block (within the same macroblock) would use the data from its previous block. The other way around is also possible I believe.

This is all due to the variable length the VLC-codes can have. And they are the ones that have the "stop-bit". See here for more info on 8x8 block decoding.