As you probably have guessed from the latest developments (QEMU (http://www.magiclantern.fm/forum/index.php?topic=2864.125), EDMAC graphs (http://www.magiclantern.fm/forum/index.php?topic=18315.0), JPCORE (http://www.magiclantern.fm/forum/index.php?topic=18443.0), EEKO (http://www.magiclantern.fm/forum/index.php?topic=13408.msg175656#msg175656)), our understanding on how LiveView works has improved considerably. Finally, all my fiddling with QEMU, at first sight with little or no purpose for the everyday users (http://www.magiclantern.fm/forum/index.php?topic=2864.msg169970#msg169970), starts paying off.

Today, Magic Lantern proudly announces new ground-breaking features that were previously thought impossible or very hard to achieve.

The last feature complements the well-known full-resolution silent pictures (http://www.magiclantern.fm/forum/index.php?topic=12523.0) - the new implementation will be usable at fast shutter speeds, without the exposure gradient (http://www.magiclantern.fm/forum/index.php?topic=12523.msg120750;topicseen#msg120750) - but with rolling shutter (just like regular LiveView frames).

*)Continuous recording for the above resolutions can be achieved as long as you can get a LJ92 (http://www.magiclantern.fm/forum/index.php?topic=18443.msg182074#msg182074) compression ratio (compressed / 14-bit uncompressed) of about 50-55%, with preview set to Frozen LV (previously known as Hacked Preview) for an additional speed boost. Otherwise, you'll have to reduce the resolution or the frame rate.

The following table shows how compression rate changes with ISO and bit depth; please check the figures for your particular scene in the raw video submenu, as they can vary a lot, depending on the scene content.

This is only a very rough proof of concept. It has not been battle-tested and has many quirks. Some of them may be easy to fix, others not so. In particular:

* It feels quite buggy. I'm still hunting the issues one by one, but it's hard, as Canon's LiveView implementation is very complex, and our understanding on how it works is still very limited.* Write speeds are high. For example, 10-bit 4096x2500 at 15 fps requires 180 MB/s. 1080p45 should be a little more manageable at 111 MB/s.* Canon preview is broken in most modes; you need to use the grayscale preview in the raw recording module.* High-resolution modes (in particular, full-res LiveView) may cause trouble with memory management. This is very tricky to solve, as we only get 3 full-resolution buffers in LiveView, with restrictions on the order in which they must be freed, and lots of other quirks.* Since these settings were pushed to limit, the risk of corrupted frames is high. If it happens, decrease the vertical resolution a bit (from the crop_rec submenu).* When refreshing LiveView settings, the camera might lock-up (no idea why). Pressing MENU twice appears to fix it.

May I fine-tune the new modes?

Yes! I've included some of the knobs on the user interface. Normally you shouldn't need to touch these buttons, but if you do, you might be able to squeeze a few more pixels.

Does it work with FPS override?

Sort of. It's not reliable at this point, so it's best not to try yet.

Overheating?

During my tests, I didn't manage to get a sensor temperature higher than 60 degrees. Your mileage may vary.

Risks?

This mod changes some low-level sensor parameters that are not well understood. They were all figured by trial and error, and there are no guarantees about the safety of these changes.

As usual, if it breaks, it's your fault, sorry.

Will it work on other camera models?

I hope so; however, this is an area where I hope to get contributions from others (yes, from you). If these new features don't motivate you to look into it, I wonder what else will.

I'll explain how all this works in the coming days or weeks.

Is it difficult to port to other camera models?

So far, the 3x3 720p mode from crop_rec was ported to EOS M (rbrune), 700D (dfort) and 100D (nikfreak). So it shouldn't be that hard...

Will you port this to my camera model, please?

No, sorry. I have better things to do - such as, preparing the April 1st prank for next year :)

Wait a minute, didn't you say you are primarily a still photo user? Why are you even doing this?

If you look close, the usefulness for video is fairly limited, as the write speeds (and therefore the recording times) are not practical.

But the full-resolution LiveView is - in my opinion - very useful for still photo users. Although the current implementation is not very polished (it's just a proof of concept), I hope you'll like the idea of a 7.4 FPS burst mode, 100% silent, without shutter actuations.

Right now, you can take the mlv_lite module with pre-recording and half-shutter trigger: at 10 bits per pixel, you get 5 frames pre-recorded, and saved to card as soon as you touch the half-shutter button. Or, you can capture one frame for each half-shutter press, with negative shutter lag! (since the captured frame will always be pre-recorded).

And if a burst at 7.4 fps is not enough, you may also look at the 4K modes (12-15 fps).

(I know, I know, GH4 already does this, at much higher frame rates...)

The raw recording modules have a couple of alignment constraints (e.g. can only start cropping from a multiple of 8 pixels, and the size of the cropped area (that goes into the MLV file) must be multiple of 16 bytes (that is, W*bpp/8 + H mod 16 must be 0).

To capture the full resolution, you may use the silent picture module. However, this module is not the best when it comes to memory management and buffering. Currently, you'll get an impressive buffer of 2 frames in burst mode :)

But hey - it outputs lossless DNG!

What about that lossless compression routine?

It's included, although I didn't manage to test it much. There is a lot of room for improvement, but for a proof of concept, it seems to work.

With our latest achievements in wizardry with ARM programming and DIGIC reverse engineering, we can speak of a new era of raw video recording.

On models like the 5D Mark III, the next upcoming releases will feature an improved version of our crop_rec module (http://www.magiclantern.fm/forum/index.php?topic=17021.0) that delivers the following new resolutions:

The last feature complements the well-known full-resolution silent pictures (http://www.magiclantern.fm/forum/index.php?topic=12523.0) - the new implementation will be usable at fast shutter speeds, without the exposure gradient (http://www.magiclantern.fm/forum/index.php?topic=12523.msg120750;topicseen#msg120750) - but with rolling shutter (just like regular LiveView frames).

Please understand that providing the source code for those highly DIGIC optimized routines is a bit troublesome and will need some extra legal care. After this step is taken and as soon we are finished with ensuring the product quality you are used from Magic Lantern, we will upload the code to our repository.

Consider this being a huge leap towards our next mind boggling goal:

8K RAW Video Recording!

(http://a1ex.magiclantern.fm/2017Apr01/fishy_8k_april_fools.png)Sample DNG from 5D Mark III, to show that our proof of concept is working:

Only the simplest crop mode (1920x1080) has good real-time preview. The modes with higher vertical resolutions have color preview too, but it's not centered (only the top of the frame is shown).

In the regular crop_rec branch, there's also centered 5x zoom. Here I've removed it temporarily, as it increased the code complexity quite a bit, but I'll probably add it later when things will settle.

I read all this and thought Great Joke ! :P but then I read the Source Code Holy Cow :o :o :o :oThere goes next few days , I'll be living in front of my computer exploring this and hopefully I can compile for 5d2 :D

reddeercity do you think that's possible yet? Wouldn't it require tons more reverse engineering? Damn it I just happen to be away from proper computers that can compile ML for a week right now so I won't be able to try it if it is possible :'(

Will not sure yet but I'm looking in to it . If you read this post Full-resolution silent pictures (http://www.magiclantern.fm/forum/index.php?topic=12523.msg120497#msg120497) there's info about 5D2 and lead me to believe it maybe possible 8)I'm currently try to compile the code now , but I get some error's .some tweak.o stuff , I working of older code , so I think that could be the problem so I'll update to the latest source code and try again . I don't think 4k is a reality on 5d2 but maybe 3K as the bandwidth for 4K is around 150MB from what I read.Since we are limited to max. 80MB/s and by your Yet Another RAW Video Calculator (http://rawcalculator.bitballoon.com/calculator_desktop) at 10bit 3x crop 2496x1134 @ 2:2.1 A.R. is 80MB/s unless Lossless compression is used and that's another thing by it's self .

Greg's original proof of concept (full-width LiveView (http://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909)) was done on 500D (same generation as 5D2 and 50D).

Sure, it probably needs a bunch of reverse engineering, but it probably boils down to changing some ADTG registers and video timers.

I've actually got these presets working in adtg_gui first. A proof of concept for 3K was committed back in 2016 (https://bitbucket.org/hudson/magic-lantern/commits/50d8f06018) (also mentioned (http://www.magiclantern.fm/forum/index.php?topic=17021.msg171809#msg171809) in the "classic" crop_rec thread).

So far, the 3x3 720p mode from crop_rec was ported to EOS M (rbrune) and 700D (dfort). So it shouldn't be that hard...

I'll take that as a compliment! :D :D :D

That reminds me, I should do a pull request for the 700D but it looks like there's more work to do now. How about getting some other cameras up to speed on the crop_rec module? By the way, don't expect 4k on anything other than the 5D3--then again maybe there will be more surprises.

Yep - my point was that people outside the core team are definitely able to play with this code (as in, it's not pure black magic).

Took this build out for a test (nothing fancy, just taking pictures of the kids with full-res LiveView and pre-recording). Noticed a major limitation (could not set shutter speeds fast enough for daylight) and pushed a fix for that.

After the fix, modes with reduced FPS (4K and full-res LiveView) now have the usual range of 1/4000...1/30 (or 1/60) mapped to 1/15000 to 1/fps. That's right - 1/15000 exposure time with 128ms rolling shutter :D

Next annoyance was the grayscale preview, which is slow and not working at all in 10-bit. Also, lossless MLVs are not playable in mlv_play. Will look into those.

Yep - my point was that people outside the core team are definitely able to play with this code (as in, it's not pure black magic).

My point too. I'd like to get some more people to start playing with the code. There's lots of information in the comments and on the forum that takes the mystery out of what's going on--though some of what you're doing sure seems like pure black magic.

@Greg and the rather slow 5D2: 21.2MPx x 3.9fps = 82.68MPx/s. Does this basically tell us the limit of the sensor? So this calculation works: 82.68/23.976 = 3.45MPx limit at 24p on this camera? So I guess it could do 2560x1320(3.38MPx). :D how does one implement the "Greg's resolution hack" that I've been hearing about?

Updated the preview with something a little more usable. It's color (though still low-res and non-realtime) and only drops to grayscale when recording speed becomes critical.

Unfortunately, g3gg0's raw_twk (https://www.magiclantern.fm/forum/index.php?topic=16682.0) routine doesn't appear to work in LiveView, so we are stuck with (slow) CPU-based previews for now :(

I can probably get faster previews in 14-bit lossless mode, by configuring the source raw stream as 16-bit (unpacked), since the compression routine accepts 10/12/14/16-bit input (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620). For uncompressed modes, I'm afraid we currently don't have a way to get both unpacked data (for fast previews) and bit-packed (for recording) while in LiveView (so all my previous ideas (http://www.magiclantern.fm/forum/index.php?topic=5601.msg177531;topicseen#msg177531) are currently fiction).

Trying to see what the next step is to see if there are any modes 700d can get, first i messed with atdg tool to try to reproduce the 3x3bin mode while in 720p by changing 0x4 to 0x2 on 800c just to understand how the tool works since those values are posted. I then set the tool to show values that have been changed more then twice and screen captured every value for 1080p, 1080p x5 and x10, 720p, 720p x5 and x10, crop mode in 1080p and also photo mode just to track all the values that are changing in case it was needed.

Thats where i am at now though, not sure what values to change though, gets scary when you see the crazy images on live view that make u think u burned sensor out. Only thing i saw that did anything as far as the size was 800c changing to 0x0 makes a zoomed version but its stretched tall. Other then that another one moved the image up and down.

Dont know how to cross reference with the 5d3 cause i see Pack12(?,?) for example and the values are not like what you see in the tool. So pretty much clueless on where to go from here??

Wow! That's a hell of a lot more useable and the preview is centered which is absolutely kickass.

I get 30 frames in UHD, but 3K is still continuous for me in crop_rec. The preview does drop back to greyscale about 50% of the time in 3K. The colour preview is more detailed and has a higher refresh rate... it's pretty much usable, I think.

I'm trying it out but unfortunately when I try to watch it on MlRawViewer or export it to DNGs using RAWMagic I'm left with this: https://drive.google.com/open?id=0B1lko58z3g6lRTlKUWc1YzlzOFk. I tried re-downloading the build and formatting my card but I can't seem to get anything out of any of these crop modes...

I've found a couple of issues in both builds. The first is nothing major: when you change a resolution in the crop mode menu, say 3K to UHD, you have to exit the ML menu before you can adjust the resolution size in the raw video menu. The other issue is making me pull out what little hair I have left: the lossless compression ratio appears to have a mind of its own, it's like a recalcitrant AI. At one point it was compressing at 51% on the 1st build:

(http://i.imgur.com/6IX4XKI.jpg)

When I loaded the 2nd build it started at 85%:

(http://i.imgur.com/HB2uNHV.jpg)

So I had to scale down to:

(http://i.imgur.com/bXY36uK.jpg)

After a few seconds of recording the compression went to 70%:

(http://i.imgur.com/Ojd4ykL.jpg)

Then when I try to increase the resolution I get this [Expect xxxx frames]:

(http://i.imgur.com/NckArF4.jpg)

A battery pull gets rid of the [Expect xxxx frames], but on the 2nd build it reappears when I try to scale up the resolution. So I went back to the 1st build to see if I could record at 3.3K 51% lossless again. Initially I was able to get this setting:

(http://i.imgur.com/Mj734Xe.jpg)

That was a bit much for the card, it stopped recording after a few seconds resulting in the dreaded [Expect xxxx frames]:

(http://i.imgur.com/kR4G8cb.jpg)

Taking me back to square one where I had to scale the resolution back down:

(http://i.imgur.com/gNnZrKY.jpg)

Ideally the compression ratio would either be fixed, or manually adjustable.

Ideally the compression ratio would either be fixed, or manually adjustable.

That's mathematically impossible, and not how lossless compression (https://en.wikipedia.org/wiki/Lossless_compression) works. The compression ratio you get is totally dependent on the scene, and it's even theoretically possible that the compression results in larger file sizes than the original uncompressed data. This typically happens when you give a lossless compression algorithm data that has very little redundancy (already compressed), or that is not of a type the algorithm is "tuned" for, for example feeding the compressor 10-bit raw data results in the compressed version actually being larger (http://www.magiclantern.fm/forum/index.php?topic=18443.msg176699#msg176699).

For the most part, lossless compression algorithms work by simply eliminating redundant data via more efficient encoding. Take some text for example, encoded using ASCII, each character takes 1 byte to store and therefore all characters have equal weight even ones that are rarely used. However we know that in any particular language there are characters (or even sequences of characters) that typically appear more often than others, so if we gave the characters that appear more frequently, shorter encodings (which subsequently requires less common characters to have longer encodings), then overall we can save space. However if we were to feed random gibberish to such an algorithm, it might actually take more total space the store than the original simple encoding, since the characters with longer encodings are just as likely to appear as ones with shorter encodings.

Understood, but from a cinematography perspective; I need to figure out what resolution will give the card enough headroom to write at that resolution continuously at least 90-95% of the time; so far it looks like that's somewhere in the 2.5-2.6K range for 24p 2.39:1, which is still great.

2.7-2.8K 2.39:1 looks like the sweet spot for repeatable continuous recording on the Toshiba 1066x card. By "repeatable" I mean repeatable near 100% of the time. Same goes for 3.3K anamorphic scope: 2272x1364 with a 1.5x anamorphic lens.

I salute the dedication but I wish squeezing every last pixel of out of the camera would be secondary to allowing audio with simple RAW. I would trade every last improvement for 1080p RAW with audio. For some reason, this worked for me once, but it just randomly stopped. It used to create a WAV file along with the RAW file.

I need to figure out what resolution will give the card enough headroom to write at that resolution continuously at least 90-95% of the time

Latest update estimates compression ratio on the fly, from the current scene and exposure settings. Also a few minor bug-fixes.

To get the best possible compression, all you need is to set ISO to maximum, shutter to 1/FPS, aperture wide-open and use a bright scene. You'll get close to 35% (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620) (apparently the limit for this codec, or maybe just for Canon's implementation).

Generally, if pixel values are predictable (from their neighbors), the image will compress well. That usually means low noise levels (low ISO, shadows, fully clipped highlights) and few details (large out-of-focus and/or low-contrast areas). Note that absolute noise values are higher in highlights, and the SNR is higher too, so exposing to the right will give higher file sizes (but that shouldn't stop you from using it).

2 / in the same way of the 14 bit losless mjpeg compression is an 8 bit mjpeg compression possible? so we can do low file size and get the real sharpness the 5d3 can offer (clean HDMI recorder on prores does not get any better sharpness than the in camera h264 file - whereas the raw files is always resolving much more sharpness)

3/ finally how far would you say you are from a working preview for the mode 3K, UHD, or zoom x5 and is it even possible?

Also you probably have thought this but the basic full sensor preview with blacktape on the sceen could make it and if the 1:1 sensor readout cannot show original preview then maybe a x3 preview instead of the x5. Would not be perfect but would definitly make it usable.

I started testing 14 bit lossless with 1080P 24 because thats what I use the most. File sizes are much smaller, thats nice. Something funky is happening though. Using the histogram overlay with 80% or more of my exposure on the right half I'm getting horribly, horribly underexposed results in davinci. The live view looks great, histogram looks great, but then it hits davinici and the footage looks awful. Not sure what to make of it, but thought you guys might want to know.

I was able to shoot up to 19 seconds at 3504x1312 at regular crop with Framed preview and Compressed Raw 14 bi, lots of movement, I was panning slowly downwards. But I only got so much recorded this one time, I never achieved the same length of recording again.

Is there a way of centering the crop? It looks weird not being centered.

1. If you half press shutter button while recording the preview stays in colour rather than alternating between colour and greyscale. Frame rate still lags but it makes things a hell of a lot easier to know what's going on in front of the camera.

2. Filming the moire pattern of a computer screen's pixel array and refresh rate will absolutely kill the compression and stop the recording, even at lower resolutions!

Is there a way of centering the crop? It looks weird not being centered.

It should already be centered. Are you loading the crop_rec module of the latest experimental build and then selecting 3K or UHD? Framing is off in 5x but should be centred in these modes. Sometimes 3x is shifted up for me. I just restart the camera and it's fine.

Yeh I figured it out. I guess I was blind while looking at the modules or I wrote the files to the wrong sd card from my lexar workflow hub.

Getting pink frames and crashes in the 50fps modes.

Getting unreadable files in some other modes like the 3k and UHD. It was just an initial test, I will test more tomorrow. I am using 1066x 256gig lexar cards. However, they aren't formatted and have a lot of shots on them. I will format one and see what difference that makes.

1080p 48FPS seems to be working (It seemed to have recorded fine, but I must have deleted the file.)3520x1320 24FPS(ISO 100-200 iso seems to be continuous and at a 5x crop mode) ISO 400 recording stops after a few seconds. In no crop mode preview was colorful noise. in 5x crop preview worked great. I recorded 1510 frames and I ended it.3584x1320 24FPS I recorded 1443 frames and I ended it.1080p 60FPS(buggy, stops recording quickly) gray preview was glitchy and lagged.

3k ([email protected]) I got 66 frames.UHD ([email protected], couldnt get 3840x1600) I got 66 frames as well. It states its 120mb/sec, this might require the lossless compression with 10bit?Both 3k and UHD frames are good, used MLV_DUMP from the page I downloaded the firmware to extract these.

Ah, a little late to yet another party once again and after playing around in the past 48 hours or so I've finally come to a conclusion to be able to confirm that 3584x1320 in crop_rec (3K 1:1) can be recorded continuously if shot in between ISO 100-200 and more often than so in 400 depending on scene.

This was all tested with both 128gb 1066x & 256gb 1066x CF cards from KomputerBay on two separate 5D3's running 2017April03.5D3123 builds.

Seems that ISO 800 and higher doesn't get along well with 14-bit lossless compression which makes sense as previously stated by others in here.

Here's an example of what it looks like upon running this beautiful hack via LiveView from 5D3 below and watch what happens when I push it up to ISO 800 (and higher) towards the end:

I was able to convert all 14-bit lossless files with ease from @Danne's latest cr2hdr.app_4k (includes the required certain mlv_dump which was modified directly from @a1ex) so I am more than flabbergasted by the fact that all of this is storming down on us like there's no tomorrow.

Love it and will continue to play with and hopefully run into some more bugs!

Also I can confirm that if I point the camera at a computer screen while recording and it'll crash once I focus upon the pixealated monitor (indeed this is quite strange but good find @hyalinejim) and I am actually loving the half shutter press for FULL COLOR LIVE DISPLAY or whatever that's called (plus it zooms in a bit more to be sure everything looks spot on) before recording which is very handy.

Thank you @a1ex for yet another petrifying update (in a good way!) and don't ever change who you are! Ha. 8)

I cant seem to get that lovely liveview on 113. Mine is low fps, desaturated and choppy.Running crop_rec_4k.2017.Apr03.5D3113 and I copied your settings..

Hmmm then perhaps I should downgrade to 113 and try to reproduce this?

Did you press 5x zoom after using ML menu or before? Sometimes it goes into a weird phase which may seem like having its own quirks but a simple restart (on/off) of the camera would help at least it does oh my end.

But it could very well be a bug for 113 and I'll let you know once I've downgraded.

Thought not too long ago @a1ex made a pull request for both firmwares (113 & 123) to be merged together after each update or does this only apply to the Nightlies and not experimental?

Well I'm actually tempted to check out the good old speedy 113... Just been using 123 as of late for HDMI output on gig's.

Actually this one trick may be worth to try (Thanks @dfort for the tips) perhaps reset your settings to default (from Canon menu) not the ML version and let me know if that helps either way.

Testing latest build (April 3th crop_rec) on 5D3 113, for now only this mode because is which I find more useful for me: 1920x872 50p 3x3 (aspect ratio 2.20:1) 14bit lossless.

Yesterday I was able to record without problems in continuous with normal preview, in color.

Today, with the same settings as yesterday, it has been impossible to boot the camera. In the SD card there are text files called ASERT, one for each boot attempt. In them the following appears:ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_tasklv:1 mode:3Magic Lantern version : crop_rec_4k.2017Apr03.5D3113Mercurial changeset : 7b0cb9532629 (crop_rec_4k) tipBuilt on 2017-04-03 14:54:32 UTC by [email protected]Free Memory : 173K + 3807K

If I change SD card for another with 2017Feb11th 5D3113 (crop_rec proxy) all works again.

Testing latest build (April 3th crop_rec) on 5D3 113, for now only this mode because is which I find more useful for me: 1920x872 50p 3x3 (aspect ratio 2.20:1) 14bit lossless.

Yesterday I was able to record without problems in continuous with normal preview, in color.

Today, with the same settings as yesterday, it has been impossible to boot the camera. In the SD card there are text files called ASERT, one for each boot attempt. In them the following appears:ML ASSERT:0at mlv_lite.c:1935 (compress_task), task compress_tasklv:1 mode:3Magic Lantern version : crop_rec_4k.2017Apr03.5D3113Mercurial changeset : 7b0cb9532629 (crop_rec_4k) tipBuilt on 2017-04-03 14:54:32 UTC by [email protected]Free Memory : 173K + 3807K

If I change SD card for another with 2017Feb11th 5D3113 (crop_rec proxy) all works again.

So what is the bottle neck for transfer speeds in the 5D3? Is it still the card reader in the camera? I thought cameras card reader was limited even with really fast cards?Is spanning the trick for UHD?

Took a quick look - after the latest fixes (see below), the following resolutions are continuous (1.2.3, lossless ~55%, Preview: Frozen LV, with grayscale preview):

- 1920x800 60fps- 1920x960 50fps- 1920x1080 45fps

1920x1080 48 FPS is a bit unreliable (corrupted frames with Global Draw on, but appears OK with it off). This mode is really pushed to the limit (and that's the reason I've also added a 45-fps option). Feel free to play with the knobs from crop_rec submenu, if you think you can find the sweet spot without corrupted frames.

I would expect the 1.1.3 firmware to have a little more headroom (maybe even allow a few more pixels), but didn't test it. You can fine-tune vertical resolution from the crop_rec submenu (and if you have corrupted frames, decrease it from there).

Fast half-shutter trigger (with negative shutter lag):- crop_rec.mo, mlv_lite.mo, configured as above- raw recording: enable REC trigger (half-shutter: pre only; without pre-recording enabled, that means only 1 frame)- no need to use FPS override, as we are only saving the frames triggered by half-shutter- start recording- press half-shutter every now and then to capture a frame- stop recording

Capture unpredictable action (animals, kids etc):- crop_rec.mo, mlv_lite.mo- crop mode: full-res LV or 4K half-FPS- raw recording: max res, 10-bit (here, pre-recording is a bit confused by variable frame sizes, so let's not be adventurous)- pre-recording: 1 or 2 seconds (actually "up to")- half-shutter trigger: "hold" (which will pre-buffer up to about half of available RAM), or "pre only" (which will pre-buffer all the available RAM)- start recording, press half-shutter etc- caveat: preview is not working at all with 10-bit (try 14-bit)- to review in camera: 4K 10-bit can be played back with mlv_play + raw_twk, full-res 14-bit with mlv_play only.

so we can do low file size and get the real sharpness the 5d3 can offer (clean HDMI recorder on prores does not get any better sharpness than the in camera h264 file - whereas the raw files is always resolving much more sharpness)

Hello thanx for taking the time to answering the question this is highly appreciated. i'm on the curve to go to URSA 4.6K but this new fonctions let me think my olds 5d3's have still some life to live.

Are you absolutely sure regarding the sharpness. I'm asking because this is a fact that the clean hdmi as absolutely zero additionnal sharpness from the h264 files i have tested this thousands times. But the raw definitly resolves way more sharpness similar to what produces an A7sii, gh4 or gh5 in HD so i was sure that the 5d breaks something in between what you catch with rec_mod and what you get as h264 or hdmi out.

Now what i'm undestanding from your saying is that the compress Canon's YUV422 buffer is what is sent to the hdmi out right? If it's the case it's useless for sure but can it be something else. Can you think of any other way to compress to something like 15~25% even if it means we loose all the color depth from the raw but just keep that sharpness from it. I'm insisting on this because Personnally I would be so much be happy with an MLV without any color depth additional value but which would just have the real sharpness we can get from the current raw and that could allow like 60~90 minutes on a 128GB card.

Also I want to congrats on the slowmotion mods all options are awesome i just did a 1080 60P in 2:35 flawlessly. As mentionned the 1080 48P as a few corrupted frames and its preview is also a little off towards the down of the image compared to the canon preview but appart from that we could say this is 99% usable.

Finnally. The 3K Mode and UHD are absolutely awesome. I personnally don't see how we could use over 3360x1528 with spanning and sound in the future so any preview that would be x2 would just be the ultimate thing to make this usable.

I undestand that for now it's complete unknow hope someone will find a solution one day otherwise in regards of you're sayinf:

Quote

I can try to include the centered x5 zoom mode (which you can already use in the "classic" crop_rec builds), if that helps.

Sure It's definitly better than nothing to have access to canon x5 center preview again instead of pink frame eventhough it will not help with framing but it will help to pull focus correctly durin shooting.

Bravo again keep the good work. Looking very forward to integration to MLV2 with spanning and sound enabled...

The 1:1 1920 and 1:1T work awesomely fine for me with crop_rec enabled and raw disabled. It would be so good to have 1920 60p (1920x800) in non raw mode but if I choose either 720 60p or any other Canon frame rate/resolution combo, I just get the standard rates.... Is there a trick to it or is the crop_rec just for ml_raw ??

WOW!!!!! :o just saw this new development are you serious 4K on the old 5D Mark III. Like I've said before Magic Lantern Team never sleeps, thank you @a1ex and the whole Magic Lantern Team for those awesome new features, and previously impossible features :D

Actually, the preview is off as well as the file. In 3K mode framing is correct in preview:

(https://s11.postimg.org/xpb3jor03/20170404_233407.jpg)

But in 4K it's totally off (in the last build it was fine)

(https://s14.postimg.org/n9x110mb5/20170404_233433.jpg)

I tried altering CMOS[1] lo hi values. At lo 10 the image was centered vertically but with a weird overlay on half of the screen:

(https://s10.postimg.org/e8armmym1/20170404_235712.jpg)

It's a transparent overlay of what's actually beneath the frame. I tried playing with different values for CMOS[1] lo but with no luck. In the previous build this problem cropped up (excuse the pun!) from time to time, but I could restart the camera and it was back to normal. With this build UHD is vertically shifted whereas 3K is correct.

TANGENTIAL SUGGESTION: If there's some way to reliably control the positioning of the crop area, this is a great way to turn a wide angle lens into a digital shift lens, for those interested in architectural stuff and precise composition.

Another quick suggestion: It would be nice to have more control of the vertical resolution, if that's possible. Let's say I wanted to shoot in 4:3 using a 2x anamorphic. The max res I can get is 2048 x 1536, even though my card could handle a higher resolution.

I got that overlay when I messed with the 1920 tall positioning on the 1st or 2nd build.

I shot standard 1080p raw vs 2.8K raw at the same ISO and aperture, taking a few steps to get close to the same framing; the 1:1 2.8K crop appears to be about half a stop brighter than the 3x3 binned 1080p. Somebody else might wanna test that to confirm.

Canon's focus box appears for 1-2 seconds when starting recording, then again when stopping. No BUSY message (I was getting it with previous builds). Menu works fine, both before and after. Tried 3K 24p and 1080p45, starting from ML defaults. Also tried frozen LV.

If the link wasn't obvious: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

In case anyone finds what I've done helpful, I offer the following as we experiment with this exciting new ML development.

This is only for the still photographers ;)

As at the moment LV preview is not working, so composition and exposure setting are not possible, I've written the following simple script to toggle the crop and silent features on and off (only works in LV).

This way you can compose without the feature on, toggle to use the feature (I use RATE as the toggler), do a half shutter press to get your DNG (I use DNG for stills), then toggle back.

The only 'strangeness' is that after toggling you will need to go in and out of the ML menu (TRASH) to reset the shooting state. The bottom left of the ML menu will give you feedback, eg FLV.

Canon's focus box appears for 1-2 seconds when starting recording, then again when stopping. No BUSY message (I was getting it with previous builds). Menu works fine, both before and after. Tried 3K 24p and 1080p45, starting from ML defaults. Also tried frozen LV.

Things started to get a bit screwy yesterday after I put ML on an SD card. I'm thinking doing the install with an experimental build may not have been a great idea. I'll do it over with a nightly.

Testing the last build (magiclantern-crop_rec_4k.2017Apr04.5D3123) on 5D 123 and after few tests camera freezed.I had to remove the battery and now the camera doesnt' boot as usual.The LCD stay black, back buttons don't respond, video mode no longer works, camera can only take pictures.

Magic Lantern version : crop_rec_4k.2017Apr04.5D3123Mercurial changeset : e7e1fdb85f6b (crop_rec_4k) tipBuilt on 2017-04-04 15:42:08 UTC by [email protected]Free Memory : 176K + 3964KI went back to previous working version (compressed_raw.2017Apr01.5D3123), same symptoms and I had this crash log :

ALSO don't type in INSERT FILE LOCATION. Thats where you place the location of the folder you're working in. :p

2. Save that file and what ever you name it (I NAMED MINE DUMP) be sure to add .cmd at the end of it. Which will make it a CMD file.

3. Place the MLV_DUMP file into the same folder as the the previous file we just made.

4. Drag you're MLV file to that location and then drag it over the CMD you made.

5. You should now have a folder with a bunch of DNG files.

6. If you can't open the dng files and get errors in let's say After Effects. You need to deleted the The first DNG file in that folder. It's the one with just zeros at the end of it.

I'm use to magic lantern MLV files working with my normal programs. So if this is already old news i'm sorry for posting. I did however see a few questions regarding this issue. Some are getting pink dots or weird rainbow-ish frames.

Now with new April4th build, yesterday problems are gone. Testing 1920 50/60 3x3. Camera boots every time I turn on as expected... WAIT A MOMENT !!!

While I was writting this post with camera on, I've heard the camera disconnect live view and connected again. On screen find for a moment a error message. I didn't have time to read it, just "created a LOG file" or something similar.

This ML ASSERT message appears when in crop_rec mode 1920 50p 3x3, I change (in Canon menu) from 1280x720 50 All-I to 1920x1080 25 All-I before change crop_rec to OFF. I need to reboot 2 times and then all come back to normal operation...

I shot something like 12 takes.I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)and a series of many warnings.I removed the battery, start from scratch, default ML settings, load modules.I set 10bitSet the Canon video to 1280x720, NTSC.Everything as expected.

In order to reach optimum performance, I set the Canon video back to 1920x1080, PAL.The resolutions 3K, UHD and 4K had less height that the promised in ML forums.e.g 3072x1728In these resolutions, I couldnt record more than 3 secs maximum.Surprisingly I managed to record 3568x1320 for much longer duration. <--- pretty strange not to be able to record longer for lower resolution

I was experimenting with the lossless lj92 compression in MLV files. I wanted to see if I can decode the image and reinterpret it as a raw cfa pattern. I came across a number of problems that could also be a problem in mlv_dump. I took the liblj92 implementation by Andreaw Baldwin found here:https://bitbucket.org/hudson/magic-lantern/src/7a9b6805756c3b86f2174bac00433c544a976501/modules/mlv_rec/lj92.c?at=mlv_rec_lj92&fileviewer=file-view-default (https://bitbucket.org/hudson/magic-lantern/src/7a9b6805756c3b86f2174bac00433c544a976501/modules/mlv_rec/lj92.c?at=mlv_rec_lj92&fileviewer=file-view-default)

But I could not make it work. I had decoding problems due to the fact that the library is missing some support for multiple components. I then came across the implementation in tiny_dng_loader by Syoyo Fujita here:https://github.com/syoyo/tinydngloader/blob/master/tiny_dng_loader.h (https://github.com/syoyo/tinydngloader/blob/master/tiny_dng_loader.h)

I then extracted his changes back into a simple standalone .c and .h file and tried that and viola, the decoding result was much better:(http://i.imgur.com/5C88tC6.jpg)

I posted the extracted version of the library on GitHub:https://github.com/martinhering/liblj92 (https://github.com/martinhering/liblj92)

Just saying, if you encounter any decoding problems in mlv_dump with the mlv_rec_lj92 branch, give my version a try.

I shot something like 12 takes.I tried 14lossless and got a warning at the top of the screen (couldnt read due to age)and a series of many warnings.I removed the battery, start from scratch, defauly ML settings, load modules.I set 10bitSet the Canon video to 1280x720, NTSC.Everything as expected.

Yes, same as me. Now message is:

[76] CEC: stack warning: free=128 used=3968

I'm totally lost...@A1exI'don't know what to do, so if you can point me to what to do to help, just let me know.

After the messages I can reboot and I can record continuous without corrupted frames (allways talking about crop_rec April 4th build, 1920 50p 3x3 mode)... so this is not a big problem, but I have no idea of code so no idea what all this messages mean and what to do... sorry :-[ I'm limited to explain what happens...

Did some changes that might help with memory corruption (new build posted).

In particular, I think I've found a way to limit compressed data size when it ends up larger than the allocated buffer. Previously, passing an arbitrary limit for buffer size simply locked up the camera at certain resolutions - it appears this limit must be aligned at 4096-byte multiples (maybe a bit less, but in any case, at least 1024).

If you still get stack overflow, please report even if you don't have a way to reproduce it. This kind of bug is usually very serious.

Is this mean we could get a semi brick or definitive brick ? What is the long-term risk ?

Yes, as with all ML builds. Explained a little here (http://www.magiclantern.fm/forum/index.php?topic=12627.msg170133#msg170133) and also somewhere in the Tragic Lantern threads. Currently I'm diagnosing this report (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182395#msg182395), and it appears to be, indeed, caused by some bad settings reflashed by Canon code into ROM - possibly after memory corruption caused by ML.

The warnings from the download page are not there for fun - this kind of errors cannot be prevented on the current processor design, as it doesn't have a MMU (also valid for many other simple CPUs that use non-volatile storage updated frequently). Canon started to use a processor with MMU with DIGIC 7 (https://chdk.setepontos.com/index.php?topic=13014.30), but they don't seem to use its memory protection features (they simply do a flat memory mapping, except a 0x1000-byte block unique to each core).chd

Just in case, make sure you have a copy of your ROM in a safe place (any recent installer tells you how to do so).

Fortunately, that's no longer causing memory corruption (so it's a minor bug for now). It's simply invalid image data being compressed (and output size resulting higher than initial size). The extra data is no longer written to RAM on latest build (and before, it was written in the place reserved for future frames, not on DryOS data structures or persistent setting areas - to my knowledge).

The stack overflows are worrying me, though (as they do indicate memory corruption in the middle of DryOS).

Thanks for those groundbreaking efforts :) :) !!!!!!Don't know if it helps. If I can remember...

I got just while moving in the menu but without CF card (only SD):white flashing "[93] audio-common-task stack warning free=180 used=3016"then red flashing "[91] clock-task st(don't remember the end of the word) free=36 used=8156"

Really surprised for the extremely low rolling shutter in 1920 50p 3x3 mode... but panning, I get a cut line in the top part of the frame, only with panning! it dissapears in static frames.Edited: I can't see the cut line in live view, only in processed dng's.

I returned to crop_rec 113 april4th build due to the short recorded times I get with april6th (I don't know why...) :(

Proof that it has shorter recording times, please! A static test scene (without varying light levels and with the same focus), 10 test runs with each build, starting from a formatted card, at the same resolution and ISO, should do the trick.

I then kept this exposure and used the full res LV in the experimental build.

As soon as I pressed the half shutter press, I got the saving message. I pull the card immediately.

The two DNGs were the same, ie I seem to get a 30s silent in a couple of seconds.

The cr2 and the two DNGs were exposed OK, ie ETTR like and not overexposed.

I obviously don't understand how long exposures are working with the crop/silent stills.

Best guess: you have tried in photo mode (which I didn't try yet), with exposure simulation. So, if you have metered a 30s ISO 100, the LiveView is probably at 1/FPS and a very high ISO. An example, if FPS override was set to 5, the LiveView frame might have been exposed at 1/5 seconds, aperture wide open - even if you have selected something different in menu - and ISO as high as required to get the same brightness as with a 30-second exposure (considering the other two variables).

@A1exThanks for your assistance ! Don't hesitate to ask more if necessary ! I am not stuck ;) .

a) more infos about my problem:

a.1) without ML card (I use the SD card for ML) a.2) CF card in camera (formatted in camera) a.3) I go to Canon menu : yellow settings menu / SETUP3 / "sensor cleaning" and set "auto cleaning" to "enabled" a.4) I switch off the camera : the sensor cleaning screen appears as before; the red bottom led is not lit. a.5) I then switch on the camera : the sensor cleaning screen does Not appear; the bottom red led is lit. a.6) if I go again to the Canon menu : yellow settings menu / SETUP3 / "sensor cleaning", the "auto cleaning" has been set to "disabled"

b) reinit all settings : no change

b.1) Canon menu / yellow settings menu / SETUP4 / "reinit all settings" b.2) the "auto cleaning" feature is actually set to "enabled" if I verify in the Canon menu b.3) then I get the same as above in a.4) , a.5) and a.6)

[EDITED 2017/04/07]c) circumstances ? no CF card ?

Unknown

I think (not sure at all) it may be due to the fact that I tried to repeat the problem I got with no CF card in the camera (silly thing : should not be done without further investigation): the camera got stuck; no message ; I had to remove the battery.Investigated by A1ex (thanks a lot) : no such problem when switched on with no CF card (see post #156).

Side note: I've let the camera on for 10 hours, on a power adapter, with today's build, on the 3K video mode, 3072x1920 in mlv_lite menu, lossless compression, "framing" preview, in standby - so it would estimate the compression ratio every 2 seconds. Every 30 minutes, it closed LiveView for a second, and returned to LV (I could hear it when I came home).

Temperature after this experiment was 62 degrees C. The camera was enclosed in a drawer during the entire experiment.

I've now downloaded the previous build (Apr04) and will perform the same test overnight (or until it crashes).

edit: it crashed after 15 minutes or so of recording short test clips in 50p 3x3; back to Apr06, no crashes yet.

Trying to reproduce the live view disconnection, I let camera on... This time get this message:CTRL SRV Stack Overflow free=84 Used=163008176[...]Suddently get this message in top of screen (in red letters):[103] COMPRES_TASK: TASK WARNING: Free=164 Used=3932

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

Photographers need to look to using the crop approach and the current FRSP, together.

I'm looking forward to trying to script these two techniques together in Lua :)

Another case of bad setting flashed somehow into ROM. Reproduced the error in QEMU. Appears easy to fix, since these settings fortunately have backups (wear leveling?). I've already unbricked a similar case a while ago (also 5D3), and I've already booted the GUI in QEMU on the affected ROM.

Two failures on the same build are a bit too much to be a coincidence. For now, my advice would be to avoid the Apr04 build, even though some reports say it's faster (I'm willing to bet it's not, but I'll benchmark it, just in case).

I'm using MLV_Dump for the newest crop_rec module with higher resolutions (4K, 1080p48 etc) build - And I'm getting vertical stripes on the .dng exports.Is this normal for now?...or am I missing something simple? I have not used MLV_Dump before now (and I'm a code ignorant idiot)Test shots were lower ISO 100-400, various resolutions tested from April 4th build - all exhibited some vertical stripes (some more noticeable at lower ISO)

Running this code on Win 7 works for me to export the .dngs's, I just don't know if vertical stripe correction is an option yet in this version of MLV_Dump...or I simply need to somehow amend the .cmd code to 'switch on' the correction in MLV_Dump?

Link to screengrab of most prominent vertical stripes:https://www.dropbox.com/s/md2inmxhaj604gv/M06-1750%20%2800123%29.jpg?dl=0 (https://www.dropbox.com/s/md2inmxhaj604gv/M06-1750%20%2800123%29.jpg?dl=0)

Two failures on the same build are a bit too much to be a coincidence. For now, my advice would be to avoid the Apr04 build, even though some reports say it's faster (I'm willing to bet it's not, but I'll benchmark it, just in case).

So ISO was the culprit for not get continuous recording with april6th?... My fault. :-[

No STACK OVERFLOW error for now...

Edit: in this tests I got the same corrupted image that before when panning (top part of frame with motion cutted). Can I ask if is easy to adress this, or not? That would make the recordings usable...!!!! (at least fro me).

Edit: in this tests I got the same corrupted image that before when panning (top part of frame with motion cutted). Can I ask if is easy to adress this, or not? That would make the recordings usable...!!!! (at least fro me).

Here's a boring task: what's the number of pixels before the cut? (min/max/median would be great)

Here's a boring task: what's the number of pixels before the cut? (min/max/median would be great)

:-[ Sorry for that!!!... Its only thing that makes image unusuable.

I've measured px and the pattern is very strange... min distance I've found is 7px, max is 125px, but it doesn't follow a logic. At first it seems increassing with the panning velocity, but looking close, during panning, one frame have 98px (from top to cut) the next 112px, the next 18px, one frame (in between all of this) is correct, and for example, the next have 73px...

Min I found: 7pxMax I found: 125pxAverage from random 17dng:83,8px@A1exIs this what you are asking for?

To decode, you will need a custom mlv_dump (also included). It works by simply dumping the lossless JPEG payload to DNG (that means, processing options such as bad pixel or vertical stripe fixes are not working yet).

@A1exI don't know how to save the debug msgs after a "manually started" cleaning process:

Quote

Before the log gets saved, you should try to perform the sensor cleaning, so messages about it will get included:

When I switch the camera on, the "auto sensor cleaning" is set to "disabled";The bottom red led is lit a while and then a few times (blocks of data written to the card ?)It ended at 14:42I went to the Canon SETUP3 menu to start a manual immediate cleaningThe camera made some noise as if taking 2 pictures then this greyed menu item became yellow again.I then went to "auto cleaning" item to set it to "enabled" I then went in the ML DEBUG menu and pressed SET button with cursor on DebugMsgLogThe bottom red led was lit a very little while (but I saw no file on the card created at that time = 14:46).Finally I switched the camera off.

The following file is generated just when the camera gets started (gets the creation time = 14:42):

Diagnosed the bricking issue - it was caused by a null pointer bug in ML :(

Only Apr04 is affected - both older and newer builds are safe.

Fortunately, it's easy to recover (at least in the two previous cases). One uint32_t at an arbitrary address (between 0 and around 60MB, with a non-uniform probability distribution) might be overwritten with 0xA5A5A5A5 (and eventually end up saved into ROM at camera shutdown, when Canon code saves camera settings).

@BBA: if you believe your camera might be affected by this bug, please send me a recent (bad) copy of your ROM, alongside with an older (good) copy.

If you don't have any backup, copy the oldest files from ML/LOGS you can find on a ML card (hopefully these are "last known good" configuration).

To get a fresh copy, simply delete ML/LOGS from a ML card, and it will be re-created at next startup.

@A1ex I got the nasty error explosion today using the April 6 build. I think it was probably because I was using an odd resolution3008x1280 or something like that. Pulled battery everything seem fine with the camera.Tried again with 3072x what ever 2.35:1 is and it seemed to work ok.Would you like me to report the bug. Here is a link to what was on my card. Error codes and all. https://www.dropbox.com/sh/8lmopv8z7p2i91f/AAA4saYTmLE7rFQ3t7nSUBCaa?dl=0

Can you share a small part of your MLV file ? Or record a very short part of movie for get arround 10 frames.Be sure you're using the good version of mlv_dump (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/7/artifact/modules/mlv_rec/mlv_dump.exe) too.

First up, it's a bit sketchy getting signal from the HDMI - have to power on, go into Canon menu, back out again, screen flashes all kinds of weird, live view then appears. First time I tried the camera crashed.

Successfully unbricked the two cameras that had problems earlier in this thread! (ju64 and filipe), and also solved BBA's issue.

More cameras might be affected (actually up to 817); will add an automatic check to identify them in the next build. The side effects range from nothing at all (overwriting an unused memory location) to soft-bricking (recoverable) and anything in-between (e.g. BBA's issue). The bug can overwrite any setting (aka property in Canon code), or the data structure itself. My own 5D3 was also affected, without any obvious side effects though (as I don't know where the affected setting is used in Canon code).

I'll probably stop adding features for a while and will look into ways for preventing this class of bugs.

Strange how plugging in the HDMI seems to lower the data rate by 10mb/s?

It's not strange at all, as the camera has to work 6 times as much (1920x1080 vs 720x480) to create the HDMI image (strictly talking about the LiveView buffer). Most of the work is done by the image processing chips (external to the main CPU) and the DMA (EDMAC), so most of this load is on the memory bus.

The "Frozen LV" (previously known as Hacked preview) works by stopping LiveView buffers from being updated (therefore freeing the memory bus and a bit of CPU time).

Same for me, but the same thing happens to me with older (and more stable) builds... To get hdmi signal I often go to play button and back out again. In my case using a Zacuto EVF.

I get continuous recording with april6th build with ISO100 1920x960 50p GD off 14bit losseless with or without hdmi in 5Dmk3 113.

I didn't have much problem with previous builds. Might be because right now I'm using a Leica prime with no electronics, so that might be throwing things off a bit (camera probably thinking a lens not attached).

Bit weird how the 60p mode seems to have a much lower MB/s (57MB/s with HDMI, 67MB/s without), so I'm guessing its something a little more complex than just write speed which is the issue there. a1ex / devs if I can help test in any way, please do let me know.

Off to Bermuda tomorrow, so I'll do some more testing of this mode there. Thanks for the suggestion :)

Be aware with ISO. For me 800 is where problems start. I can get only few seconds with 800 ISO. And as I report in post #148, I get a "cutted" image in the top of frame with paning movements... So no usable for serious work...

I'm testing this out on my 5D Mk III on 1.2.3. I'm really missing the "centered 5x zoom" option from the older crop_rec module. I was previously running crop3x.2017Jan13.5D3123 version and using the centered 5x zoom was killer for avoiding the warping from the corners of the frame.

Am I missing something, or is this something that I just have to wait on future release?

@a1ex What if you don't know how to run that script.. Can you point me to a detailed instruction on how to run that code, would be greatly appreciated.

This is what I just did to test it (for Windows):1. Install python for your computer: https://www.python.org/downloads/ (https://www.python.org/downloads/)2. Make sure to check add python x.x to PATH 3. Make a backup of your current camera's ROM (using magic lantern)3. When done make a new folder and copy your ROM1.bin to that folder (i made E:/tmp)4. In the same folder make a new .txt file and rename it to testrom.py --> make sure you have file extensions visible5. Open the file in your favorite text editor (i like notepad++) and copy the code in a1ex's post6. Open a command window in the folder you just created (press right click on the folder while holding shift key, select Open command window here and type this:

This is what I just did to test it (for Windows):1. Install python for your computer: https://www.python.org/downloads/ (https://www.python.org/downloads/)2. Make sure to check add python x.x to PATH 3. Make a backup of your current camera's ROM (using magic lantern)3. When done make a new folder and copy your ROM1.bin to that folder (i made E:/tmp)4. In the same folder make a new .txt file and rename it to testrom.py --> make sure you have file extensions visible5. Open the file in your favorite text editor (i like notepad++) and copy the code in a1ex's post6. Open a command window in the folder you just created (press right click on the folder while holding shift key, select Open command window here and type this:

Near instant? I believe when you "update" firmware with ML it will backup the ROM then. I just formatted my SD card in PC, format in camera, load ML from PC onto card, update firmware to setup ML, grab the ROM1.BIN and follow instructions.

Nice save, @Danne! :) (will have to check mine first thing in AM after I get back home from work) -- good thing I only used the April 4th build on the 'B' cam that I just scooped up from Craigslist earlier this week and not my original baby, Ha!

Pushed the saturation a bit during the grade because I wanted to see how it holds up. I'm impressed.

Indeed, extremely impressive. What were the highest ISO's you were able to use for the night scenes @pocketrubbish?

My apologies for lack of paying attention to details in your clip as I was obviously all eyes on the static shots. Similar effect if one were to stare at a Television showing nothing but static and it just sucks you right in which means you got the job done well even if it wasnt intended for.

The reason why I asked (hope you're reading this @a1ex) is because I'm an avid fan of using DarkFraming average processes for high ISO shots and ever since this whole lossless compression 4k crop_rec stuff came out for the 5D3 -- haven't been able to figure out how or why when I apply DF avg process it just don't correspond properly and not sure if this is related to the modified mlv_dump spitting out different headers for each DNG or some sort?

I think because of this it's making it a bit trickier for the DF avg process to work properly (at least in @Danne's cr2hdr.app is what I use) so not sure if this at all possible due to the limitation factor of this new lossless compression codec in MLV 14-bit lossless or perhaps just a simply fix in the code would do the trick?

@alex:i guess its a good thing to add this checker to the module itself.so all users get informed who potentially don't realize there are broken properties.e.g. not crashing due to the property being bad, but weird image metadata or correction curves.

would warn them on display, telling them that the old version had a bug and this cameraseems to have a property being flipped.

I did several 4K recordings with 14bit lossless. However I noticed some fat blotchy areas of compression, easily detectable.I was shooting with 50mm 1.8 and compression was applied naturally in out of focus areas.I thought 14bit lossless was visually the same as the ordinary 14bit.

I was using 4K 1:1Selection PAL/NTSC mode or 1080/720p through Canon Menu, changing Aspect Ratio, was giving me some more resources for more FPS.Then I switched to 1920 50/60p, 1920x1080.I pressed REC and got a warning "EDMAC timeout"Brick camera

Then, get a fresh copy of your (bad) ROM with the portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.0) and contact me on IRC (online for the next 2-3 hours, but busy, then tomorrow at about the same time).

Then, get a fresh copy of your (bad) ROM with the portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.0) and contact me on IRC (online for the next 2-3 hours, but busy, then tomorrow at about the same time).

a1ex, how to connect to you in IRC#magiclantern? I login in IRC#magiclantern with my nickname, but there is no yours nickname in the nickname list.

To decode, you will need a custom mlv_dump (also included). It works by simply dumping the lossless JPEG payload to DNG (that means, processing options such as bad pixel or vertical stripe fixes are not working yet).

I wanted to record h.264, my daughter with her new bicycle. No RAW, no anything.Crop Mode and Raw Recording werent loaded.The camera was recovered after a bad crash and removed the battery.I never reset to ML defaults, or reload the modules as the usual procedure.

I wanted to record h.264, my daughter with her new bicycle. No RAW, no anything.Crop Mode and Raw Recording werent loaded.The camera was recovered after a bad crash and removed the battery.I never reset to ML defaults, or reload the modules as the usual procedure.

Was ML still in the camera? After you pull the battery​, on the next power up the modules won't load. If you power down then power back up again, the modules will load again with your settings and all. Could this be what happened?

After removing the battery, its was an up normal shutdown.I havent touched anything.Next day I turn on the camera to shoot my daughter.I saw that the modules didnt load. Τhe setting werent reset.I shot h.264, (I thought)When I load the CF card into the computer, I was surprised.There were 4 videos, two MOV and two MLV.

@quentin Do you remember what the record symbol looked like for any of the clips? Was it the Canon rec icon or did the cam beep then record like when it does for RAW. Also did you try processing the MLVs ? And did you test your Rom with the Python script?You could also try recording again. Making sure that the mods are disabled for sure. Remember that this build is bleeding edge.

The recording icon was normal, h.264 and MLV however I didnt pay attention to the change.I processed the MLVs, they were normal.Nope, unfortunately I have no clue about python scripts, ROM processing etc.

I just checked the camera now.Modules restored as the crash never happened.Definitely I didnt reset to defaults and didnt select which modules to load.I am familiar with this first-aid procedure and did it many many times.

If anyone is interested I think we have everything to do it for 500D (Full res LV others will be useless - too slow sensor).3 years ago I increased the width, but I did not know how to increase the height.

If anyone is interested I think we have everything to do it for 500D (Full res LV others will be useless - too slow sensor).3 years ago I increased the width, but I did not know how to increase the height.

Very useful , thanks :D I was wonder on this , I was going to pm you if you have any notes that could help.I still can't reproduce you test in the 500d , registers are different on 5d2 but close thou.my problems is the timer A can't lower it lock to 610 , and ever time I try agjusting the HeadtimerI get a frozen camera/lockup (battery pull) so if I can get timer A to lower I think I will have it I have the width , but compressed to the left in a 2K crop window , anyways thanks it helps .

This is where I am at with 700D so far, just trying to figure out how to change height without it getting all scrambled, was able to change width past 4096 but was getting borders so brought it down to an even number, am able to go to almost 1500 on height until it scrambles, so I am assuming other registers need to be adjusted to make it work.

Also, fps drops to 6.5fps with the register changes to make it work, was able to force it to 9fps with C0F0[6014], any higher caused some sort of overlap on image.

Here are some examples of resolutions I would get by adjusting C0F0[6804]. It seems as I adjust the 3 on the left, the height would change, the 3 on the right, the width would change, the middle 0 seems it has to stay 0. I can go much higher on the height, but as I said, it gets crazy looking, shifty and predator vision like.

A bricked camera is easy to identify - it does not boot (in other words, it's as useful as a brick). So far, there were two cases, both successfully recovered.

However, the Apr04 null pointer bug might (or might not) have changed some of your settings somehow. Since you have not noticed any side effect yet, your camera is probably OK, or you may find out some strange behavior when using some functionality that you don't use very often. That doesn't count as "bricked", but still, those (persistent) changes that appeared because of that bug are not desirable.

By any chance could it be possible for some of us to have used this particular build and NOT be affected by the null pointer bug at all @a1ex? :P

Of course - what happened was not deterministic.

I've just posted a new build that checks whether your camera was affected by this bug. All you have to do is to upgrade - if no message pops up, your camera was not affected by that bug at all. Otherwise, just follow the instructions.

For this particular bug, the persistent changes are easy to identify, because the value that could get written into ROM is always the same - 0xA5A5A5A5.

Note that any other code (from ML and even from Canon) can cause similar issues (and probably much harder to identify). We have patched one such issue many years ago - on most camera models, a bug in Canon's startup code overwrites one single (predictable) memory address with 0 (4 bytes). You can check it by looking for ARMLIB_OVERFLOWING_BUFFER and tracing the patched address in QEMU (e.g. set a value at startup and watch it getting overwritten). In some camera models, this address ended up right in the middle of ML code (thus causing unpredictable behavior that changed with... the amount of code before the affected spot - e.g. insert a printf and watch things working in a totally different way).

This is a limitation of the current CPU and operating system - any task is allowed to write at any memory address. On top of that, Canon code saves their settings to ROM (by reflashing), so all it takes to soft-brick a camera is a programming mistake that results in writing at the wrong memory address.

I can think of a run-time check that detects any writes to Canon's property data structures (some of these are the persistent settings that get flashed into ROM) from tasks other than (Canon's) PropMgr, but it will definitely have a noticeable overhead (impact on performance). I can also think about adapting g3gg0's mem_prot module (which, back then when g3gg0 wrote it, was way above my level of understanding), but this won't catch badly configured DMA transfers. I can also imagine a routine that would check the integrity of Canon's property data structure right before they get saved into ROM (and maybe lock up the camera if problems are detected).

nice work @a1ex. I tested new build and now message pops up.I was/am curious about this error/bug saying cache is locked.

I also can't get 1920x1920 working, can't get any higher than 1728px.Before the april6 I was able to record square footage.Not sure if you changed something, or what happend.(http://i.imgur.com/Q5dA739.jpg)

Also If I do 3k ISO400 I get a crash when there is to much detail moving.(http://i.imgur.com/27HLwt7.jpg)

Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?

I've just posted a new build that checks whether your camera was affected by this bug. All you have to do is to upgrade - if no message pops up, your camera was not affected by that bug at all. Otherwise, just follow the instructions.

Fortunately, no messages prompted up after upgrade w 2017-04-10 build though I am curious to see what the instructions would be for the affected ones.

Thanks for double checking @a1ex! ;)

Actually decided to try this (since it was reproducible last time) again w 1080p 45/48 3x3 under Crop mode while in 60p from Canon menu together with FPS override set to 45p and can now confirm that this bug has either probably gotten worse or I'm doing something incorrectly (which I doubt) but feel free to tell otherwise.

Hence the reason why I had to use an iPhone to snap this screenshot because no matter how quick I can try getting back into ML menu upon restart (after each battery pull) and getting to scroll around (both Canon or ML menus) within 5 seconds more or less was impractical. It just freezes up the LV.

Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?

How were you able to achieve 15-20 seconds without the LV display freezing up within 5 seconds more or less @Markus?

Meanwhile gonna try defaulting Canon settings to defaults and reload a fresh 2017-04-10 experimental build again just to double check.

No lockups here. Only hard to get cont rec on alot of the sweetspot resolutions. Some settings of mine: Global draw on, canon in 720p all-I 50p mode, croprec 3x3 binning 1920x1080 47/48p @ 45p. Lossless 14bit. Without global draw i could not get a bw preview in this mode. Anything that sticks out?

The file size estimation got confused some times when switching between modes. It also got the wrong fps when switching. I went to photo mode and back as a workaround when this happend.

For now, FPS override only works in Low Light mode for reducing the frame rate (known bug).

For 1920x1080 at 45 fps, set 50 fps in Canon menu.

That did the trick. Haven't use PAL in years (I'm on NTSC land). Should have thought of that to begin with and Thanks for the tips.

So far 1920x1080 @ 45p MLV-lossless 14-bit seems to be continuous even w HDMI out to an monitor (clean native feed :D) which is a PLUS for gimbal work because obviously the 5D3 will have the grey previews while mounted on the Ronin (which will go off from 'Turn off LCD' within 5 seconds during each recordings -- useful feature from 'Powersave in LiveView' under Prefs within ML menu) and can then focus on the external monitor instead with a peace of mind! 8)

@a1ex Just tested April 10th Build so far so good. One thing that I've noticed is that I can record at higher resolutions for longer now for example 3520x1498 - 2.35:1 I can get over 12 sec on a Lexar 128GB 1066x card before it would record for like 2 sec on the April 6th Build great work. Also one thing that I've noticed is LiveView in 3K mode when recording @ 2.35:1 is all grayscale now, on the April 6th Build it showed both color and grayscale.

So far, I've got 3 users with warnings about the null pointer bug (besides the 2+1 ones already solved over the weekend). Two of the new users had a false alarm (the modified value from ROM not affecting the functionality at all, being in unused areas), and the third user had his camera successfully patched.

Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?

Pulled Battery then restarted and reloaded mods.The only things that I don't like are the current gray scale that it jumps to when it records (I know its purpose is to reserve resources)The inconsistency of clip length due to changing compression.And I find I cant get over 4 seconds on average with a 128 GB Lexar 1066x 160MB/s card.I know this is very experimental I am just reporting my findings so far.

Ran ML April 10 4K test my 5D mk3 and is indicating the problem of null pointer bug from April 4 ML 4K experimental. Reverted to latest nightly 1.2.3 2017-3-30 build which seems to work fine. Casual user of ML & not sure if I need to do anything to address potential bug?

Tried some of the new experimental modes on Fw 123 today. Looked really good but i could not get 1920x1080 45p continious on my Komputerbay 1066x I got around 10-20sek rec time. Has anyone achieved 45p continuous on FW 123?

I failed to recognize how noise at higher isos gives less effective compression. 1920 1:85 45p at iso 800 seemes to be max cont rec for me.

The last feature complements the well-known full-resolution silent pictures - the new implementation will be usable at fast shutter speeds, without the exposure gradient - but with rolling shutter (just like regular LiveView frames).

That footage is recorded unstretched right? So cr2hdr.app shouldn,t add the compensating destretching tag to the dng file. I could maybe put in a "not stretched footage" in the menu so it would leave the tag alone.Maybe the info is already in the rawc metadata? Don,t think compressed_raw mlv_dump keep that code yet? Can you upload asmall mlvv sample Beauchampy?

That footage is recorded unstretched right? So cr2hdr.app shouldn,t add the compensating destretching tag to the dng file. I could maybe put in a "not stretched footage" in the menu so it would leave the tag alone.

Yes, it's unstretched 960 50p 3x3 binning.

Do you think this is cr2hdr rather than ML?

Edit; I also tried turning the preview to auto (switched to greyscale low fps) and the problem persisted.

Definately tag added not ML. Could you upload a short sample? In latest cr2hdr.app you can create a sample package in mlv_dump menu. Selection (12).Latest version also combines both compressed and non compressed versions of mlv_dump so no need for any 4k version now.

Definately tag added not ML. Could you upload a short sample? In latest cr2hdr.app you can create a sample package in mlv_dump menu. Selection (12).Latest version also combines both compressed and non compressed versions of mlv_dump so no need for any 4k version now.

Tag is added by mlv_dump(ml-dng version, dng.c code from dmilligan).Could you upload the zip package? Much smaller in size. I have no good internet connection.yes.https://bitbucket.org/Dannephoto/cr2hdr/downloads/

I took this as you were getting stretched footage? Anyway, the tearing issue is as you say something coming from the compress raw itself not from mlv_dump. About stretching issue it all looks ok to me.

Ok, just took a quick lookI took this as you were getting stretched footage? Anyway, the tearing issue is as you say something coming from the compress raw itself not from mlv_dump. About stretching issue it all looks ok to me.

That was only happening when my screen suddenly populated with errors and logs. I think crop_rec had crashed. When it was working properly (non stretched 3x3 50p @ 960px) I get the line tearing.

@beauchampy I got 1920 x 1080 48p 3x3 without tearing but rec time was inconsistent.I also got 1920 2.35:1 @ 48p without tearing. That was on the very first build

However, on the 4-10-2017 build I took about 5 or 6 clips of 3072 x 1728 @ 23.976 last night and notice that 1 frame out of all the clips had a small horizontal row of pixels that seemed like it was offset from the rest. Ill upload if I get a chance.

I was using RAWFlow with an updated MLV_DUMP to convert 14 Bit lossless to DNGS.

As far as the frame tearing issue, I saw it on nightly builds a while back on my 7D in crop mode and eliminated it by not using the accurate preview (ml greyscale). It was suggested that processor overload was responsible.

@beauchampyI'm really interested in 192050p 3x3 mode and while I wait for a solution for the tearing issue, for me the solution is 1080p45 3x3 mode. Not same slowmotion like 50p but no tearing issue and continuous recording. Tested at ISO3200 and it works for me continuous!!!. Only framing is a little bit offset...5Dmk3 113 Lexar 64Gb 1066x

So I can use a little education here. Sorry Im behind. If I get the:ML ASSERT:RAW_IS_IDLEat mlv_lite.c:584 (measure_compression_ratio), task shoot_tasklv:1 mode:3where the text appears on the screen, is this a crash? I have been assuming it is and pulling the battery. Should I be doing this?

is this a crash? I have been assuming it is and pulling the battery. Should I be doing this?

Yes, taking the battery out after a crash is what I always do.

Unfortunately, it doesn't appear to prevent Canon code from saving corrupted settings into the ROM. Still looking into it.

edit: opening the battery door seems to be interpreted as some sort of emergency shutdown, but a clean one - settings are saved into ROM, but ML shutdown hooks (e.g. saving ML config files) are not executed. But, if you somehow hold the battery door switch pressed, and just remove the battery, the ROM is not updated.

Re: tearing I noticed this happening after the 2017-04-06 build and still persist with 2017-04-10 build as well when shooting in 1080p 3x3 45fps (50p Canon) and could this strange pattern of corrupted (plus dropped?) frames in several MLV's that spat out be related to the tearing at all?

Because my instincts are thinking that it may have started it after I had a bunch of random corrupted footage similar to the one below (is this what you were seeing also @beauchampy?) otherwise correct me if I'm wrong.

https://vimeo.com/212794399

Re: 2017-04-03 build I am now getting corrupted frames similar to the one above when shooting in 1080p 3x3 45p until I reduce the resolution down to 1920x960p which all seems fine.

Though on the older build I seem to be able to get clean recording w full 1920x1080p in 3x3 45fps which is rather confusing. Gonna need to take a step back and check again with all of this before I lose it for good. literally.

Hey a1ex,With the latest build from April 10th (the one to ignore false warnings), I am still getting a message about the null bug. I have not run into any major issues or soft bricking, but I also do not have a backup or unaffected copy of my ROM from before the 4th. Am I screwed, any solution to this without a clean ROM from before?

Unfortunately, it doesn't appear to prevent Canon code from saving corrupted settings into the ROM. Still looking into it.

edit: opening the battery door seems to be interpreted as some sort of emergency shutdown, but a clean one - settings are saved into ROM, but ML shutdown hooks (e.g. saving ML config files) are not executed. But, if you somehow hold the battery door switch pressed, and just remove the battery, the ROM is not updated.

@a1ex Hmmmmm So if I mod the switch with say.... tape, I can pull battery with out afflicted ROM being saved? Should I do this as a extra precaution when heavy testing? Would this have adverse affects on Canon code being that its an "unsafe" battery pull at that point? So many questions sorry.I suppose there is always a chance I might not even know that the ROM is affected and I could clean shutdown anyway.

I may have missed this but once you set the settings for 3k 1:1 crop, what's the diff between the regular record mode and then x5 crop mode? They seem the seem on preview but regular mode will record around 3072x1228 but x5 Crop just stops. Is it cranking the data rate or something?

Can anyone else confirm that it seems when you go into Full-res LiveView (it becomes 2+ stops brighter according to the LV as well as the ML histogram) than if in any other settings within crop mode from the latest experimental build?

Here is a quick test I did using 5D Mark III in 3k - 3:1 / Build April 10th latest one. I just Love the images that I'm getting & specially the motion, the images speak for themselves even when youtube super compresses it. Let me know what you guy's think.

I tried today with April10th build. Since I normally shoot 1.85:1 I tried 3K 1:1 (2928 × 1580), ISO 640, shutter 1/50. But the cam can only record for 5 sec with 14 bit lossless. Is that normal? Or how to be able to record constantly with this resolution, if possible? CF card is Sandisk 160mb r/w. Thanks for al the heavy work and fun to reach new heights!

Today I was doing a studio shooting for 4-5 hours, several shots.I had my camera plugged on electricity as well as the monitor.5d mk3, 1.1.3Experimental Build 10 AprilI used 1920x800 60FPS 16:9 14bit losslessAll manual, no AutoETTRWhile I was working on the shot, to adjust lights etc, Camera crashed and saw several messages.Turn it OFF/ON fixed the problem.The recording was not continuous but sufficient enough to record the useful action.I noticed that the Aspect Ratio of the monitoring wasnt right.Later, I saw that the monitoring the colours lost saturation and the Aspect Ratio was jumping from wrong to right.I also noticed that although the ISO was set manually to 1250, on ML monitor was still 1600.The temperature of the camera was 40 C

What kind of work flow are you guys using for the 1920x960 3x3 binning 14bit raw mode?

In camera? Or post processing?For post I used the RAWFlow app and just replaced the MLV_DUMP with the newest version and it processed the 14Bit Lossless just as expected. Drag n Drop.RAWFlow - http://www.magiclantern.fm/forum/index.php?topic=13338.0MLV_DUMP Update - https://builds.magiclantern.fm/experiments.html

In camera? Or post processing?For post I used the RAWFlow app and just replaced the MLV_DUMP with the newest version and it processed the 14Bit Lossless just as expected. Drag n Drop.RAWFlow - http://www.magiclantern.fm/forum/index.php?topic=13338.0MLV_DUMP Update - https://builds.magiclantern.fm/experiments.html

Nice, Premiere or DaVinci Resolve? The old DNG-files opeened right up in Premiere but these compressed ones does not.

Has anyone notice an issue re: Dual-ISO 14-bit lossless MLV seems to be spitting out with correct black levels but incorrect for original 14-bit MLV's. Could this be related to the new mlv_dump not corresponding properly to cr2hdr or at least confused with each other atm?

Anyway here are some short samples in two of each (138 MB spat out w ease from latest cr2hdr.app -- Thanks @Danne!) and all shot within crop_rec @ 3.5k: https://mega.nz/#!OwV2DTLC!rb39cVOrAlXXuSer2M6fw3vSuvMfiSG508hewIQJvmg

Found a way to prevent Canon code from saving some of the settings at shutdown, including the settings block that caused issues earlier (reverse engineering notes here (http://www.magiclantern.fm/forum/index.php?topic=19369.msg182959;topicseen#msg182959)). While not perfect, if Apr04 had this safeguard, all the cases affected by the null pointer bug would have been caught. Still, there are ways to brick the camera, just a bit less likely to do so accidentally (and, as long as the bootloader is not erased, recoverable).

How it works: whenever a crash is identified, or whenever you open the battery door, Canon code no longer saves the usual setting groups at shutdown (RING and RASEN, if you look in the above link). There are still setting groups not covered by this safeguard (still looking into it, but so far, all the null pointer errors were in the RING group).

To test - the PAL/NTSC setting is in the RING group (and, to my knowledge, only saved at shutdown). You also have a dummy crash under Don't click me.

After a bit of battle-testing, I think this should be back-ported to all models.

Wait..... what did I do wrong here. I couldn't even get one solid rec out of the April 12 build so far. I had to pull the battery every time. @a1ex Im sending you a message to a zip of my card its peppered with ASSERT messages.

@a1ex Just tried the April 12th build it's solid great work. The only thing that I've noticed is when recording the minute / second green counter stays @ 00.00 it doen't show the actual recording time.

Just wanted to give heads up to those who want most of resolution with lossless, but high ISO prevents you from it. You can sacrifice dynamic range but have a lower bandwidth requirement by using lower ISO and doing the pull in the post: the noise will be below the original gain line. Sure enough, it is worse than we'd like it, but...

Unfortunately, ISO 50 doesn't work with video, that would've basically removed one bit out of equation. Maybe there is another cheap (performance-wise) trick to lower effective number of bits, so lossless is more effective?

Here's a question: is there a way to have GD when I'm focusing, etc., but have it auto turned off (with everything else that could be turned off) during the recording?

I got the red dot in the memory patches again, no big deal right? It seemed like most of my shots using crop_rec had a bad first frame regardless of resolution. Shooting at 3504x1536 at 24fps I only got 157 frames with a Lexar 128gb 1066x card.

Also, something odd kept happening and it kept reverting back to 19fps with FPS override off.

Maybe there is another cheap (performance-wise) trick to lower effective number of bits, so lossless is more effective?

Yes - adjusting the digital gain in the raw backend appears to help (as it reduces the range of the data and cuts off the noise bits). Added a (somewhat fake) lossless compression at reduced bit depths, based on this concept (theory explained here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620)).

Also added some suggestions about what ISO range to use with each option. See the commit for more details on how I've chosen the recommendations.

Edit: just noticed the lower bit depth lossless modes only update the raw buffer every other frame, for reasons not yet understood... if you want to try, compile the sources, but you'll get files with duplicate frames (good for estimating recording times, but nothing more). If you look at a static scene (as I did before posting the build), everything appears fine.

Edit2: looks like all raw types (http://www.magiclantern.fm/forum/index.php?topic=18393.0) that include digital gain are created every other frame :( (that means, I need to find some other method for darkening the image...)

Edit3: progress narrowing down (this used to work before merging 5D3-123 into main builds)

You still have the option to fine-tune the internal parameters until you get continuous recording with no errors or corrupted frames. No programming knowledge is required (just fiddle with numbers from the menu), but it can easily take hours of tinkering until you find a sweet spot.

First of all, thank you so much for the tremendous work and the joy that you give us.At me a following problem - on all last experimental insertions in a сropmode the screen becomes pink and there are diagonal strips. In the raw mode this does not happen

Upscaling in post is a thing. I've shot stuff in 720p crop mode that I'd be comfortable projecting on a 40 foot screen. I can put nitrous in an old Honda, that doesn't make it smart. Reliability trumps (for want of a better word) a few more pixels every time.

Testing April 13 build. Much more stable for sure! Compression seems to be tricky still. I'm not the biggest fan of how slow settings (shutter ISO ect) appear to change when in Framing LV but I know it serves a purpose to get proper preview with low processing power. If we can some how cap the data rate that would be insane. But then I guess it would be lossless at that point.

On same scene I changed the shutter speed F1.6 1/50 (1/98) ISO 800 then none of the above modes made there estimated rec time.It seems that if more detail is brought out of low exposure zone, less info gets tossed which means higher data rate.

No crashes/bad frame warnings until I lowered the shutter down to 1/30 in 3072 2.35:1 14 Bit lossless 3k 1:1 23.976 preview auto. Then it crashed/bad frame warning ML assert

Anyone else just try 1080p raw with the 1080p all-i proxie with the new april 13th build? Is it broken? Here is what happened to me today.

1080p raw w/ proxie broken, raw to CF and all-i to SD - Stops recording after a few seconds, camera locks up, preview still works, global draw on or off. After messing with proxie, 14bit lossless no longer working? recording at 90+mb/sec for 1080p as stated on my liveview monitor.

I uninstall ML, format card in pc, then format in camera, then load ML software and reload firmware to set bootflag.

Still recording at 90mb/sec and in canon menu at format menu it shows no use of CF card. I am also now having issues recording with any of the crop_rec options. Recording stops very quickly. What did I break? :(

EDIT:Lossless is still working, looking at footage the 1080p raw stills are only 1.90MB

@a1ex Yeah LiveView seems to be very complicated wish I knew coding to help out. Do you think LiveView will ever be as smooth as 1080p in 3k - 4K mode? Gonna try them in 1080p today, and I'm gonna keep testing. All those new feature ad-ons makes this camera an elite camera. Thank you for your hard work.

I'am able to record in crop mod with 8 10 and 12 bits lossless. (but now in crop mod, i can't use more than the 2:35 ratio "eg : 3104x1320" and FPS are locked to 29 if we dont use FPS override. Even if we are in PAL)

Hey guys, here's a quick 23 second shot of the 8 bit lossless 3520x1322 at 60 fps with the latest April 14th 1.2.3 build. My hands are really shaky, I'm not a pro and I had my shutter at 1/80th. ISO 200, F8, 1/80th - SHAKY HANDS - straight out of camera no corrections/lut - 60mbit vbrIts working :)

Have you noticed the ISO suggestions when selecting a lower bit depth from menu? (at the bottom of screen)

Please refer to this commit (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267#Lmodules/mlv_lite/mlv_lite.cF2996T3033) for more details, and to this article (https://theory.uchicago.edu/%7Eejm/pix/20d/tests/noise/noise-p3.html) for some theory regarding bit depth, noise and posterization.

I've considered acceptable a quantization step of twice the noise stdev, which would be just a bit better than "gradient 3-bit tonality" from Fig. 3 in the linked article (log2(256/24) = 3.3), which gives the lowest ISO usable at each bit depth. For the highest ISO, I've considered "wasteful" a quantization step smaller than noise stdev / 2.5.

Have you noticed the ISO suggestions when selecting a lower bit depth from menu? (at the bottom of screen)

Please refer to this commit (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267#Lmodules/mlv_lite/mlv_lite.cF2996T3033) for more details, and to this article (https://theory.uchicago.edu/%7Eejm/pix/20d/tests/noise/noise-p3.html) for some theory regarding bit depth, noise and posterization.

Impossible (please check the metadata with mlv_dump -v).

Yes, I did notice the ISO suggestions. I have no idea why I thought I was shooting at 60fps. I re-rendered at 30 fps, I also cant hit 60fps when attempting to re-create. but, 8 bit lossless works! :) I was trying to use the lowest bitrate to obtain the highest fps and I thought it was 60, I was mistaken. Thank you for those links.

What would be the reason the DNGs from the April 14th build would not open in After Effects? I'm getting can not parse errors. Worked fine with April 10th. Using Footage.app on mac. Shot 14bit lossless.

What would be the reason the DNGs from the April 14th build would not open in After Effects? I'm getting can not parse errors. Worked fine with April 10th. Using Footage.app on mac. Shot 14bit lossless.

I think I answered my own question. The first frame of the sequence in all the MLVs I shot tonight were black (I assume corrupted) I removed them from the sequences and then AE and Camera Raw could load the sequence.

Have you noticed the ISO suggestions when selecting a lower bit depth from menu? (at the bottom of screen)

Of course i noticed these suggestions, now i read your article i'm able to understand better.

I will do differents tests in low light condition with different range of ISO. eg : 12800 ISO to 800.

So, if i understand well, in this way, the noise is our friend if we need less posterization because the 8bits depth ? So it means that if we film a scene very illuminated outside with the 8bits , we have all interest to use a variable filter ND.

So, if i understand well, in this way, the noise is our friend if we need less posterization because the 8bits depth ?

Correct.

To be effective for reducing posterization, the noise must be applied before the quantization (therefore, analog ISO noise is fine). Adding it in post will not help.

Quote

So it means that if we film a scene very illuminated outside with the 8bits , we have all interest to use a variable filter ND.

I disagree here. Lower bit depths were added to reduce the data rate at higher ISOs (where the lowest bits are mostly noise, and because of them, the image doesn't compress well).

I don't think filming with 8 bits is actually your goal. More likely, I assume you actually want to minimize the data rate. With uncompressed data, it's easy (all frames have the same size: W x H x bpp/8). With lossless compression, you can't choose the compression ratio in advance (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182229#msg182229); this is given - roughly - by the entropy of the image.

These "fake bit depth" settings (actually dividing the signal by a power of 2) are one way to reduce the entropy (by removing the least significant bits, because of integer division). That's a lossy operation (going from 14 bits to any lower value). After this step, compression is lossless (that is, the uncompressed data will be identical to what was fed to the compressor).

(That means, 14-bit lossless will give raw data identical to uncompressed 14-bit (100% identical, not one single bit different, contrary to some earlier report (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182741#msg182741)), while "12-bit lossless" should be interpreted as "14-bit to 12-bit conversion - lossy by definition - followed by lossless compression", so it should have the same number of useful levels as uncompressed 12-bit. Please note the 12-bit lossless is not identical to 12-bit uncompressed - they differ by a constant value and possibly by some round-off error, and the same is true for lower bit depths.)

Back to entropy - increasing the ISO and underexposing (to keep the image brightness unchanged) will increase noise in the image (at all brightness levels). So, what you may gain by reducing the bit depth, you will lose from the added noise. Here's a quick test (static outdoor scene exposed to the right, plain 1080p, compression rates taken from ML menu):

=> there's nothing to gain by using a lower bit depth at a higher ISO + underexposure.

Of course, if your goal is to record at 8 bits, no matter what, then of course, the best thing to do is to increase ISO at 12800 (or at least 6400) for half-decent results. It's also trivial to change the code to get 6 bits, 4 bits or even one single bit, if that's your goal ;)

My advice: first expose properly (ETTR, zebras, raw histogram, whatever you prefer), and afterwards choose the optimal bit depth. Not the other way.

:oNow with these fake bit depth options, this is getting interesting for the SD card camera's :D

@Alex,The red numbers in your bit depth / compression ratio table, do you mean that the serious loss of info/shadow detail is where the red numbers are ?And if I understand correct, you're using digital gain, which cut's of the noisy bit, but the image technically stays 14 bit for the compression algorithm, but because of the noisy bit cut off, the compression rate goes even beyond the 50%.

Correct - the red numbers match my ISO recommendations (https://bitbucket.org/hudson/magic-lantern/commits/2028d73f6f34dcdaaa4c241824e538d0e0e47267#Lmodules/mlv_lite/mlv_lite.cF2996T3033) and are below "gradient 3-bit tonality" from the third figure (labeled Fig. 18) from the noise article (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html). Also added orange = wasteful (there's little or no additional image quality to gain at these settings).

If the theory is correct (which I didn't have the patience to check by pixel peeping), then it's probably best to stay near the transition line to red, on the black side (that is, 11-bit for ISO 100-800, and reduce the bit depth once you need ISO 1600 and beyond).

I believe the recommended settings are visually identical to regular 14-bit (while in 14-bit lossless, the raw data is 100% identical to uncompressed 14-bit). Again, this is not confirmed by pixel-peeping (it's just noise theory). There may be a small color cast in shadows, because the 0.5 LSB offset is not yet applied (but it's easy to fix once @martinhering helps us decode the JPEG stream (http://www.magiclantern.fm/forum/index.php?topic=18949.msg183230#msg183230)).

The estimations were done for 3x3 binning (plain 1080p); the 1:1 crop mode may require slightly different values.

At these (probably optimal) settings, the compression rate appears to be consistent (50%), at least on my test scene (it was some foliage exposed to the right, but didn't save it).

Well, i could not duplicate it. I updated Martin's footage app this morning and they process fine. I do however remember it happened right after i tried to capture 12bit and 8 bit lossless and saw this on my screen. This happens whenever I try to capture any lossless bit rate than other than 14bit lossless. Trying to record like this results in a ton of scrolling errors. (none of which I know what they mean) Have to power off to stop. Pressing x5 crop WILL ALLOW capture however in this mode. 3072x1308, FPS Override 23.976

Testing last build april14th.Tearing issue in 1920x96050p3x3 is gone (I understand because modification of buffering). It is almost perfect for me. I'm really grateful, thanks A1ex!!!... It was a dream some time ago, and now...Thank you very very much...

Now the only downside is recording time with hi isos in 14bit lossless and incorrect framing, but I can live with that.

Great job for the low bit depth it opens great perspective to play around the 3.5k.

Here is my bug encouters for low bit depth 8,9,10,11:

- the expo is darken in the realtime preview and hdmi out. realtimre preview for 8 bit totally unusable (this is bas for focusing) framing mode is working better but focus is still not possible from there.

- 1:1 Tall have the lower of the image not displayed correctly like mentionned for slow mode

- non working on 3K mode and higher

Can we expect a mlv sound coming back? post synchro is so burden with just the beep :P

Have you tried running firmware update again from Canon menu and restart camera?

If no messages prompts up afterwards then you haven't been affected from that April 4th null pointer bug.

Thank you, Deafeye. I did as you said, update from Canon and restart. But could not see any test being done. Thought I saw a screenshot from someone doing the test and there was like this 0-100 Counter, maybe I am mixing it up with something else.

Its just a pain for me downloading builds cause I am on Satellite internet, I get 4-5 kb/s on a good day and then there are constant dropouts which which require me to start the download all over again... Just loading this forum page takes forever, and its only text more or less..

So its safe to assume that I need to download apr10 build because it has the Rom Test ?

I get 4-5 kb/s on a good day and then there are constant dropouts which which require me to start the download all over again... Just loading this forum page takes forever, and its only text more or less..

Here's what helped me on a similar connection, besides obvious things such as disabling images or using a text-mode browser (mobile connection with poor signal, far away from the city, 4K/s at best):

http://lartc.org/wondershaper/

There are probably equivalents for other operating systems. The idea is to reduce download speed so you can load other pages (or use a chat client) while a long download is running.

Having a remote server accessible with VNC, with a very low image quality setting, can be also helpful (as loading a JPEG preview of a webpage can often have much lower bandwidth than its HTML+CSS+JS+images).

For a download with frequent dropouts, just leave a "wget" command open in background (as it doesn't give up easily, unlike most browsers).

There seems to be lots of interest getting crop_rec on steroids working on other cameras.

The EOSM can already do basic 3x3 720p crop_rec and I got it working a while ago on the 700D but was trying to get the other options available on the 5D3 working before doing a pull request. I haven't gotten very far but since other users are also experimenting with adtg_gui, dm-spy and other ML tools to get 4K working on their cameras, maybe we should learn to crawl before attempting to fly so I posted a pull request for limited support of the crop_rec module for the 700D.

On the 700D, I finally found the reg needed to fix the darkened lower half of the image when trying for higher resolution than 2048x1152.It was ADTG2[82B6].Best fps I can get without a freeze at this res is 10.639fps.Don't mind the crappy image quality, room was dark and was to lazy to turn on the light for testing, LOL, was shooting 3200ISO.(https://s4.postimg.cc/xkgbmbgz1/4kfix.jpg)

Sorry for the double post!!! But I Just tested 4-14 build 12 bit 3072 2.35:1 at 23.976 ISO 3200 with the old 5x zoom method and it seemed like it was endless. Over 20 secs for sure!!!! DNGS are clean and processed fine and all worked in resolve. So the this will work for sure!!!! Just need to get the crop rec working. Way easier said then done no doubt. this is crazy tho. 3k 12bit lossless 24p on a 2012 5D

Admiration for many reasonsFor revolutionary development of out of this world features , for so many camerasBaby sitting the forums with so many users of so many experiences and variation of needs and knowledgeI surrender, I could not do that

Can anyone post their advanced settings on the crop_rec on 5D3123 so that I can get the crop from the center? Currently it looks the crop start from the extreme right side of the sensor (3K).

Thanks for all your hero members for the advancement.

Don't use 5x mode like before. If your in regular live view you will get proper framing. It's a bit faster than before also! If you go to 5x mode the framing is off. Took me a bit to figure this out. HDMI preview isn't working yet in this mode but it's amazing none the less. I'm standing by until it does so I can use the 5D on my Movi again.

Can ANYONE point where is a precise step by step instruction of using mlv_dump?

Yes, I know - noob! how da hell ... and so

But I'm using ML almost since it's first build, but thankfully chmee wrote his tool very soon...I was always using RAW2CDNG, but now I have to deal with this rocket science - command line stuff (( And there's no clear and simple TUTORIAL here on forum or youtube video tutorial. Everything I've found is either outdated or people being sarcastic (not helping) and in most cases for MAC (why mlv_dump discussions are always mac oriented?)

A1ex is so helping and responsive. I don't get it why didn't he created a normal software with UI ...or clear and simple tutorial or list of commands for us to just copy/paste? Why not implement at least base and commonly used commands into mlv_dump somehow? Or enable drag'n'drop functionality? anything simple :-\

who can help - what should I do with mlv_dump on PC to simply convert MLV to DNG?

Hey guys.. i'm kinda new to ML.. i've loaded this experimental build and started trying to record in UHD. I've loaded the Rec_crop module and tried the UHD setting. Now the 1080p/50fps setting works sortof(it's not actually "slow motion" since windows still recognizes it as 29fps) the 3K/UHD/4K setting won't work. There's a weird purple/pink artifact going on when i try to film.

Would that mean i have to record in RAW? If so, what are the appropiate modules i have to activate to make it work?

A1ex is so helping and responsive. I don't get it why didn't he created a normal software with UI ...or clear and simple tutorial or list of commands for us to just copy/paste? Why not implement at least base and commonly used commands into mlv_dump somehow? Or enable drag'n'drop functionality? anything simple :-\

pay him 4k€ a month and he will probably do this additional work on saturday and sunday night - the only time when he usually sleeps.

ah and i forgot to mention that i implemented mlv_dump as reference implementation, tool for ML devs, users on nerd level 100 and fallback solution.*if* there is a proper GUI frontend - go for it, else mlv_dump has to do the job :)

My opinion: Get rid of those experimental builds. It invites users not ready for experiments on this level.Those wanting to go there may a) have an environment to make their own buildsorb) rely on a helping hand building it for them on demand.

My opinion: Get rid of those experimental builds. It invites users not ready for experiments on this level.Those wanting to go there may a) have an environment to make their own buildsorb) rely on a helping hand building it for them on demand.

Hmmm I not sure I agree when it comes to experimental builds. Although I guess I don't quite get what you mean by "rely on a helping hand building it for them on demand."

On one hand, the admins hold our hands and babysit us while testing. Time consuming to say the least.On the other hand, some of us who have been using ML for years get a chance to learn about the inner workings of ML and learn how to become better testers. Also, when it comes to working out the bugs, us inexperience ones are the best at breaking things because we don't listen.

However I do feel that people should not get upset or wine or cry or point fingers when they can't figure something out. I think people need to be more aware that this is all volunteer work. No pay for hours of endless coding. One should not complain especially when one has no clue what is involved with coding.

But that is my humble "I don't even know what I'm talking about but should since I have been using ML for so long and I am being grossly off topic and I should know better about that too run on quote" opinion. An opinion to which I feel I don't even deserve to give. Compared to Walter Schulz I am a dumb dumb head.

Experimental builds are the soul of magic lantern right now, it allows people to test lit the latest and give very valuable feedback, @walter if someone dont like them, or are scared it may harm they are free not to use them, but i for one love them.

The experimental version of magic lantern is on the cutting edge, frustratingly cutting edge. It’s part of the struggle for the code-blind mortals who’s pulling their hairs out and ask the stupidest questions on the board. The struggles make magic lantern exhilarating, and inspires participants to make better picture, better videos. Most of us do not post anything on the board just for the fear of sounding really really….. stupid. However, we do really appreciate the “geniuses” and generosity of those immortals such as Alex. Please bear with us.By the way, I finally get UHD to work as well as 1080p @45fps. The DNGs were generated with Rawflow (page 13 of this board). Somehow, the first frame is always bad and the rest are splendid.Thanks for all the immortals on the board and hope you can improve magic lantern further and let the mortals to try the experimental versions. The ‘code blind” mortals can’t contribute much to the development of magic lantern. But we will do more volunteers to places such as “soup kitchen” to feed the homeless as our appreciation.

My opinion: Get rid of those experimental builds. It invites users not ready for experiments on this level.

I was posting test builds for the 12-bit (and 10-bit) RAW video development discussion (http://www.magiclantern.fm/forum/index.php?topic=5601.0) before the Experiments (https://builds.magiclantern.fm/experiments.html) page went live so I feel partially responsible for rolling that snowball.

It would be great if more users would set up development environments or at least take a good look at the code. Even if you can't compile or understand the code, there are some great comments in there that explains what the developers are doing and perhaps more importantly, what areas still need some more testing. For example, in the crop_rec.c file of the crop_rec_4k branch (to stay on topic) you'll find this:

So instead of simply reporting corrupt frames, indicate the resolution where you start getting corrupt frames and how far you had to back off to get consistently clean results. If you can compile, try pushing it a little further and report how far you can go before it breaks.

For this particular snippet, vertical resolution is actually adjustable from the crop_rec submenu (no need to compile). I don't remember anyone trying to change it, even though I've mentioned (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182320#msg182320) it a (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183060#msg183060) few (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183112#msg183112) times (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183115#msg183115) (including first post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182052#msg182052)).

I agree with your 'set up a development environment', and, as you know, I wish to do this.

However, and this is a big however, not all of us are competent to do that without very clear and simple instructions.

I have tried to set up such an environment, and although the instructions I was following were good, they still assumed some knowledge I don't have. I kept getting error messages and gave up :-(

From my perspective it would be good to reset/start an ML page specifically directed at getting your own compile environment up and running, and keep Win and Mac instructions separate. Also this page should be written from a nil knowledge perspective, i.e. for 'idiots' like me.

Of course, I appreciate I'm asking a lot, i.e. someone who knows what they are doing sitting down and writing that page.

But until I can follow a simple workflow for setting up a compile environment, I flummoxed.

Just start out with what´s not working in one of the compiling threads and take it from there. It´s impossible to account for every single thing that comes up when compiling although dfort´s efforts made it much easier, especially for mac users. I´m pretty confident the solution won´t be far when out in the open.Here are two threads if anyone is interested. There are more efforts and threads out there.Machttp://www.magiclantern.fm/forum/index.php?topic=16012.0Windowshttp://www.magiclantern.fm/forum/index.php?topic=15894.0

For this particular snippet, vertical resolution is actually adjustable from the crop_rec submenu (no need to compile). I don't remember anyone trying to change it, even though I've mentioned (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182320#msg182320) it a (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183060#msg183060) few (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183112#msg183112) times (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183115#msg183115) (including first post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182052#msg182052)).

@a1ex I have to say, I was super nervous about changing those settings since I didn't do enough research on what they mean but Ill see if I can figure it out. Has anyone one tried tweaking those? What did you find?

Simple example: if you often get corrupted frames in 1080p48, dial 1040 (or some other value lower than 1080) in the Target YRES field. Then refresh the LiveView manually (e.g. press MENU twice) and the resolution in RAW video menu should update.

Simple example: if you often get corrupted frames in 1080p48, dial 1040 (or some other value lower than 1080) in the Target YRES field. Then refresh the LiveView manually (e.g. press MENU twice) and the resolution in RAW video menu should update.

Set 0 to disable the override and go back to default value.

@a1ex Ohhh cool! Do you have to match the value in the MLV Lite options?

New build:- centered x5 zoom is back (3.5K with 8...12-bit lossless compression)- disabled 8...12-bit lossless in incompatible modes (sorry, still couldn't figure out how to fix them)- for bit depths between 8 and 11, the choice is now automatic, depending on ISO (see this post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231))- fixed silent pics (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) (a bit sad to see them broken for 3 weeks with no bugs reported)- minor UI fine-tunings.

- fixed silent pics (https://bitbucket.org/hudson/magic-lantern/commits/a9c8ad412af5c282bbad6889ce5f65d974ca3ada) (a bit sad to see them broken for 3 weeks with no bugs reported)

Sorry I noticed that while trying to get lossless compression working on other cameras but didn't create a proper bug report--though I did point it out. I'm working on the compressed_raw branch but maybe the crop_rec_4k branch has the latest changes? Noted that lossless.c and silent.c aren't in sync (https://bitbucket.org/hudson/magic-lantern/branches/compare/crop_rec_4k%0Dcompressed_raw#chg-modules/silent/lossless.c) between these two branches.

I have the null pointer bug warning coming up with April 21 build. Message says to contact ML developers on IRC Magic Lantern freenode. Tried but not sure I was at correct place to do this - Alex can you supply URL please. Unfortunately don't have old logs.

I've been experimenting with the 3K crop and 1080p slow motion, etc from the experimental build. Amazing stuff. It's hard to say, of course, how long it will be before usability is nailed down. Wondering if there might be a shorter term move to combine the "simple" 1080p24/30 lossless recording that's been developed along with the MLV Sound module? I would love to incorporate something like that to my everyday run and gun interviews and b-roll without the occasional random breaks in recording and the extra large files that quickly fill up my cards.

(this was posted in reverse engineering but it was recommended that I delete it and post here)

Scene is well exposed fine detail @ ISO 32003072 x 1308 (or what ever 2.35:1) Is working extremely well in 5x zoom 12 bit lossless fps override 24.006 and framing preview. I disable the crop rec mod while recording in 5x (not sure if I should use crop rec and 5x at the same time) converting DNGs as I type. Edit: Out of about 10,000 frames not one bad frame!!!!!!

@hjfilmspeed: the centered x5 preset from crop_rec is for x5 zoom; all the others are for non-zoomed modes. The help at the bottom should tell you. You can also choose 23.976 for FPS override.

@csound: type IRC in your browser's search box (CTRL-F) ;)

@dfort: yes, I've looked into it because of your report, but it was my explicit request to try the bad commit, not a regular user trying it for e.g. timelapse or whatever. Also, this branch has a lot of stuff that can be broken into smaller branches and back-ported to all other models, in the main builds (already started with this PR (https://bitbucket.org/hudson/magic-lantern/pull-requests/825/prevent-canon-settings-from-being-saved/diff)).

@lostfeliz: I was also thinking to add mlv_sound compatibility to mlv_lite, just didn't look into it yet (since, for my own use, only the full-res and 4K half-FPS modes are interesting - for extracting still frames from a burst sequence, or for timelapse). That's the reason you see weird changes such as single-buffering, ability to record with only 2 buffers, half-shutter trigger etc.

@hjfilmspeed: the centered x5 preset from crop_rec is for x5 zoom; all the others are for non-zoomed modes. The help at the bottom should tell you. You can also choose 23.976 for FPS override.

@a1ex OHHHHHH I didn't even see that option! That mode also lets you use 12bit and at 3200 iso im getting what seems to be continuous 3072x1308 24 FPS!!!! And its centered!!!!! and you can use the zoomed real time preview if you don't want to use the framing preview!!! No crashes so far. The only thing I noticed is the shutter speed in at the bottom didn't seem to be registering the right number. Unless something else is going on here. The cameras shutter speed was 1/50 but when zoomed + fps override it showed 1/27 or 1/37 or something like that. But wow .... This is impressive!!!! THANK YOU FOR THIS!!!!This is much more exciting then the C- log announcement.

I also started seeing these stuck pixels pop up in 1:1 5x liveview real time iso3200 1/50 that wouldn't be there in normal 5x zoom real time. I think its Just in the liveview. I didn't try to see if it would appear in RAW file. Focus box was in was centered in both cases. I feel like this has been covered and I appollowgize if it was.

@a1ex Tested the latest build it's solid.. Only got one crash when I was using slowmo 45p / 48p, but when I switch to 50p / 60p mode it seems to work fine. What's the difference between those two slowmo modes other than fps because on the 50p / 60p mode I was able to record @ 48p?

@hjfilmspeed Awesome didn't think the 5D III could be sharper than the Mark iv in 3k, and definitely the motion in 3K is very cinematic I posted about that. All I'm gonna say is a1ex got the keys.. keys.. keys.. Can you share a short clip of each camera? Did you zoom in 200% - 400% to pixel peep.

@goldenchild9to5 I'll post my quick test but before I hard conclude this I'll have to give the 5div a more suitable test. I zoomed in and its really hard to to make a conclusion here but in both cases I gave the 5div the advantage especially with autofocus. First 2 shots are 1080 shots 3 and 4 are the higher resolution. I tried to grade the RAW fairly. See if you can tell what's what. I like the RAW better for sure. If you download it it's a Cineform AVI in UHD Also keep in mind I can only export UHD from Resolve. https://vimeo.com/214307580

The cameras shutter speed was 1/50 but when zoomed + fps override it showed 1/27 or 1/37 or something like that.

Solved, it was just an oversight from my side.

Also fine-tuned the preview behavior: you can use a long half-shutter press to change between framing and real-time (e.g. if you normally use the centered real-time preview in x5 zoom, but you also want to check the framing every now and then, just press half-shutter for 1 second or so).

Also fine-tuned the preview behavior: you can use a long half-shutter press to change between framing and real-time (e.g. if you normally use the centered real-time preview in x5 zoom, but you also want to check the framing every now and then, just press half-shutter for 1 second or so).

This is incredible! Soooo impressed!

Also I just replaced my test with a much better one

https://vimeo.com/214307580This means Either Canons 5Div 4K is actually 3Kor That we get to call 5D3 ML 3K ... 4K4 Clips2 are 5div 1/100 f11 ISO 400 Sigma Art 35mm 4k2 are 5Diii 1/50 f11 ISO 400 Sigma Art 35mm 3k ML14bit lossless using 4-21-2017 4K Crop Rec build(I underexposed the 5Div because I was clipping highlights at 1/50 shutter)Both rendered to Cineform UHD and both were precisely focussed on the crosshatch on the color chart. Can you tell which is which.Download and zoom in. It's a UHD Cineform AVI 5Diii was quickly graded to match Canon's neutral picture style of the 5div

Sorry to be a party pooper but can anyone else confirm that this is reproducible (it seems to be on my end w build 2017-04-22 for 5D3.123) if one were to record in 3504x1400 resolution (and only in that resolution which is 2.50:1)?

It seems to produce a pattern of corruption after each 36th DNG frame from 14-bit lossless shot in 24p.

Settings used in ML:(https://c1.staticflickr.com/3/2834/33393407053_c0143f495d.jpg) (https://flic.kr/p/SSRWvt)

SO far in other resolutions seems to be getting better results other than the random first DNG's being either black (blank) or half screen corruptions, etc from each certain MLV's. Gotta troubleshoot some more on this to narrow down the culprit. Don't think it's random at all.

I was able to replicate the 19 FPS bug in the menu I reported a while baclk. If I forget to turn FPS Override off, turn on Crop Mode module for UHD recording, enter canon menu then go back into ML menu, when I turn off FPS Override it defaults back to 19 FPS. If FPS Override is off when I turn on Crop Mode, everything is fine.

Sorry to be a party pooper but can anyone else confirm that this is reproducible (it seems to be on my end w build 2017-04-22 for 5D3.123) if one were to record in 3504x1400 resolution (and only in that resolution which is 2.50:1)?

Narrowed down and solved; excellent catch @DeafEyeJedi!

That's how a proper bug report looks like - it contains all the details to allow others to reproduce it. This bug was quite hard to trigger, given all those reports with no corrupted frames.

Also got 4096x3072 working at 12 and 12.5 FPS (24/25 in Canon menu) and did some more fixes for very large resolutions. Even if you only get a few frames, I still find these modes useful with half-shutter trigger and pre-recording (e.g. for taking photos of really unpredictable subjects).

@dfort: yes, I've looked into it because of your report, but it was my explicit request to try the bad commit, not a regular user trying it for e.g. timelapse or whatever. Also, this branch has a lot of stuff that can be broken into smaller branches and back-ported to all other models, in the main builds (already started with this PR).

Oops, sorry. I bit off more than I could chew when I came across that bug and it was just easier for me to switch to an earlier (smaller) branch. I'd like to get lossless compression working on other cameras because that seems doable while 4k on an EOSM is pie in the sky. Though I wouldn't rule it out considering how quickly ML development has been progressing lately.

Thanks for the wonderful work. I Got some problem when update to the new buuld, don't know how to fix it.

My camera is 5D3, 1.2.3

I have installed the Apr-04 build before, it works, and then update to the newest Apr-23 build, then I got this scene on top.(http://i.imgur.com/czeNgzz.jpg)I tried to Uninstall, and format CF card, Install again, disable all modules, but got the same scene. then I tried degrade to the earlier build, the same, But the Apr-04 and Apr-06 build works fine.

I'm getting frame and block size mismatch errors. I saw a few other people were getting various versions of this error as well but no responses with a possible solution. I know it's related to the 14 bit - lossless format as I don't get the message when I turn lossless off.

@lanhl Follow what it says on the screen. There was a dangerous bug in April 04 build so DON'T go back to it, and the new ones are detecting if that bug has affected your camera. It's Magic Lantern looking out for you ;)

I used mlv_dump in mac for creating dngs. but unable to open them in After Effects CC 2017. Photoshop opens files with no problem. but AE gives me this error: "after effects error: the file formay module not parse that file. (45::35)"

@hjfilmspeed Check it out Zoomed in to 200% & 400% Scenes 1 & 3 is a tiny tiny bit sharper than Scenes 2 & 4. But here is the thing the tree behind the color checker seem a bit sharper in Scenes 2 & 4.

1 and 3 are mark iiiHmmm Ill double check my lens details to see if f-stop was different. It could be possible that focal planes where not exact. I focused both on the crosshatch with autofocus. but in any event at 400% we should be seeing a HUGE difference in sharpness here. 5dIv has 1000 more pixels. Any way I don't want to go more off topic then I have.

On 12 April 2017 I took several shots.I was recording with that days latest build, 1920x800 60FPS 14bit lossless with 5dmk3.On some of the shots I took, on the top of the frame there is a horizontal sharp edge that flickers, about 100 pixels away from top

Here is a linkhttps://drive.google.com/open?id=0B-d8ARtc7xwWT3BwbkdQYjdaLUE

That's how a proper bug report looks like - it contains all the details to allow others to reproduce it. This bug was quite hard to trigger, given all those reports with no corrupted frames.

Great to hear. Thanks for the acknowledgement!

Quote from: a1ex

Also got 4096x3072 working at 12 and 12.5 FPS (24/25 in Canon menu) and did some more fixes for very large resolutions. Even if you only get a few frames, I still find these modes useful with half-shutter trigger and pre-recording (e.g. for taking photos of really unpredictable subjects).

Oh absolutely this will be useful. Especially for birding, sports photography & whatnot. This is incredible and should be a bit more forgiving than Canon's stock full burst mode which is only 6 frames per sec for 5D3. :P

1 and 3 are mark iiiHmmm Ill double check my lens details to see if f-stop was different. It could be possible that focal planes where not exact. I focused both on the crosshatch with autofocus. but in any event at 400% we should be seeing a HUGE difference in sharpness here. 5dIv has 1000 more pixels. Any way I don't want to go more off topic then I have.

For those who havant seen them up-close w comparisons yet in between 200% & 400% (Thanks @hjfilmspeed for sharing them!):

Thanks @hjfilmspeed -- fixed! (It's even more flabbergasting to know that the 5D3 still looks sharper even when upscaled to 4K) :o

DeafEyeJedi Thank you for posting those crops!Actually, you might as well call the the 5DIII 3K - 4K. Why not canon apparently seems to think it is okay. I thought to myself, maybe its the lens.. then I was like no way, the Sigma Art prime is design to resolve way high resolution. It could be the 422 vs the .... 444 or what ever MLV 14Bit lossless is.Regardless, this build gets better every time. Can't wait to see what the future holds!

Oh I meant to ask this, should we be using 123 vs 113 for 5d3? It was my understanding that 113 was fine unless you need hdmi features and AF @ f8. Is this still true?Or should we be testing 123?

Hey all, how are you guys addressing the Vertical Lines or other pixel issues with these new builds? I'm on OSX. Martin Hering had created a great mlv to cng converter app called Footage that addressed them but he seems to be letting it time bomb in the 15th of May to pursue a revenue generating path. So now we're back to square one.

MLVDump does not fix them. Is there another app that i'm not aware of that does?

Also has anyone tried these settings? 4/23/2017 buildCrop Rec with 5x zoom 3.5k enabled And 12bit lossless with 3072 2.35:1 ?It's very stable to me! Continuous with a good card! And you can hit the DOF preview button or half shutter to zoom in with real time to focus! I saw this mentioned but didn't try it yet. This is awesome!I feel like real time framing preview of 3k is so close now! (Coding wise it's probably not)

Care to share some original short samples @giarcpnw? (use @Danne's cr2hdr.app if you haven't already)

cr2hdr didn't seem to fix when i tried last week. unless it's been upgraded since.

The are DNGs from MLVdump, you can see the banding/lines in the mids. They get worse once you adjust exposure and sharpness. These were shot 14bit lossless with the April 10th build I believe. My understanding is this is inherent in the raw data and the converters are coded to adjust for it. I know there was an early post in this thread that said most of the apps or command line converters had not been adjusted to address this with the new 14bit 4k build's mlvs. Just wondering if they had yet or I missed one. Martin added it to his at my request last week, but...

MLVDump does not fix them [vertical lines]. Is there another app that i'm not aware of that does?

I've explained here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg183653#msg183653) what needs to be done for other converters (including mlv_dump). I'm not really familiar with compressed formats, and there are people in the ML community who can do this faster than me (hint, hint).

I've explained here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg183653#msg183653) what needs to be done for other converters (including mlv_dump). I'm not really familiar with compressed formats, and there are people in the ML community who can do this faster than me (hint, hint).

If only i had studied computer science in college instead of theater, I might be able to fix this myself. Instead, I'll have to recite shakespeare to myself while I wait for someone else with programming knowledge.

Can someone please point me to information on procedure to fix null pointer bug from April 4 build. Screen alert on camera indicates to prepare logs and go to freenode on IRC - what happens from there? Sorry, unfamiliar with all this.

Just tried the new Davinci Resolve 14 Beta with 5D III 3K footage man this new update is smooth. I've noticed as well that It might be processing dng's better, seems to resolve images detail a lot better. If anybody else used it let me know if you are experiencing the same thing.

Just tried the new Davinci Resolve 14 Beta with 5D III 3K footage man this new update is smooth. I've noticed as well that It might be processing dng's better, seems to resolve images detail a lot better. If anybody else used it let me know if you are experiencing the same thing.

Same experience here - realtime playback on my 3k 2.39:1 14bit CDNG files, up to a few nodes. Extremely pleased with the new Resolve so far.

Also got 4096x3072 working at 12 and 12.5 FPS (24/25 in Canon menu) and did some more fixes for very large resolutions. Even if you only get a few frames, I still find these modes useful with half-shutter trigger and pre-recording (e.g. for taking photos of really unpredictable subjects).

Quote from: DeafEyeJediOh absolutely this will be useful. Especially for birding, sports photography & whatnot. This is incredible and should be a bit more forgiving than Canon's stock full burst mode which is only 6 frames per sec for 5D3. :P

A dumb question maybe... but... someone are getting realtime preview in this mode? How can you focus and framing for sports and birds?... I think I'm missing something...

I'm taking a look to the Starting Point... and I don't know what to do... I have no knoweledge in any coding languaje. For me is like cheneese :( :-[ I'm sorry but if I can do something more just let me know.

The difficult part is simply understanding what Canon code does (and what we have to change to fix the preview).

Those numbers from the logs and adtg_gui are like Chinese to all of us; the trick is to fiddle with them until a tiny part of them starts to make some sort of sense. That's how we figured out how to reconfigure the sensor to 4K in the first place ;)

Some useful tricks: comparing all those numbers in different video modes (adtg_gui has a simple register diff tool), recognizing values that could look like resolutions, logging the I/O activity in QEMU, or analyzing the still photo mode first (it's simpler).

And, of course, don't give up after reading a single post. Make yourself comfortable with what's happening in the Reverse Engineering area. Some relevant threads are linked in the first post; make sure you can follow them.

Of course, you will need some basic programming knowledge.

Bottom line: the real-time preview is not implemented because... I have little or no idea about how Canon configures the image processing path.

Sorry don't be pissed at me, but what is the best solution to convert 14 bits lossless .mlv files?I've red we can use cr2hdr.app_4K, but i don't seem to find a link to it, only the cr2hdr.app version. Is this application compatible with all the .mlv flavours?Is there better way to do so? I've always used MLVMystic which is simple and elegant to use…

@A1lexThank you very much for the explanation!!!... I feel really dumb but I guess not everyone will know about everything. Coding is a unknown world for me. The most obvious things for you are absolutely unknown for me. :-[I would like to do more than only download the builds, but outside testing that builds I don't know what can I do...

Sorry don't be pissed at me, but what is the best solution to convert 14 bits lossless .mlv files?I've red we can use cr2hdr.app_4K, but i don't seem to find a link to it, only the cr2hdr.app version. Is this application compatible with all the .mlv flavours?Is there better way to do so? I've always used MLVMystic which is simple and elegant to use…

Thanks,

S/.

cr2hdr has been updated, so no need for the '4k' version anymore. Just grab the latest from here - https://bitbucket.org/Dannephoto/cr2hdr/downloads/

1) for decode dng files of this exp.release , must be used only with last release of mlv_dump (line command without interface) ? because I try anothers programs, but are not compatible, only pink and black dng.

2) any one can please tell me, the right setting for recording in 1920 50p ? must be select also in menu canon 1280x50p or can be 1920 x 25 all-i) ? must be set FPS over to 50 ?

I've tested 1920x960 50p 14 bits lossless, convert the .mlv file with mlv_dump, all .dng are black for the moment, don't know why. I had no error message during the recording and stopped it myself.I've also tried 4K at 24p, but I haven't found the way to get the 4096 x 1440 ratio it is written everywhere?In 4K, preview isn't working, right?

Thanks and many thanks to all the team which is doing so much work to fill the gap Canon has let behind.I'm currently finishing my second short with the help of Magic Lantern and it is great to be able to get such video quality!

I've tested 1920x960 50p 14 bits lossless, convert the .mlv file with mlv_dump, all .dng are black for the moment, don't know why. I had no error message during the recording and stopped it myself.I've also tried 4K at 24p, but I haven't found the way to get the 4096 x 1440 ratio it is written everywhere?In 4K, preview isn't working, right?

Thanks and many thanks to all the team which is doing so much work to fill the gap Canon has let behind.I'm currently finishing my second short with the help of Magic Lantern and it is great to be able to get such video quality!

Do we have to do something when the Memory patches is turned red, red on Code:FF28CC3C?It seems to have appear after my first test with the Apr23.5D3113 build when testing 1920x960 50p, I had the screen filled with error alerts, then this...

Can someone write tutorial about how to rec on these modes? Why do I get this http://joxi.ru/YmEqBpZF06jjnm.jpg instead of DNG? What mlv_dump for?

Also I don't get how card spanning is working (it seems it doesn't) for when I enable this mode then at 3200x1200 it record 4 seconds (I checked it with various superfast CF-cards), but with only one CF card it also rec 4 seconds only!

Can someone write tutorial about how to rec on these modes? Why do I get this http://joxi.ru/YmEqBpZF06jjnm.jpg instead of DNG? What mlv_dump for?

mlv_dump is lovely!! make sure you use the mlv_dump in experiment page. otherwise you end up with noisy pink DNGs. been there before. or you can try the amazing cr2hdr.app. (note: dont judge the frames by their thumbnails. open them in photoshop to see if they are converted right. and not the first frame also!)I see you have opened the first frame which is usually pink and corrupted with this mode. have you tried other frames too?!

this is MLV_Lite with this build which is different than the old MLV. MLVRawviewer and raw2cdng can't open it or they deliver pink DNGs instead.

@DeafEyeJedi has the magic settings here. I can verify most of this. But you need to make sure ML is on your SD card and you are recording to your CF card that is formatted exfat from your computer. DeafEyeJedi settings are exactly what I have been using. It you want a smidge extra headroom you can set it to 2.35:1 which brings your vertical resolution down a tiny bit ... I think.

Edit: actually... I just got these settings to spike! ISO 100 at some very bright trees! It went from like 89 MBs to 114 MBs I'll upload a dng

At the 11-8 bit Lossless with 3008x1280 I was able to get the same scene down to 90ish A1ex was right! (Not hard to believe) that you can get seemingly continuous settings to spike high! Very high

@Ali Oliyamlv_dump is lovely!! make sure you use the mlv_dump in experiment page. otherwise you end up with noisy pink DNGs. been there before. or you can try the amazing cr2hdr.app. (note: dont judge the frames by their thumbnails. open them in photoshop to see if they are converted right. and not the first frame also!)I see you have opened the first frame which is usually pink and corrupted with this mode. have you tried other frames too?!

this is MLV_Lite with this build which is different than the old MLV. MLVRawviewer and raw2cdng can't open it or they deliver pink DNGs instead.

But where is this experiment page? I just download build pack and mlv_dump.exe but what should I do with this exe file? I actually can't understand why it so difficult to write tutorial about how to use these.

http://www.magiclantern.fm/forum/index.php?topic=19300.msg183363#msg183363A user makes fun of this site and calls it "shark bay". If that is true you are pretty much bathing in Piranha infested waters right now.

But where is this experiment page? I just download build pack and mlv_dump.exe but what should I do with this exe file? I actually can't understand why it so difficult to write tutorial about how to use these

@SavelyWhat modes exactly? This tutorial would be a lot to go through. I assume you read DeafEyeJedi settings and you are up to speed on the new build. I have some that would offer even more stability depending on what you are filming. 3008x1280 11-8 bit lossless MLV Lite with preview set to auto.Crop rec mode set to 5x zoom 3.5 k modeAlso it looks like you are having post processing issues. If so it's easy.For windows: 1) Download RawFlow v0.2 http://www.magiclantern.fm/forum/index.php?topic=13338.02) Download new MLV_dump https://builds.magiclantern.fm/experiments.html3) Open up the Rawflow folder and delete current the MLV_Dump that came with Rawflow and replace it with the new MLV_dump you downloaded from here https://builds.magiclantern.fm/experiments.html leave this folder open for now.4.) download your MLVs off your card to a new folder on your computer.5.) Open up the RAWFlow folder if you closed it.6.) Select all your MLVs and drag and drop them right on the Rawflow.exe application icon in the Rawflow folder (note you do not have to open the exe or run it. It will do that on its own after you drag and drop mlvs on it)7.) The app will open a command prompt window. Let it do its thing and it will place new folders in the same folder as you MLVs with DNGs for you to play with.

Hope this helps and hope this doesn't break your camera or computer. I think this workflow is similar on MAC but not sure.

@DeafEyeJedi sorry to double post but it appears that even 3072 x 1308 12bit lossless 23.976 fps ISO 100 under 5x zoom can cause a spike in data rate. Like I think it said 117 MB/s@A1ex told me that if you try hard enough you can find a scene that can do this. It appears that it's not just high iso that can cause a data rate spike. Even at iso 100 12bit with a brightly exposed detailed scene that data rate can shoot through the roof. I assume if the details are bright enough to stay out of compression then this can happen. That's my guess anyway. I was able to get the scene back to 90MBs by using 11-8 bit at 3008x1280 here is the DNG that went off the charts.https://www.dropbox.com/sh/8lmopv8z7p2i91f/AAA4saYTmLE7rFQ3t7nSUBCaa?dl=0

1920x1080 14bit lossless 24p was still okay. Not even close to danger zone.

EDIT!!!!!!! I actually tried to rec this scene at 1920x1080 14bit lossless 24p even tho MB/s was good but it gave me an ML assertML ASSERT:0at mlv_lite.c:2344 (compress_task), task compress_tasklv:1 mode:3

Then I notice that the compression ration was at 106% or something like that. @a1ex I found the scene you were talking about!all frames are here:https://www.dropbox.com/sh/8lmopv8z7p2i91f/AAA4saYTmLE7rFQ3t7nSUBCaa?dl=0

It will be nice to have an option for manual bit lossless setting in order to reduce data rate.

Not after seeing samples like (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183187#msg183187) these (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183208#msg183208), but feel free to prove me wrong (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231).

The automatic choice is on the gradient 3-bit tonality (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html) border (any lower bit depth would result in obvious posterization).

Thanls for your really important info about! Cf is a CB 128 GB x1066 tested ,formated exFAT, and seem to work very well with normal raw 14 bit 1920 25p + audio , no stops.In the setting you talk I become automatic stops , sometime with few beeps and error....BUT I think is some problem her of many another people....also if you read the setting you do, work on continued recording.... iso and LIGHT of sky example...give a big input data that overwriting buffer and STOP the recording.With 1920x960 50 P 14, 12 LOSS bit....begine well and suddenly break the recoding, why ? I just do few test before....if I begin recordin in the dark area/room ALL ok....when I turn to the windows light....BREAK recording, so also if was around 60/70 Mb...the light jump over 100.... this do in all the LOSS bit.

i think this is the bottleneck which does not hold recording on some resolution, though not exaggerated and low frames. or not?

Not after seeing samples like (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183187#msg183187) these (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183208#msg183208), but feel free to prove me wrong (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183231#msg183231).

The automatic choice is on the gradient 3-bit tonality (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html) border (any lower bit depth would result in obvious posterization).

Yes, I understand. But this build is experimental one. A person who is going to use it must understand or try to understand what's going on. But it's not easy to set a proper exposure and sometimes the exposure is set with some shift intentionally. In these cases the relation between ISO and bit depth is not strictly defined. So manual bit depth option will be usefull.BTW, is there a probability to get 12-8bit lossless for other modes, espesially 45/50/60fps?

In these cases the relation between ISO and bit depth is not strictly defined. So manual bit depth option will be usefull.

Please show me one (1) non-trivial example where reducing the bit depth below the recommended value is useful.

Quote

BTW, is there a probability to get 12-8bit lossless for other modes, espesially 45/50/60fps?

Probability to get lower bit depths in other modes is zero with current method (I'm not aware of any situation when it could work by chance, no matter how many times you would try).

If you ask about future possibilities, I'm unable to answer (because it's something that, at the moment of writing, is beyond my understanding). Whether I'll be able to figure it out tomorrow, or next month, or within the next 10 years, I have absolutely no idea. Same for real-time previews with 4K or other similar questions.

If you ask whether the hardware can do this, I'm pretty sure it can. What I don't know is how to program it to do so. You (http://www.magiclantern.fm/forum/index.php?topic=14656.0) can (http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force) help (http://www.magiclantern.fm/forum/index.php?topic=2864.125) with (http://www.magiclantern.fm/forum/index.php?topic=6751.msg71720#msg71720) that (http://www.magiclantern.fm/forum/index.php?topic=10111.0).

If you ask whether the hardware can do this, I'm pretty sure it can. What I don't know is how to program it to do so. You (http://www.magiclantern.fm/forum/index.php?topic=14656.0) can (http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force) help (http://www.magiclantern.fm/forum/index.php?topic=2864.125) with (http://www.magiclantern.fm/forum/index.php?topic=6751.msg71720#msg71720) that (http://www.magiclantern.fm/forum/index.php?topic=10111.0).

@vstrglv:please find the requested download here: [this feature is only available for members having a gold subscription]if that doesn't work, and the arguments of the developers isn't reason enough: feel free to implement it

Hey, installed latest build on 5D3 today as I was really hoping to record 24fps 1080p 14bit lossless on the CF and proxy files on the SD.

Unfortunately, the proxy files will only record for about 6 seconds or so before live-view locks and the battery needs to be pulled. Recording both the 14bitlossless and proxy on the CF card results in the same/similar result.

With the latest crop mode build recording 10bit 1080p on CF and proxy on SD works great!

Anychance I'm missing something or there is a quick fix for this? Cheers

Right, I had two commits in the raw-h264-branch that I thought were included, but looks like they were not (I've never actually tried this feature on this branch). One (https://bitbucket.org/hudson/magic-lantern/commits/d0b78dfc82) is a workaround to prevent hard lock-ups when the H.264 stream stops, and the other (https://bitbucket.org/hudson/magic-lantern/commits/9465a180dfe4) attempts to sync the file names.

Last time I've checked (a long time ago), I had good results with H.264 IPB on SD, but recording stopped very quickly with both on CF (don't remember much about ALL-I).

Currently, the only reason I could think of would be in-camera playback.

I'd actually recommend lossless compression with 12-bit (high quality, as the last 2 bits are mostly noise (https://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html)) or less (to be tested). Keep in mind I'm not an active video user, so the recommendation is based on theory, not on practice.

WOW........ @a1ex you did it. I just tried the new build, it recorded @ 3.5K 12bit Lossless 2.35:1 continuously.. Zero Hick-ups so far. Converted the images with no problem they came out beautiful :D

@goldenchild: are you sure that you managed real 2.35:1 with 3.5K? 3.5K @ 2.35:1 (3520x1498) only 14bit lossless is supported with this vertical resolution and here you end up at >105MB/s at 50% compression --> not continuousAlso 1320pixel is max vertical resolution for the 5x crop mode (3.5K 1:1 centered x5 as well as 5x live view crop) and this is currently the max. vert res for 12bit lossless and lower

By the way I tested 3.5K 2.39:1 at UHD crop mode to get a vert res >1320 but at 14bitlossless. Depending on the brightness of the image I manged to get 120 to nearly 200 frames in this resolution.Here is a quick test:

https://1drv.ms/i/s!ArsIFng0eZ9dgYhqj9iycEK3sGsFSQ

3520x1472; 14bitlossless; 23.976; UHD crop mode --> ~105-110MB/s

ISO 400 --> 120 framesISO 320 --> 150 framesISO 200 --> 186 frames

as mentioned above 3520x1320 works continuous due to 12bit lossless capability, but is 2.67:1...

By the way there is a small bug: when accidentally entering the UHD crop mode from 3.5K (where 5x lv zoom + fps override to 23.976 needs to be enabled) at a resolution of 3520+ the FPS override changes to an odd 33.. fps speed. When turning fpsoverride off it changes to 19.. fps. While turning off, the Picture Style window from Canon menu flashes up once.To eliminate this you need to turn off crop mode und raw video, close ml menu and turn everything on again (without x5 live view zoom + fps or!)... Nothing substantial but a bit confusing.Can anyone confirm this?

Right, I had two commits in the raw-h264-branch that I thought were included, but looks like they were not (I've never actually tried this feature on this branch). One (https://bitbucket.org/hudson/magic-lantern/commits/d0b78dfc82) is a workaround to prevent hard lock-ups when the H.264 stream stops, and the other (https://bitbucket.org/hudson/magic-lantern/commits/9465a180dfe4) attempts to sync the file names.

Last time I've checked (a long time ago), I had good results with H.264 IPB on SD, but recording stopped very quickly with both on CF (don't remember much about ALL-I).

Build updated.

Hey Alex, unfortunately those changes you made to the latest build (2017-04-27 21:51) didn't do it. It still only records 6 seconds ALL-I or IPB before I get an "emergency stop" notification on the live view. Tried 12bit lossless, 12bit raw, and 10bit raw, makes no difference. I made sure to set record to card 2 (SD) in the canon menu each time.

By the way there is a small bug:when accidentally entering the UHD crop mode from 3.5K (where 5x lv zoom + fps override to 23.976 needs to be enabled) at a resolution of 3520+ the FPS override changes to an odd 33.. fps speed. When turning fpsoverride off it changes to 19.. fps. While turning off, the Picture Style window from Canon menu flashes up once.To eliminate this you need to turn off crop mode und raw video, close ml menu and turn everything on again (without x5 live view zoom + fps or!)... Nothing substantial but a bit confusing.Can anyone confirm this?

By the way there is a small bug: when accidentally entering the UHD crop mode from 3.5K (where 5x lv zoom + fps override to 23.976 needs to be enabled) at a resolution of 3520+ the FPS override changes to an odd 33.. fps speed. When turning fpsoverride off it changes to 19.. fps. While turning off, the Picture Style window from Canon menu flashes up once.To eliminate this you need to turn off crop mode und raw video, close ml menu and turn everything on again (without x5 live view zoom + fps or!)... Nothing substantial but a bit confusing.Can anyone confirm this?

Trying to reproduce this on my end to no avail. Was this done on 113 or 123?

@vstrglv -- perhaps you didn't press magnified zoom to get into x5 mode (which is required for this mode - read the notes on bottom of LCD LiveView) while having '3.5k 1:1 centered x5' enabled from crop-mode within ML menu and then select 3520x1320 in 2.50:1 w 12-bit lossless.

I'm liking the latest build, full res liveview is getting pretty darn reliable at 7.5fps, 2.35 crop. Almost continuous recording, I had a few shut off after 5 seconds. I drove up into the mountains to see how some of my older lenses handle 4k+ and made a short video.

I did notice that it would shut off right away if I was shooting right into the sun at ISO100 but I kind of expected that after reading through the thread. It's getting damn good though.

Hey Alex, unfortunately those changes you made to the latest build (2017-04-27 21:51) didn't do it. It still only records 6 seconds ALL-I or IPB before I get an "emergency stop" notification on the live view. Tried 12bit lossless, 12bit raw, and 10bit raw, makes no difference. I made sure to set record to card 2 (SD) in the canon menu each time.

Confirmed - it was broken by an experimental change in this branch - the way we allocate memory for the main raw buffer (during standby) conflicts with H.264. I've assumed Canon's memory layout does not change as long as you are not leaving LiveView, but it looks like it does (so our raw stream ended up overwriting the memory areas used by H.264). Looking into it.

I'm liking the latest build, full res liveview is getting pretty darn reliable at 7.5fps, 2.35 crop. Almost continuous recording, I had a few shut off after 5 seconds.

Nice to see this mode working! You should be able to reduce the memory workload (and hopefully free some resources to make them available to the card writing task) with some tricks:- dial down the resolution in crop_rec submenu (target yres); note the FPS will increase- re-center the image using CMOS[1] hi and lo (trial and error)- enable FPS override in low light mode and dial it down to 7.5 fps or whatever you need- try the "Frozen LV" preview mode for a little more writing speed.

Nice to see this mode working! You should be able to reduce the memory workload (and hopefully free some resources to make them available to the card writing task) with some tricks:- dial down the resolution in crop_rec submenu (target yres); note the FPS will increase- re-center the image using CMOS[1] hi and lo (trial and error)- enable FPS override in low light mode and dial it down to 7.5 fps or whatever you need- try the "Frozen LV" preview mode for a little more writing speed.

@beauchampy No shutter actuations and a slightly nicer workflow IMO. The only issue would be battery life for really long lapses. The Mark iv shuts off the LCD. But for 1fps or maybe .5 fps it would be awesome. Havent tried it yet.

Hey Alex, unfortunately those changes you made to the latest build (2017-04-27 21:51) didn't do it. It still only records 6 seconds ALL-I or IPB before I get an "emergency stop" notification on the live view.

Solved. The changes were non-trivial, but since it was a memory management issue, it got higher priority.

At 1920x1080 24p 12-bit lossless, and a bit of luck (because of the variable bitrate) recording can be continuous even with both RAW and H.264 on the same card. With H.264 on SD and RAW on CF, both recorders have *much* more headroom.

Careful when the card gets full, as this event is not handled very well (and it requires more reverse engineering to figure it out).

Solved that too (although I wish I had a proper bug report). The real-time estimations (and the indicator color) were completely messed up in lossless recording modes, and just OK-ish in uncompressed mode (with estimations being far away from actual recording times).

Wow, the latest build is very stable and much more fool proof! Got 48p 1080 working flawlessly and the full res live view puts up no fuss at all. I really love the total frame count at the end, very useful.

Now I'm trying to decide if I should drive out somewhere this evening and do another test... maybe I'll go to the beach.

I've cherry-picked some changes from this branch, to be included in the main builds (for which I need some help with code review). First round:

https://bitbucket.org/hudson/magic-lantern/pull-requests/827https://bitbucket.org/hudson/magic-lantern/pull-requests/828https://bitbucket.org/hudson/magic-lantern/pull-requests/819 (last few commits)https://bitbucket.org/hudson/magic-lantern/pull-requests/825 (older, but no reviews yet)

Next on the list:- compressed_raw (without all the changes required for 4K, which are mostly 5D3-specific), so you can start porting it on other cameras- compressed_raw with lower bit depth (I'd like to keep them separate, so you can focus on plain 14-bit lossless and then move on to more complicated stuff)- H.264 proxy (there are few changes, but currently entangled in the middle of other 4K experiments)

If you are wondering why: this branch became pretty complex (includes many changes that are not exactly related) and many of these things are not portable to other cameras, or just highly experimental. Splitting it into small logical chunks should make it a bit more manageable.

Because the preview in 3k and above is so laggy, I hadn't thought of plugging in my monitor to it. But tonight I tried it out. The field of view is about 40% of what's on camera screen and being recorded. But it's great for focusing and no lag. If you can get used to operating with two monitors (one for composition and one for focus), you can really shoot a 3k project with a 5diii. Hell, if you had a focus puller you'd be in even better shape.

The HDMI monitor output (when mirroring, at least) is completely centered. Short of being the correct frame without jaggy lag, I'm wondering if the HDMI mirrored output could be panned as desired to frame what you want to see to keep in focus.

The cropped panning control knob does something when you use it but the screen then pop back to how they were.

Because the preview in 3k and above is so laggy, I hadn't thought of plugging in my monitor to it. But tonight I tried it out. The field of view is about 40% of what's on camera screen and being recorded. But it's great for focusing and no lag. If you can get used to operating with two monitors (one for composition and one for focus), you can really shoot a 3k project with a 5diii. Hell, if you hades a focus puller you'd be in even better shape.

The HDMI monitor output (when mirroring, at least) is completely centered. Short of being the correct frame without jaggy lag, I'm wondering if the HDMI mirrored output could be panned as desired to frame what you want to see to keep in focus.

The cropped panning control knob does something when you use it but the screen then pop back to how they were.

Thanks for posting this info! Just curious. Are you using 1.2.3 firmware or 1.1.3? Thanks!

Hello :)I wanted to ask, if, with all the new development and discoveries you guys made lately, there is a way to improve/shorten rolling shutter ms for normal 1080p recording.

Already done (http://www.magiclantern.fm/forum/index.php?topic=12656.0) a long time ago (actually since 2012 (https://groups.google.com/d/msg/ml-devel/tSg5v99I7lQ/uL-XTHQ8fBQJ); we just didn't know what we had done, back then).

It might be possible to push it a little lower. With adtg_gui and enough patience, you can probably figure it out. All you have to do is to reduce FPS timer A as much as you can, but you'll also have to reconfigure some other parameters to allow this (don't know which ones - that's what you have to find out).

Sorry guys, with the new MLV_Dump the DNG file sizes are coming out much larger, for 3k 12 bit they are 7mb each. I am using this script to run MLV_Dump, please help me make corrections. Also, it seems to be taking much longer as well.

If you used compressed raw, the MLV files are always smaller. The dng files that are made by MLV_dump are not compressed, so bigger.And as far as I know, the output in dng from MLV_dump is always in 14 bit. So you could have a compressed 12 bit MLV file, but the dng's that come out mlv_dump are uncompressed 14 bit.

If you used compressed raw, the MLV files are always smaller. The dng files that are made by MLV_dump are not compressed, so bigger.And as far as I know, the output in dng from MLV_dump is always in 14 bit. So you could have a compressed 12 bit MLV file, but the dng's that come out mlv_dump are uncompressed 14 bit.

I installed the firmware version magiclantern-crop_rec_4k.2017Apr26.5D3113. I shoot 1920x1080 30p 14bit losless, it works fine, but when I open the video on the computer, in all the videos I see on the left a black bar about two pixels wide. What is the problem?

Sorry guys, with the new MLV_Dump the DNG file sizes are coming out much larger, for 3k 12 bit they are 7mb each. I am using this script to run MLV_Dump, please help me make corrections. Also, it seems to be taking much longer as well.

EDIT: MLV files are 16.6gb, dng output are closer to 30gb.

you now can choose betweena) output uncompressed DNGsb) compress DNG content (bit slower bit same as uncompressed, just many tools have trouble with lossless compressed DNG)c) pass through the original lossless data (which will be fast but skip all processing like stripe and hot pixel detection)

you now can choose betweena) output uncompressed DNGsb) compress DNG content (bit slower bit same as uncompressed, just many tools have trouble with lossless compressed DNG)c) pass through the original lossless data (which will be fast but skip all processing like stripe and hot pixel detection)

you now can choose betweena) output uncompressed DNGsb) compress DNG content (bit slower bit same as uncompressed, just many tools have trouble with lossless compressed DNG)c) pass through the original lossless data (which will be fast but skip all processing like stripe and hot pixel detection)

Haha, I was doing the same pass-trough stuff yesterday but you really did it a lot better (and faster) than me!!! :D plus there are also some more fixes/changes. Thank you so much!

There is a small question. When -c not specifyed e.g. 'mlv_dump -o result.mlv source.mlv', the standard output is:

@jankrueck Great job footage looks awesome.. You also captured great dynamic range, windows are well exposed & still having enough dynamic range for the interior superb.

thanks! my maingoal was to test around with DR, detail and cameramoving, as most of the test I saw where handheld or tripod landscape. the Dynamic Range is just insane. Its out of Camera. Didnt lift shadows or saved highlights at all.

Before reporting any differences in recording times, please make sure they are not caused by small variations in the data rate (unavoidable with lossless compression) or by different settings (e.g. global draw, preview options).

I'm currently benchmarking the two builds with this method (http://www.magiclantern.fm/forum/index.php?topic=17091) (this script (http://www.magiclantern.fm/forum/index.php?topic=17091.msg165766#msg165766)).

Just FYI, it's not the first time people are reporting speed differences between nearly identical builds (http://www.magiclantern.fm/forum/index.php?topic=17091.msg165688#msg165688) (where the only change was something like fixing a typo in XYZ language (https://bitbucket.org/hudson/magic-lantern/commits/2a1f972cdbd87e77059539622fd4308618fa9ae6)).

to me and a friend (we have same CF CBx1066 tested) with 27 and 29 april , vers. 123 , orange icon at first 20 seconds circa red after and stop! no way ...tryed 20 times...if we set 1920x800 orange icon at first 10-15 seconds...green , orange, green.... but seem to be continued...(i try more of 2-3 minutes without stop)

so I don't know if 1920x960 50p was working better in old version...this I can't say....

I think is something related to compression and how both builds manage it. With April14th I get green icon from start to end of recording, no matter what image I'm recording. With last buid April29th, I get green icon if I'm recording a dark image with no detail, and when I point the camera towards a detailed and well iluminated image, icon changes to orange and quickly to red. Then stops.But I can be wrong... It's just an assumption from my ignorance.

Does anyone else besides me have a black bar two pixels wide on the left side of the frame on all the videos? Firmware version magiclantern-crop_rec_4k.2017Apr26.5D3113. I shoot 1920x1080 30p 14bit losless.

With April14th I get green icon from start to end of recording, no matter what image I'm recording.

... and stops with green icon when you do this:

Quote

With last buid April29th, I get green icon if I'm recording a dark image with no detail, and when I point the camera towards a detailed and well iluminated image, icon changes to orange and quickly to red. Then stops.

As mentioned here (http://www.magiclantern.fm/forum/index.php?topic=19300.msg184055#msg184055), I've fixed the indicator, so it will actually warn you when it's going to stop.

Each number from the "big matrix" shows how many frames were in each test clip, and a new line means the card was formatted (and possibly other build was tested meanwhile - they are chosen randomly after filling and formatting the card). 1920x960 50p, 14-bit lossless, dark scene with artificial light (not changing during the experiment), preview set to Framing (which is not continuous here).

So, I've added some intermediate versions to the test script to narrow it down. The scene was a bit different (moved the camera), so the numbers are not directly comparable to the first run (variable bit rate):

It looks like the first 3 builds (in the sorted list) have the regression; that means, it must have been introduced in the earliest changeset from this group (since the previous changeset was also tested): e67fac (https://bitbucket.org/hudson/magic-lantern/commits/e67facae488444a132aaa6107de0e3eb4e3129eb?at=crop_rec_4k).

That means:- The regression only affects the behavior when recording is about to stop (how many frames it manages to squeeze in the last moments)- It does not affect the ability to record continuously. This piece of code starts kicking in when the buffer is about 90% full.

Is the regression caused the overhead of printf (possibly delaying the decision by a non-negligible amount of time) or it's because of the change in thresholds (resulting in smaller blocks being written to the card)? Need to restart the experiment to find out.

to make mlv_snd work with mlv_lite, the buffering structure must be changed to allow other modules to queue blocks.that are only a few callbacks, but this might have a negative feedback on write speed.

currently mlv_lite is meant to be a rewrite from scratch to get the maximum write speed.when all experimental stuff settles, maybe this will be added.

This test was moderately helpful. I wanted to see how the 3k and HD compared with a high ISO -- 1600 (high for a 5Diii, at least). Three different lenses all shot at f2.8 at 1600 ISO. The field of view for ML cropped 3k is about 1/2 of what it is for full frame HD. So for the HD version of the shot, the camera was moved from 48" away to 24" away. I down-rezzed the 3k to HD.

I didn't have the patience to get the scale exact. I kept image pretty neutral. I used ETTR for exposure. I think it helped lessen the shadow noise (I tend to underexpose ML RAW, not in a good way). Anyway, the noise isn't too awful especially since you'd most likely crush the blacks some if you were using this shot. I was expecting more noise at 1600 ISO.

Hey guys, just curious. I shot some footage, about 4 takes of a local artist and its about 114gb. 3k 12bit lossless 3.5k centered mode and it all went very well. It flashed between orange and green but I also had the global draw on. Camera temp hit orange a few times but it all worked great and it is super sharp!!

My question has to do with MLV_dump.exe, I left it overnight to process the DNGs but I think it got stuck because it never got past the first set of mlv files. I had selected all the time, 32 in total. I am re-attempting to do them one clip at a time now, but I'm curious if there is limit on GB size or amount of files I can run at once.

Is this normal? Does it need to process the images more than once? Thanks for all your help guys!

Hmm, I've been using mlv-dump (and the dumper batch) a bunch lately and haven't had any issues. Do you have the latest version of mlv_dump.exe? And are you sure you have enough hard disk space?

Yeah I am using the latest dump, and I have plenty of HD space. All the files get extracted, I can stop mlv-dump and my files are there for 1 clip. Ive been using it fine for the past week or so this just occurred the past few days. I havent changed anything so I'm not sure whats going on.

Is the regression caused the overhead of printf (possibly delaying the decision by a non-negligible amount of time) or it's because of the change in thresholds (resulting in smaller blocks being written to the card)?

Results of over 30 hours of benchmarking (Apr29 codebase with minor changes - included in the raw log - to check the effect of these changes on performance). During this time, the camera was under the control of this script (https://bitbucket.org/hudson/magic-lantern/src/raw_benchmark/scripts/extra/rawbench.lua), unattended.

But, the assumed write speed (when deciding to limit the number of frames saved in a single file write call, to avoid running out of space during the call) appears to have a sweet spot between 80 and 90%:

So, while the first place (which assumes 90% of the measured speed for the last file write calls) is clearly better (squeezes more frames) compared to last two places (which assume either 100% or 50% of the measured speed for the same calls), we cannot tell the same about the first two winners - they are tied (could not prove one is faster than the other). Same for the last two places. The sweet spot is probably somewhere near 85% (not tested, just guessed).

(http://a1ex.magiclantern.fm/bleeding-edge/raw/ovf-sweetspot.png)

However, the best thing to do for speed is to write blocks as large as we can (in other words, as few file write calls as we can), and free the buffers as soon as we know they have been written. Here's an experiment on this:

With large blocks, and reusing image buffers *during* a file write call, performance is clearly better (possibly also helping with overall speed, as the large buffers are freed and can be reused quicker).

(this experiment is a litte risky in my opinion, as I have not tested it for data integrity, other than self-checks built in mlv_lite, so I'm not comfortable including it in the builds, but you can build it from source - diffs available in the raw log)

Does anyone have tried Dual ISO in crop_mode? It seems there are problems there. Frames (after Dual ISO Processor) comes with stripes like unprocessed dual ISO frames. At least with x5 (x3) zoom. Is there some solution?

Ok, trying to understand what to look for.The non crop file seems ok. After running the file through cr2hdr I get this.(https://s18.postimg.org/72li7s4jt/Screen_Shot_2017-05-06_at_16.46.00.png)

Quote

Quote

Does anyone have tried Dual ISO in crop_mode? It seems there are problems there. Frames (after Dual ISO Processor) comes with stripes like unprocessed dual ISO frames. At least with x5 (x3) zoom. Is there some solution?

Checking your file frame_000008frame_000024_crop_mode_with_zoom.dng it doesn´t look healthy. So you´re problems are only the crop mode recording with compressed 4k modes? If yes, could you share a small MLV sample instead of a single dng file?

Ok, trying to understand what to look for.The non crop file seems ok. After running the file through cr2hdr I get this.(https://s18.postimg.org/72li7s4jt/Screen_Shot_2017-05-06_at_16.46.00.png)

Checking your file frame_000008frame_000024_crop_mode_with_zoom.dng it doesn´t look healthy. So you´re problems are only the crop mode recording with compressed 4k modes? If yes, could you share a small MLV sample instead of a single dng file?

Try to zoom it. Non crop I mean. And how did you running it through cr2hdr? I tried but it just flashing and nothing.

Thid mlv file with no zoom. https://yadi.sk/d/7nw0j1Fq3HmYpN (with zoom pretty big for half Gb mininum). As I see, problem with stripes is in unzoomed DNG's (RAW Dual ISO on crop_mode build just fine as well as Dual ISO MLV's on older builds).

I´m already a big fan of the crop_rec4k mlv_dump version and the compression of dng and MLV files are great. Encountered an issue with eosm footage. Files won´t parse(open) in acr and dcraw reveals what´s wrong. Corrupted file. Other footage from eosm is working.

Unexpected end of file(https://s29.postimg.org/r72sssd6v/Screen_Shot_2017-05-07_at_20.27.28.png)

*update.Maybe metadata related. Other random ml builds is working with eosm. Guess I was lucky with this corrupted footage 8). Wish I knew what build I was using when filming. Have a bunch of files om my computer which comes out like this. The transcode just fine with other builds of mlv_dump.

*update 2It seems it´s because of being 10 or 12 bit footage. Guess that´s not working at the moment with the crop_rec4k mlv_dump build.

I'm certain I can get 3520x1320. Not sure that offers a 3x crop option though, I just didn't dive into the settings that hard since i was trying to capture the little guy before he took off.

3520x1320 works for me but only up to maybe 100 frames (KomputerBay 128gb 1066X CF card). But I have only tried it in the 14-bit lossless mode, 24fps. Is anyone getting longer recordings using 12-bit lossless instead?

I realized after posting yesterday that 12-bit lossless and 10-bit lossless aren't enabled yet for the 3K/4K crop modes anyway. So 3520x1320 was continuous for you in 12-bit uncompressed? I'll have to try that out! I just assumed the larger bitrate would be prohibitive, but maybe it's something else that inhibits the 14-bit lossless from doing it continuously at that res?

I'm not sure these are accurate rec times mentioned above. Possibly I guess but as far as my testing has shown, 3072 2.35:1 23.976fps override on the 5x mode in the crop rec menu @12bit lossless are continuous. Even with these settings there could be scenes that can spike the data rate so that it is not continuous. Above that resolution, I don't THINK you will see continuous. Remember, if you are using the 5x option under crop rec menu, you have to hit the zoom button to actually rec the resolution. If you don't it will just rec 1920. I'm thinking maybe that's where some confusion can happen. If your not zoomed in the 5x option in the crop mod you will not be recording higher then 1920 resolution. Also if you are using the 5x mode you will need to enable fps override since canon live view defaults to 30fps when zoomed in.

Recording @ 3.5K 12bit Lossless is really tricky it all depends on your scene. 3072 - 12bit Lossless is working lovely.. Live View stay's full color and I'm able to focus properly. Not sure if anybody experienced spiked data rates using the Sigma 35mm ART lens, or any of the ART lens. I'm thinking that because those lens are soo sharp it's causing that issue.

Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183548#msg183548)).

Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183548#msg183548)).

Cinelog_Rec709 has been applied for comparison viewing purpose. Full album can be found here: https://flic.kr/s/aHskVk85io

Pre DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4158/34450561991_2c584c7d1a.jpg) (https://flic.kr/p/Uuh8QK)

Post DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4168/33738123384_3ddba92881.jpg) (https://flic.kr/p/TpjGEu)

Pre DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4174/34450578201_a8ffa6712b.jpg) (https://flic.kr/p/UuhdEe)

Post DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4161/34450549851_0bec0a5d1f.jpg) (https://flic.kr/p/Uuh5er)

Pre DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4179/34450580751_e333e68295.jpg) (https://flic.kr/p/Uuheqc)

Post DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4183/33738094614_51049bfdb3.jpg) (https://flic.kr/p/Tpjy7s)

Pre DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4169/33738115394_d00f1a6282.jpg) (https://flic.kr/p/TpjEhJ)

Post DF (pushed exposure up to +2.50 in ACR):(https://c1.staticflickr.com/5/4194/34194921370_04b217357d.jpg) (https://flic.kr/p/U6FUMQ)

Such a relief that @bouncyball, @Danne, @g3gg0 and the rest of you all have made this possible to bring back this useful feature in DarkFraming Average Processing. Especially for lossless compressions.

Yes, all of this was solely done with the latest and greatest cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) (Thanks @Danne for standing by with all of your love & support!)

Looking good DeafEyeJedi! Do you create the darkframe by shooting (with the same settings) with the lenscap attached? Can you do it anytime after the shot (like when i go on holiday and i make the darkframes when i come back home) or does it have to be close together in time? For windows users mlv_dump can do this right?

Cinelog_Rec709 has been applied for comparison viewing purpose. Full album can be found here: https://flic.kr/s/aHskVk85io

Such a relief that @bouncyball, @Danne, @g3gg0 and the rest of you all have made this possible to bring back this useful feature in DarkFraming Average Processing. Especially for lossless.

Yes, all of this was solely done with the latest and greatest cr2hdr.app (https://www.magiclantern.fm/forum/index.php?topic=15108.0cr2hdr.app) (Thanks @Danne for standing by with all of your love & support!)

There are a few other buttons that send the same event (it's hard to tell the difference between them, other than waiting about 0.5 seconds to see whether it starts autofocusing or not). Use any of them ;)

Could someone explain, please, what i do wrong? My brain is just exploding of complexity.

I've been trying to shoot 3k or 4k or 5k for a few days using instructions here, but everything either work 5 sec or switches to 1920 automatically. Every day it behaves differently. One day i managed to work each resolution but time was different - 3 sec mostly. Couldn't do what i've seen others shot.

Right now i tried to shoot 3,5K crop according to instructions and it allowed me to shoot only 5 sec, also preview freezes. Maybe i've done something wrong with FPS override? I tried 23.976 (from 25), Exact FPS - 23.976 and Low light - 23.980.

My goal is to find the maximum resolution and aspect ratio with reasonable time (say, from 5 min of recording).

Are you choosing a lossless compression? 3.5k only seems to work continuously with 12bit lossless. 3k maybe with 14bit lossless. You'll have to lower frame rates down significantly to get 4 or 5k. They are not feasible options for traditional frame rates.

Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files. https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2

What seems to work now is following:

MLV files- Compression(lj92) of all flavours of mlv files including 10bit12bit14bit, compressed raw, regular raw. Compression level is very efficient, almost 50%.

- Decompression of mlv files into decompressed dng files or back to decompressed mlv. This works with all MLV flavours going from compressed straight to decompressed dng files. This is really great since we do not need to decompress the MLV file once it´s compressed.

DNG files- Compression of 14bit dng files, all flavours works very good. Efficient compression around 40-50%.

Not working- Compression of 10bit12bit files. DNG output comes out all black but compressed and with metadata present.

I did some minort changes to get 10bit12bit working at all. There is also some testing with decompressing code since the files will come out with incosistent sizes when decompressing. Depends on if coming from mlv_block or outputting dng files. Check code here.https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2

I´m trying to see what is missing around 10bit12bit now. One funny note is that if erasing bitdepth = 14; mlv_dump will output compressed 10bit12bit files fine into original 10bit12bit. Of course this will not work with vetical stripes code since it´s written in 14bit but then again it means it´s halfway working when run_compressor is set.

If any hints to these findings I´d be greatful since I hit a wall now.

Here are some testfiles by the way.https://bitbucket.org/Dannephoto/magic-lantern/downloads/10_12_14_bit.zip

Are you choosing a lossless compression? 3.5k only seems to work continuously with 12bit lossless. 3k maybe with 14bit lossless. You'll have to lower frame rates down significantly to get 4 or 5k. They are not feasible options for traditional frame rates.

If lossless compression means 10-12 bit instead of 14 bit, then yes, i tried it as well. Thank you for the tip!

I wonder can there be a difference between 1.1.3 and 1.2.3 firmware versions.... I'm using 1.1.3.

@whysodifficult @rob_6This video is not for those new to ML. Try this at your own risk. I am not responsible for any damage this video may bring if your camera explodes or bricks. Magic Lantern is not responsible either. It will probably just brick not explode though.Magic Lantern 3072 2.35:1 12 bit Lossless RAW Recording from the CROP_REC_4K build from 4-29-2017Once you have ML installed and the correct build on your SD card, this video should help you set up your 5DIII to record 3K almost continuously depending on the scene. Remember ML on SD and Rec to CF.Remember this build is experimental and not for those that are new to ML. Try at your own risk.

Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files. https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2

What seems to work now is following:

hi dannethanks for your feedback and testing!

does that mean with that changes you managed to compress 10/12 bit images correctly?made test cases for all combinations of compression/decompression/recompression/pass-thru-unprocessed-lossless to check mlv_dump behavior.

though i didn't complete that to cover 10/12 bits, just made some sloppy dev tests.

Hi g3gg0. Thanks for getting back on this. I'm out at the moment but will describe from memory. As you describe 14bit compression(-c --dng) is working nicely with dng exports. When exporting 10bit12bit the dng files comes out all black.What is working is compression of MLV files and that works for all flavours, 10bit12bit14bit.

What is adressed in my code changes is to firstly get regular uncompressed 10bit12bit working again at all since it was broken in this build. Next was to try -c with 10bit12bit but always comes out black.

My last code changes made it possible to export to uncompressed dng files coming from compressed MLV. That works with all bits and flavours. Sorry for these messy allround tests. If I would suggest for one priority fix is to get -c --dng option working with 10bit12bit files if possble at all?

Here are a couple quick edits I did last night. First one no real color work I was chasing my dogs around testing my 3d printed electronic follow focus I made for my Movi.

https://youtu.be/pRROVuxevuU

And one I shot yesterday at Red Bull Hart Lines in Detroit yesterday. No press pass so it was hard to get where I needed to be.

https://youtu.be/Iag4strSaEg

Thanks so much to the ML team! The raw advancements made me fall in love with my camera again. Everything was shot near 3k in 5x centered crop mode as this is the best for shooting with an external monitor on my Movi m10.

Hm, dcraw outputs a valid one. Thought I tried that before. Will test on my mac soon.Fyi if compress a 10bit file(--dng -c) and export it with bit_depth = 10; I get a crisp, compressed and fully valid 10bit dng file. Makes me speculate if upres to 14bit could be applied later after compression? Of course I do not know fully how those bits behave going up and down.

Read a number of posts, and saw a 700D reference but no real explanation for any aps-c port.I am a 70D user, and was wondering what the difference is here? to get to 4k?The 70D even at 3X zoom mode, would be hard pressed to get 2k for any length of time.Is there something I am missing, and what would be a 70D (card write is still 40 MB/s) max?

Yeah, that was me. It seems that all cameras can do the ProcessTwoInTwoOut (http://www.magiclantern.fm/forum/index.php?topic=18443.msg184464#new) thing. Basically the CR2 files are run through that lossless compression process. Then it is another thing to figure out how to tweak the sensor using adtg_gui in order to squeeze every pixel you can out of the camera. I looked into it a while ago but was in way over my head. Some guys are still trying and eventually someone should come up with something that works on the aps-c cameras.

You're right though--that maximum SD card write speed is going to keep these cameras from reaching 4K, except maybe the 7D because it has a faster CF card?

Since I've tried April crop mode module, my shutter-release button has been acting up - it has reset itself twice to AE lock, instead of AFing. Plus microadjustment has switched itself once to a diff value as well. It has never happened to me before and at the time of malfunction the standard ML was installed. No warnings of any kind were shown upong booting with any of the April versions of the ML.

The half-shutter function is changed by ML and can be modified without notice if you take the battery out in the middle of taking a picture initiated from ML, for example. This is documented (http://wiki.magiclantern.fm/faq#does_ml_do_any_persistent_changes_to_my_camera).

If it's not that, find a way to reproduce (http://www.chiark.greenend.org.uk/~sgtatham/bugs.html). I'd expect some similar behavior to happen after an assertion, a crash (e.g. err70) or after taking the battery out - in these cases, ML actively prevents Canon code from saving their settings at shutdown. This is a safeguard to prevent invalid settings (that could be related to the crash) from being saved into ROM (more info (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183060;topicseen#msg183060)).

Note: I've reviewed the calls related to changing half-shutter functionality (cfn_set_af_button and its wrappers) and could not find any instance where the original setting is not restored as soon as it's no longer required. So, unless you (have to) take the battery out in the middle of some photo operation (e.g. bracketing sequence), I don't see how it would happen. Nothing on the video side changes this setting.

I suppose it has been covered, but I haven't been able to find a direct answer to my issue, so don't hit me too hard :)In short - the dng files that have lossless compression aren't supported natively in the latest version of Adobe premiere (at least on my PC). I wouldn't even bother with that because Resolve does a great job, but with the latest version of Premiere i found that it handles canon raw quite good (it can be absolutely my feeling, as didn't find anything in the release notes)

Is there any way to import the dngs directly into premiere? The simple uncompressed dng files work just fine.

You would still need to cross compile to get a Windows binary. In fact on a Windows system you need to set up either Cygwin or a Linux virtual machine and cross compile to get a binary that works on any Windows machine.

Just wanted to let you guys know that I have been recording without any crashing for quite some time with the latest build. I am using the settings recommended in hjfilmspeed's video at 3k 2.35 and it works so well! I am using a 256GB Lexar 1066x compact flash card and can record about 44-46 minutes in 12 bit 3k reliably and it even closes the file once the card is full without crashing! Amazing work A1ex and everyone else that helped make this possible. I literally feel like I have a new camera! The only thing to be careful of is the compression and not letting the camera overexpose which seems to stop the recording due to the increased recording bitrate.

I have one question. As we know, if I half press the shutter while in the 5x crop mode it zooms in and allows me to focus with a normal live view playback. I usually half press the shutter before hitting the record button to get the focus right. But, I have been successful pressing the shutter half way while recording to get focus during the recording, too. Does pressing the shutter half way while recording put extra work on the camera which could cause the recording to stop early? So far, it seems to work fine. Just curious if anyone else has tried this technique or has any insight!

...Does pressing the shutter half way while recording put extra work on the camera which could cause the recording to stop early? So far, it seems to work fine. Just curious if anyone else has tried this technique or has any insight!

I had similar thoughts. It seems to work fine on my end but sometimes I suspect that it may cause the recording to stop though not as often. Still trying to reproduce this but it could very well be related to the compression scheme from being slightly overexposed.

A sensel cannot be read out at two different ISOs at the same time (so you are going to have some missing data in either spatial or temporal domain). In the first case you'll get aliasing; in the second one, you'll get motion artifacts.

This might be doable though: http://www.magiclantern.fm/forum/index.php?topic=19315.0

Is there any way to merge DNG-sequence like AviSynth with RAW.mov do? I'm trying to shoot HDR video with crop_mode but AviSynth work just with mov files. Anyone now How to merge DNG frames with different expo to sequence with one corrected sequence?

This is a bit of topic but regarding HDR there is one open source tool that merges dng fileshttps://github.com/jcelaya/hdrmergeI tried this for a period of time but the caveat back when testing was that the tonemapping algorithms are mainly for stills. When merging movie sequences you get flicker since tonemapping will vary from image to the next.What is not that much of topic is to elaborate about mlv_dump being able to merge hdr files coming from MLV footage. I tried some crude examples with averaging code working with consecutive frames. Bouncyball helped me out here with coding but simple averaging would produce pinkish highlights. Now averaging(tblend filter) code in ffmpeg is in another ballpark when looking at the merged footage that I played around with in MLP and cr2hdr.app. No pinks, just a good looking merged file. Of course any of such changes would surely take a lot of time to accomplish and put into mlv_dump but since the question arised about HDR I thought I just put it out there.

crop_rec_4k Apr29 not working here. I have followed exactly the video tutorial, but it result in random multicolored pixels (viewed in Mlrawviewer and raw2cdng1.7.9). also tested with and without fps overide.

I would like to focus on 10bit12bit files in recent mlv_dump again. Seems the fix is very close but if it the problem passes unnoticed now it will probably be buried into other functions adding up in time. I test mlv_dump with the following command(latest crop_rec_4k code)mlv-dump --dng INPUT.MLV

The last commit that will export 10bit12bit files correctly is this1738cb0https://bitbucket.org/hudson/magic-lantern/commits/1738cb06e2652d9751f9407490de507fe30ef520?at=crop_rec_4k

Next commit is where things are getting wrong with 10bit12bit footage. Compiling from this commit will cause segmentation fault 11, however the next commit partly fix this but will export frames with missing information(see examples below)92d1b4dhttps://bitbucket.org/hudson/magic-lantern/commits/92d1b4d2958975f50ba428853545f64db8a61205?at=crop_rec_4k

Following commit, "re-adding 14bpp enforcement for DNG" produces black 10bit/12bit dng files cause it won´t fill the frame entirely(see examples). Dcraw exports the file and it shows where the image information is missing.8b31c98https://bitbucket.org/hudson/magic-lantern/commits/8b31c98380d5dc27573886d0f84ac7ed0c5d0f2b?at=crop_rec_4k

Looking at the code changes I see a lot of changes around frame_buffer_size, read, prev_frame_size so I guess somewhere in there this could be fixed. Unfortunately I couldn´t get this right with the example file provided but hopefully a dev could take a look or give a hint on how to fix it.

dcraw -T INPUT.dng gives:

Working commit1738cb0(https://s4.postimg.org/cc9w4w10d/Screen_Shot_2017-05-24_at_14.21.46.png)

Not working(https://s4.postimg.org/wide3rwnx/Screen_Shot_2017-05-24_at_14.19.39.png)

You can install *Canon* firmware 1.2.3 using Canon menu. If you do this you have to run ML builds for *Canon* firmware 1.2.3. ML is not firmware.All builds should be available for 1.1.3 and 1.2.3. There is no need to upgrade Canon firmware if you don't have to for other reasons.

@Danne thanks, you clarify issue pretty well. Just to remind you and those who develop some software solutions within magic lantern - currently mlv_dump didn't work properly with Dual ISO MLVs from crop_mode. Sometimes frames corrupted with looking-like-labirynth pattern and it seems randomly.

So it seems that currently there are just one reliable (but long) workflow for HDR-video when you shoot in crop_mode - MLV to DNG-sequence then DNG-sequence to MOV then MOV with Avysinth (in HDR-workflow pack) processed to A, B and C jpg-sequence then this C-sequence again assembled in some final video format.

It's an early build put into a 600D. Wonder if it was called raw_rec(mlv_lite). Stupid of me not to download a later build when testing. Right now I don't have the camera nor the firmware version until monday.On another note. Is recent 10bit12bit files working with latest crop_rec_4k like it should? Last I tried it wouldn't work. If so, I guess there is no issue. Don't have any cam here unfortunately to verify.By the way. That particular file will process with earlier builds. Guess there is some other buffer detectors in previous code?

if(read_size > (int)frame_buffer_size)later 10bit12bit files will process correctly. Is this the fix we could use? It still won´t work to compress 10bit12bit files(mlv_dump -c setting produces black output) but I guess they are already compressed being 10bit12bit so maybe compression for lower bits should be excluded?

but thanks for your testing. just pushed a fix (https://bitbucket.org/hudson/magic-lantern/commits/9b214f11098a3be752ca8a5cfe3a17584b3c23b8) that should fix that issue.after converting bit depth, frame size was updated, but not data size.

reason for having both is obvious when compression is used. frame size wont change with or without compression, whereas the resulting binary data varies in size.

Here's my first 3072x1320 24 fps recording:so with 14-bit lossless 3K i am seeing about 6.5 minutes of record time on a 64GB CF card... does this seem accurate? also what is a recommended Aspect Ratio?

What program are you guys using post production (after exporting via mlv_dump). I would normally use Premier pro cc 2015 but it doesn't recognize the .dngs whenever I use any new feature settings...although I can view the individual .dngs fine in photoshop or any image viewer.

I was fiddling around with the UHD 1:1 setting of the crop_rec moduleThe data I was recording was 14bit lossless, iso 1600.It was recording continuously until I pointed the camera at the PC screen and this happened! :D

(http://i64.tinypic.com/eb2zk3.jpg)

I still have the file if any of you would like to inspect it.p.s.I am sure it happened because of the high ISO and the sharp change from a dark corner of my room and the PC screen.I read that 14bit lossless works best at low iso and with not too bright scenes, therefore I believe that was what made magic lantern to do this. I restarted the camera removing the battery and all come back to normal.

p.p.s.When that happens, what button do I press to make it go away without having to restart the camera? :D

I noticed compressed raw and lossless 10/12 bits can´t be selected only 14bits lossless. Searched around but I couldn´t find out the reason why other than from a live view message and some posts over here http://www.magiclantern.fm/forum/index.php?topic=19300.msg183763#msg183763 Is it simply not possible for 10/12bit and compressed raw with the binning modes?

- Set camera to mlv_lite and select 1920x960 @ 50p in the crop_rec menu. Camera set to PAL. - Record a short sequence and stop filming. Now exit liveview closing it the normal way. - Now try to switch into liveview again.

From now on all I get is a black screen. Only way to get back into liveview is to set 1920x960 @ 50p in crop_rec menu to off, then open up liveview again and reselect 1920x960 @ 50p again.

@Danne it happened to me as well.. can't remember which build exactly, but it was one from April. What I did was to format my SD card again and reinstall Magic Lantern again. Sometimes when you use 50p 60p mode something breaks in the RAW modules I think not sure but ask @a1ex.

I noticed compressed raw and lossless 10/12 bits can´t be selected only 14bits lossless. Searched around but I couldn´t find out the reason why other than from a live view message and some posts over here http://www.magiclantern.fm/forum/index.php?topic=19300.msg183763#msg183763 Is it simply not possible for 10/12bit and compressed raw with the binning modes?

What build are you running? I cant get continues in 1920x800 60p. Neither on 113 or 123. especially when bright and lots of details, it will even crash at times saying "compressed size larger than uncompressed" or something like that. Been using the 60p mode a lot this last week.

And a question for @A1ex . Is there possibility of getting bit higher resolutions in the 60p mode in the future? 800 vertical pixel is a bit of a squeeze, i love the look, but not every project deserves the cinemascope ;) Perhaps if you get spanning to work aswell?

@echo offmd "%~n1""C:\XXXXXX\Video Program\MLV_Dump on Steroids\mlv_dump.exe" -o %~n1\%~n1_ --dng %~n1.MLVTo Batch convert Selected MLV files in a folder. But after replacing the mlv_dump with latest, I can not get it to Batch Convert anymore, it will only convert one and then exit.

I don't know of any other converter for these new MLV's?

Please help, I have so incredibly many MLV's to convert.

MLVFS, does not work for me, as AE will only take in to account the changes of the First Frame from ACR.

Was testing out the most recent build and had a slew of errors when I tried recording video (there is one possible mistake I made, reinstalling ml now to see if I can reproduce error). I tried to start recording, and it did so for a few seconds, but then said card full. I didn't realize at the time, but it didn't switch over from cf to sd card, so I couldn't test to see if it would still have worked if I had switched manually. That said, since that happened I have been experiencing two kinds of errors. One, when I go to record video if I am using anything above 3k resolution a "script" will run indefinitely, saying that "mean is incorrect" and something about black level, and a range of numbers in the 2000-3000s. At the moment, I am able to reproduce this, but am unsure if reinstalling ML will fix. I have crash logs, etc. as well. The main thing that is also happening, is that I kept seeing text saying "video end not working". When I ran the self test, this message popped up quite a bit. Any thoughts? Should I provide any of the stuff I've got, crash logs and such?

Magic Lantern updates may generate incompatibilities with existing supporting tools. This is natural.Developers of ML and ML tools are not gathered under the same roof and develop in synchronization.

ML developers specified in the first page that the only way to get DNG files is using mlv_dumpthat is available in the download / experiments page.Insisting to use last years tools, or older, together with today's ML build, is a waste of time.

You could build xmp sidecar files of every dng and after effects will treat every dng as unique despite being a sequence.The problem is not mlvfs but AE.

Hey Danne,

How do i create xmp sidecars in AE? And wont that take a lot of time? Not sure how it works, but i am imagining that ACR has to go through each and every DNG and save its xmp.. I currently have about 200k frames to process.

Was affraid you'd bring something like that up. I don't know how to do scripts or coding of any sorts. And incorporating yet another Adobe product in to my workflow is not what I want to do. I had a slow, but steadily working workflow "Send to Mlv_dump" batch convert - Transcode to Log intermediate in AE - Edit and Colour Grade in Resolve - Done.

Danne, you are good at coding. The code I posted above, it is the same I used before with the older version of mlv_dump. I assume the code is right, as it worked before. Could it be something with the mlv_dump version that is preventing it from opening the next MLV ? I have tried deleting all MLV_dump, change locations of the mlv_dump, tried having no Blanks in the Location to Avoid the "", which I understood was for Blanks in locations. But nothing works. I have not tried going back to the older mlv_dump, as i really like that I now can play the DNG's in MLRawViewer, its actually a Must-Have that I can do that.

I have no idea. Don't you're command prompt tell you why it breaks?Suggestion.1 - try an older mlv_dump file and see if that works2 - upload a problematic file if any.3 - check with one of the windows threads4 - test with latest mlv_dump and specify --relaxed which raises tolerance levels regarding broken/missing blocks.

Hey there.I know you have plenty to do sir, but do you think it could be possible to throw in a very experimental sound build?Don't care if it will be bulky to use. I have a projekt where its impossible to use a clap every time ;(

I guess he mean (not native speaker though too) ir's not just you who asked something. I did too but dropped almost instantly since nobody responded. Aaand... A miracle! Now I know almost everything that once puzzled me. Actually thanks Danne for Dual ISO problem with crop_mode since it's he's tests with my footage showed me that it is not bad camera settings problem but rather flawed Dual ISO processing software

In the end we are all german and everybody is just writing english, because they think the others don't understand :D

Guys I have on quick question. Most likely im bringing something up, that has been answered somewhere else, but I couldn't find it. Why is it not possible to shoot 12bit or 10 bit raw in modes with higher vertical resolution like 3k mode or 1920 60 fps mode? Is this a limitation in anyway or has it just not yet been implemented?

As I understand card_spanning doesn't work on experimental crop_mode build so we are restricted with ~100mb/s so there are no hopes for sustainable 4k at 24fps (half-fps or redused vertical resolution is not an option). Or it is possible to add SDcard speed and up a bit writing speed for sustainable 4k >5sec rec?

+1 for the sound also i currently have a work around which is really cumbersome but that works with Full HD defintion.

I activate proxy recordi choose the SD card in ML menu as the prefered card for writing.In canon menu I set the IPB as codec (all-i gives me more crashes but this is probably due to a weak SD card). SD mode 640 does works aswell.In the End you get 2 files : 1 proxy video with sound on SD and 1 Mute MLV on CF.

The problem with this techniques is that the 2 files does not start in SYNC (h264 always start first) so you need to resync you audio with the MLV and run an export of the audio to match the MLV sequence.It's like a clap sync without the clap but as heavy to deal with though. The benefits i that you get a proxy file you can start working with and find a little hand to do the dirty work of exporting matching audio.

The problem is that it does not work with 3K Mode as there is no preview to make image match

So if getting sound into the MLV is still hard to manage maybe enabling any kind of poor preview to the 3k mode would help to do the trick in meanwhile. or even better a way to have the proxy start in sync with the MLV

+1 for the sound also i currently have a work around which is really cumbersome but that works with Full HD defintion.

I activate proxy recordi choose the SD card in ML menu as the prefered card for writing.In canon menu I set the IPB as codec (all-i gives me more crashes but this is probably due to a weak SD card). SD mode 640 does works aswell.In the End you get 2 files : 1 proxy video with sound on SD and 1 Mute MLV on CF.

The problem with this techniques is that the 2 files does not start in SYNC (h264 always start first) so you need to resync you audio with the MLV and run an export of the audio to match the MLV sequence.It's like a clap sync without the clap but as heavy to deal with though. The benefits i that you get a proxy file you can start working with and find a little hand to do the dirty work of exporting matching audio.

The problem is that it does not work with 3K Mode as there is no preview to make image match

So if getting sound into the MLV is still hard to manage maybe enabling any kind of poor preview to the 3k mode would help to do the trick in meanwhile. or even better a way to have the proxy start in sync with the MLV

yeah nice idea. but this realy "sounds" (haha) like too much work.getting a clap with timecode and external recorder will make it a lot easier. (even though it is more expensive)

And no one care about card spanning idea to make (or at least, to try) crop_mode sustainable with 4096 24fps and reasonable aspect ratio? Because now we have essentially half 4k. If it's 24fps so it have some adsurd aspect ratio - like, just band throughout black screen. If we have descent aspect ratio - it's just 12fps... If we have both - it's just 3 seconds of record... 4k, fps, aspect ratio, continuance - choose one. What one can shoot with it?

Okay, I'm not pressing, but can we at least discuss this idea? It is even possible?

Hi there, I am in the middle of a little film project (although I have some time to complete it), and using 10 bit (2017-01-12 / crop3x.2017Jan.5D113) to focus on a water droplet (around 3K). In my setup it is good to make the final film 1080 (also to have the water droplet as big as possible). It is shot inside, and at 1600 iso there is already quite some noise (but I think manageable). I would be very happy if somebody could tell me if it is worth to put the new crop version on it, since I might have to change the setup of my project and lose some time.

- Will the lossless 12 bit (or 14 bit lossless) have less noise and better image quality compared to 10 bit (nonlossless), and more than 3K at 24fps? (there is a lot of movement in the imagery so that might limit the record time)- Will the new version also have the option to choose the 3x crop mode (in addition to the around 1.6x crop mode)? - Will the (older) 3x crop mode have less image quality considering (I assume) it is using a smaller portion of the sensor?

I might have to do these experiments myself, but it would be great if someone already knew some of the answers and help me safe some time. Thanks!

@MartijnR the new 4k crop rec mod is way nicer. Your crop area will be centered, and you can't get better preview options. It might be a little different to use though. Also it is 1:1 resolution like before. Your crop factor depends on the mode/resolution you choose.

@hjfilmspeed Thanks for letting me know! Due to the nature of the project, this will save me a lot of time :) I have seen your tutorial (thanks) and was wondering if you would recommend the 29 april built, or just the latest built?

@budafilms Probably Zeiss 35/1.4 or 50/2 (or somewhat wider, 28/2 or 25/2 if the crop requires that), and the 100/2 for the droplet. That the crop factor is huge is a great thing, considering I need to get the droplet as big as possible in my frame. I also tried a reverse ring, but that's a bit too risky with all the water involved. I will soon start to do some testing. Cheers

@lostfeliz Thanks for your reply. I use a very nice lamp (just the one in my bathroom), with great intensity and nice character. For my film idea, I need this lamp, and to replace it with another bulb is actually a bit risky (been broken before). The intensity is more than sufficient for most shots, but when going as close as possible with the 100/2, you loose quite some light, and 1600 iso seems inevitable. I am curious how the imagery will differ with the new crop and lossless 12 bit compaired to the 10 bit nonlossless raw. Thanks.

That could be related to white level settings. What settings are you using in cam? Upload a sample mlv file?You could develop with cr2hdr.app. It contains latest mlv_dump code. Mlvfs as well. Or mlv_dump binary itself of course.

What a beautiful shot. Where is it from?Usually white balance should be around 15000 but I see yours is at around 5000. Anyway seems a tricky shot, very bright so 15000 won´t work either. I keep a mlv_dump menu section in cr2hdr.app for manually set new white balance and also to create sample file packages so if you want to share a sample instead of the whole 1gb file you could go down the cr2hdr.app routehttp://www.magiclantern.fm/forum/index.php?topic=15108.0Once installed head over to the mlv_dump menu. There is a setting for samples and one for white balance.You could also share the whole 1gb file if you like.

Which means, this is a 12-bit lossless image - that's how the bit depth reduction works. The container is still 14-bit, but the usable range is restricted to 12 bits, with black level equal to whatever Canon firmware assumes (if it's 2048, that would give a valid range of 1536 ... 5586, which means 4050 levels - just below 4096).

The DNG looks just fine here, without adjusting its metadata. It's overexposed, but the highlights are white.

Danne, Alex,Thank you! It's Yosemite National Park in California (rock, called Half Dome) a few days ago, i was lucky with a cloudy weather. It was much more beautiful when i drove to that point (Glacier Point), nothing but clouds around and under you and then they slowly started to disappear, you could see clear spots of the valley down there, of rocks around, and i went to parking for my camera but it was too late when i came back :)

I've just opened this DNG in Photoshop and Lightroom and it looks overexposed (so as google drive shows), but earlier i only looked at all photos in LilyView image viewer on Mac, that shows this photo NOT overexposed, it shows the same how i saw the photo on Live View screen on camera, but the whites are pink: https://drive.google.com/file/d/0B47FS5gpVSwOS1h3b214UmZILTA/view?usp=sharing

So as Finder app (native file browser in Mac) shows — pink. So as Footage for Mac app shows it in the app — https://drive.google.com/file/d/0B47FS5gpVSwOeHptZmxkNWtGQkU/view?usp=sharing , and how it converts MLV to MOV video — pink. https://drive.google.com/file/d/0B47FS5gpVSwONG9WSWlRV3kyUE0/view?usp=sharingSo i didn't even think to open DNG in Photoshop and Lightroom because everywhere else whites are pink.

I read somewhere on this forum, A1ex replied to someone on this pinkness, but i didn't understand that, so i figured that this is the temporarily downside of 3.5K. By the way, i suppose pinkness happens only in high-resolution video (crop mode)... As i recall. Also i thought that this Footage for Mac app doesn't remove this pinkness, but i read somewhere that MLV_dump does. Maybe i got it wrong. But wanted to try.

Maybe over-exposition is my fault during shooting, i'm not a professional photographer or videographer, but that's how i saw it on camera screen.

Adjusting in Lightroom shows that this photo is not fatally overexposed, all details are kept. So i don't know what to think about it (what is the reason):https://drive.google.com/file/d/0B47FS5gpVSwOWENZWW5sZU9VV0E/view?usp=sharing

This is the MLV, video is 11 sec of static view, but you can see the clouds are slowly moving:https://drive.google.com/file/d/0B47FS5gpVSwOOEpDN19xeXIxM0E/view?usp=sharing

It was 3.5K according to instructions in this thread above from @HJfilmspeed https://vimeo.com/217313287 so it's 12 bits but the Footage app says it's 14 bits.

Here is a sample package of the 1gb provided from whysodifficult if anybody wants to check. https://bitbucket.org/Dannephoto/magic-lantern/downloads/M30-1838_samples.zip

The quick look previewer in mac doesn´t get the white balance level at all. See the last example. The other two is how DaVinci resolve and adobe camera raw interprets the dng when opened up in each program. (Added dcraw -T -w output as well).

I check out the 3.5k file that @whysodifficult posted on my PC Windows 7 Pro desktop for only WB issue .I extracted the dng's with the latest mlv_dump.exe from the experimental download page and every thing looks right .

No adjustments , just loaded in to A.E. CS6 PC/Win7 . The WB seems to be 5500K , looks ok to me(https://c1.staticflickr.com/5/4257/34822932030_12fe396ce3.jpg) (https://flic.kr/p/V4bCtQ)Test_3.5k_5d3_A.E.CS6 (https://flic.kr/p/V4bCtQ) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

I was very impressed with the image , I started to push the image around with ACR and the image really pop out .

I started with "Camera Neutral" & 7250K WB , the .XMP from A.E. is below I also exported 50Mb Mp4 H264 @ 35Mb/s link--> https://drive.google.com/file/d/0B3rFQHNAb91MQWxGUWNxM05WX2s/view?usp=sharingI was amazed how much sharpness it could handle , really made the fine detail come out (very nice) :D

For all win users -- I been looking at different ways to use mlv_dump with out the cmd line interface as some have a issue with that .So in a nut shell any of the old PC app that use mlv_dump , just replace it with the latest ver. from the experimental 4k crop rec download page.It works with MLVBrowseSharp & MLVViewSharp MLV Viewer to view .mlv (and .raw) files on windows (C#) (http://www.magiclantern.fm/forum/index.php?topic=8447.msg77759#msg77759) FYI this is cross platform (PC & MAC) . I really like this old app it just never got finished being develop , it very powerful you can browses to any file location and extract dng's or export as legacy .Raw .The reason I being that up is if you do convert to .Raw app's like MLVProducer (http://www.magiclantern.fm/forum/index.php?topic=15271.msg148538#msg148538), ML Raw Viewer 1_3_4 (http://www.magiclantern.fm/forum/index.php?topic=9560.msg132429#msg132429), can then work with the new UHD/4k resolution's when shot with compressed raw (it will be a bigger filer thou with uncompressed14bit) . I should mention with MLVBrowseSharp there's no thumbnail preview but does extract the dng's correctly by a right clickof the mouse hovering over the file and MLVViewSharp there is pink highlight (works with converted .Raw , .MLV compressed raw error out) the great thing about MLVViewSharpis you can change the raw debayering down to 1/8 of the resolution for quick playback plus many other image processing , LUTs etc. .... .The other one I check out was MLV Converter 1.9.2 (http://www.magiclantern.fm/forum/index.php?topic=10198.msg98284#msg98284) , replaced mlv_dump and I could export dng's , 1080p proxy all at the same time , I can even export with Lossy compressionwhich gave me a 1.35MB 3.5k file verse a 8MB uncompressed 14bit frame . A.E. ACR see them and can be imported but not in to Resolve 12.5 . I tested the Lossy dng compressed to 1.35 MB pre-frame in A.E. and I could not tell the different between the compressed or uncompressed raw file.Hope that helps a little.

@pc_bel you need to convert .mlv to .raw legacy , unless the developer "g3gg0" is willing to update to support compressed rawthat's the workaround at the moment . FYI you could request an update in his thread (http://www.magiclantern.fm/forum/index.php?topic=8447.msg77759#msg77759) ;)

One workaround previewing any MLV footage is going through MLVFS. Once the virtual dng files are created MlRawViever will be able to read the dng sequence. Just right click a dng file and select MlRawViewer.

I did not know the existence of MLVViewSharp, thanks you reddeercity.But do you know how extract DNG into separate folders ? All the DNG are exported in the same directory.

Just use MLVBrowersSharp , right click on DNG+wave then it open to a window to save , (see next image)(https://c1.staticflickr.com/5/4197/34863292500_20ff8c061b_n.jpg) (https://flic.kr/p/V7Kuf1)1 (https://flic.kr/p/V7Kuf1) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

Select which drive or folder you want.(https://c1.staticflickr.com/5/4195/34441432383_18e40fb1e8_n.jpg) (https://flic.kr/p/UttkVH)2 (https://flic.kr/p/UttkVH) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

FYI: 10-12bit uncompressed support with metadata , this is how it should work :D(https://c1.staticflickr.com/5/4259/34863459760_e1bcd26f30_n.jpg) (https://flic.kr/p/V7LkXN)10_12bit_5D2+metadata (https://flic.kr/p/V7LkXN) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

One workaround previewing any MLV footage is going through MLVFS. Once the virtual dng files are created MlRawViever will be able to read the dng sequence. Just right click a dng file and select MlRawViewer.

Yea I know , but way to slow especially with UHD/4K it even lags in full HD with mlvfs .On PC's best option for compressed raw is to convert to .raw legacy and use the existing PC apps .There's more reasons to use MLVViewSharp then just for viewing the files .MLVProducer developer was thinking of supporting compressed raw in his app , until then work arounds

Just use MLVBrowersSharp , right click on DNG+wave then it open to a window to save , (see next image)(https://c1.staticflickr.com/5/4197/34863292500_20ff8c061b_n.jpg) (https://flic.kr/p/V7Kuf1)1 (https://flic.kr/p/V7Kuf1) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

Select which drive or folder you want.(https://c1.staticflickr.com/5/4195/34441432383_18e40fb1e8_n.jpg) (https://flic.kr/p/UttkVH)2 (https://flic.kr/p/UttkVH) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

Thanks for your answer, but as you can see in my picture :

(https://image.noelshack.com/fichiers/2017/24/1/1497262305-batch.jpg)

all my DNG sequence are mixed in a same folder; i see you are using windows 7, maybe this issue is only about windows 10.

Yea I know , but way to slow especially with UHD/4K it even lags in full HD with mlvfs .On PC's best option for compressed raw is to convert to .raw legacy and use the existing PC apps .There's more reasons to use MLVViewSharp then just for viewing the files .MLVProducer developer was thinking of supporting compressed raw in his app , until then work arounds

On a decent hardware there is no lag on PC (w/o antivirus installed) and there is no need to use legacy RAW, you can just uncompress (with appropriate mlv_dump) to MLV again and use whatever tool you desire (mlrawviewer etc)

I don't know what I'have done with the latest build (pretty sure a bad choice in bit depth in the hurry) but half of my converted mlv's are just blacks dng (latest mlv dump).I'm asking just to be sure, but I think this shots are lost?

A sample of the problem:https://mega.nz/#!vxkkGQJT!8GjEWu1dzUq3yVL7siRQI_u7TuPnFIm-vDmFCELu594

Black or just darker? You´re uploaded file has a dark preview image but opens just fine in acr or when converted to a tiff with dcraw. I get those darker files as well. Not sure why but it seems related to 10/12bit and only preview.

Really, it open fine in acr??I was pretty upset by the black preview I come here too quickly...(You're right, all open fine in acr, it's my viewer, picasa viewer the culprit)!Thanks for the quick reply Danne!

FYIwhen fetching the latest mlv_dump (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/) and some recent mlv_lite / mlv_rec, you can use this to determine the exact build version:

Could you look in to the mlv_dump from experimental. When I convert 60P 3x3 Footage I still have a lot of Vertical Stripes in highlights, even though mlv_dump is doing Vertical Stripe Fix, I am wondering if the code is not fitting for the 60p 3x3 mode or maybe there is an offset which enhances vertical stripes?, Maybe because of the 4 dead vertical rows on the left of the frame (also a bug btw). Especially if I have lens flare or direct sunlight flaring in the lens, then the Vertical Stripes are more visible than anything I've recorded with normal 1920x1080 MLV 3x3. What could be the cause of it?

When I play the footage in mlvrawviewer and turn on Vertical stripe fix in that program, sometimes that will get completely rid of the stripes and other times it will make it worse. Its really weird to work with.

The heavier the grade, the more visible the stripes. To make MLV footage look really good, you have to push footage so much that it crosses that threshold where the stripes become really visible. Temporal Noise reduction will also at times enhance it even more, as the algorithms "think" its important information to keep.

Here is a cropped screenshot from mlvrawviewer with a LUT pushing the image from a log curve, to show you the severity of the stripes.

As you can see the, flaring enhances the vertical stripes a lot and in this shot its only part of the image, sometimes the entire image will be engulfed in flare and everything turns to Stripes.. I could not find my shot where I had a full flare, aber its egal, same thing.

Also, it would be really nice if you could add Allow Global Draw - OFF during recording to MLV lite as in the vanilla MLV. Because when I am pressing half shutter to remove overlays, I am getting vastly improved recording times, almost double the time and sometimes continues (depending on scene).

and another bug to report from 60p 3x3, when selecting ISO 100 the Raw Zebras become disabled and the Histogram maxes out at about 90% the way to the right, the channels don't go beyond it, leading the operator to believe that one is well within ETTR range. The non-raw spotmeter also maxes out at 96% (not sure if also at other iso's).

Also a general question, why is it when hitting record button, that the recording starts so "sluggish", like 1-2 secs after, does it need to write information to a buffer or something, is it not something that could be done before hitting record, I mean just by entering the mode and keeping it Armed for recording? Also when recording stops, it takes about 4-6 seconds before I can hit record again, maybe longer.. It is quite counter productive, when one wants to shoot a lot of bursts at times and having to wait these combined almost 10 seconds per clip. Is it something one can expect some improvements on in the future ?

In this particular case, I believe the vertical stripes are there because - just a guess - the first frame probably did not have relevant highlights to be analyzed.

Another approach that I didn't implement, but only documented the backend support for it (http://www.magiclantern.fm/forum/index.php?topic=17795.msg171595#msg171595): it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.

Quote

Allow Global Draw - OFF during recording to MLV lite as in the vanilla MLV

This one was a major source of confusion, so my previous answer (http://www.magiclantern.fm/forum/index.php?topic=5533.msg93763#msg93763) is still valid.

Quote

recording starts so "sluggish", like 1-2 secs after

Sorry, not sure what you are talking about. Keep in mind I'm not a regular user of this mode (actually I don't remember touching the camera in the last few weeks...)

Is it the delay between you pressing the record button and the first recorded frame? Or the lack of responsiveness on the user interface?

Quote

I mean just by entering the mode and keeping it Armed for recording?

Pre-recording does exactly this.

Of course, the preparations can be done in background, with extra care when switching video modes, or when turning off raw recording. It's doable, but will increase the code complexity (although, probably not a bad idea, since some of the preparations are done in background in order to estimate recording times).

Quote

it takes about 4-6 seconds before I can hit record again, maybe longer

Set "Show graph: Buffers" to see what happens. All the frames that were recorded to RAM before you pressed Stop are now being saved to card (about 300MB iirc).

Quote

Is it something one can expect some improvements on in the future ?

If, in the future, you will select a resolution that gives continuous recording, then yes.

If we find some way to allocate more memory (the 5D3 has 512MB), this duration will increase (along with the recording time).

One can get a minor improvement (maybe speeding this up by 1-2 seconds) by pausing LiveView during this process. The silent picture module uses this trick, so it's an extremely easy coding task for those interested.

Quote

another bug to report from 60p 3x3, when selecting ISO 100 the Raw Zebras become disabled

In this particular case, I believe the vertical stripes are there because - just a guess - the first frame probably did not have relevant highlights to be analyzed.

Another approach that I didn't implement, but only documented the backend support for it (http://www.magiclantern.fm/forum/index.php?topic=17795.msg171595#msg171595): it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.

You are exactly right, first few seconds of that clip has no flaring.

I'll take a look in to the rest of your reply tomorrow and the calibration.

Another approach that I didn't implement, but only documented the backend support for it (http://www.magiclantern.fm/forum/index.php?topic=17795.msg171595#msg171595): it would require pointing the camera to something like a gray wall and running a calibration routine that would adjust the column gain registers (8 multipliers) until the artifact disappears. Might be worth trying.

Read the thread again and I've been so interested in Darkframe Averaging and flatframe since the talk started here a long time ago, but as a Windows user there is no GUI converter with these options. Unfortunately MLVFS does not work properly with AE and AE is at the core of my workflow. Can you link me to a thread that explains how to type in the different codes/commands for Darkframing and Flatframing in mlv_dump, I cannot find any good relevant thread or post, there are so many search results of mlv_dump, but if you know where I should look, that would be nice, if not I'll keep looking.

Quote

This one was a major source of confusion, so my previous answer (http://www.magiclantern.fm/forum/index.php?topic=5533.msg93763#msg93763) is still valid.

Yeah, I just thought this 'Allow Global Draw - Off' was a specific "component" easy to port. I know what "Allow Global Draw - Off" means because I know what it does, but the wording of it makes no sense to me and even more confusing for newcomers.

Quote

Sorry, not sure what you are talking about. Keep in mind I'm not a regular user of this mode (actually I don't remember touching the camera in the last few weeks...)

Is it the delay between you pressing the record button and the first recorded frame? Or the lack of responsiveness on the user interface?

Pre-recording does exactly this.

Of course, the preparations can be done in background, with extra care when switching video modes, or when turning off raw recording. It's doable, but will increase the code complexity (although, probably not a bad idea, since some of the preparations are done in background in order to estimate recording times).

Yes, recording starts 1-2 seconds after pressing record. I've tried with Pre-Recording and it does counter it but only after it has made the Pre-Recording which in it self takes 1-2 seconds to start recording aswell. On top of that, I have to activate the Pre-recording and then I have to press Record again, something that in the "heat of the battle" can get in to kind of an opposite effect where I record when I think I stopped recording or I stop a recording when I want to start it (happened a bit too much for me in 24p Mode.. and I only found out when I checked the footage ;D ) I don't know, maybe I just need to get more used to it.

But with Pre-Recording I am getting less recording time, I usually get around 400/500 to 700 Frames and with pre-record 1 sec I get 300-450's. I use the lowest amount 1 sec pre-record (60 Frames), maybe you could add 1 frame pre-recording specific for the 60/50p mode as a standard, just so that recording starts immediately as an "easy" workaround? Not sure if easy or not. The Background thing you mentioned, sounds even better.

I can't give any technical explanation on why the recording times are lower, maybe I've just been unlucky with the compression, I must add that I did not test it extensively. Just done a few shots with pre-record in scenes where I was expecting longer recording times than the 300 to 400s.

EDIT: Just did a few more tests, switching between Pre-Recording and no Pre-Recording. The recording time is almost halved with Pre-Recording 1 Sec.

And you not touching your camera or this mode, you really have to try out the 60p mode, it is absolutely amazing! It has given my camera a new spirit. I thought I would use it more professionally but for filming family and those speedy little kids, its amazing.

Quote

One can get a minor improvement (maybe speeding this up by 1-2 seconds) by pausing LiveView during this process. The silent picture module uses this trick, so it's an extremely easy coding task for those interested.

I don't think I would mind the LV freezing during that period, I am unable to do anything in that time anyway, so if it shortens the wait time it sounds good to me. Though I would suggest a small text appearing right before the freeze saying "Writing from Memory" or something like that, so users wont experience frustration over their camera freezing after every take and not knowing why if they didn't read this thread or pull.

Good point about flat frames - they are yet another option for correcting the vertical stripes (alongside with other defects, such as vignette or dirt spots on the sensor or on the lens), and the calibration can be done afterwards.

About Global Draw while recording - the difference I've got on a quick test is pretty small: about 450 vs 500 frames with Preview set to Frozen LV, or about 300 vs 350 frames with Preview Auto, tested in 10-bit uncompressed 1920x800 60p, on 1.2.3, with default overlay configuration - which includes CPU-intensive zebras. That's only one extra second with global draw off, and I'd expect a lower difference on 1.1.3, where zebras are hardware-accelerated.

But the averaging does not work with the 60P Mode files, I tried it on 24p 3x3 Vanilla MLV files and it worked. When I input a 60P file I get "windows has encountered an error" but Command Prompt does not exit or crash. It just generates an empty averaged.mlv

I have tried with short and longer recording times, 1 sec and 6 secs in case it was too little, but its the same. I had the range from 100 - 3200 ISO, 6 MLV files and none of them worked, not the 1 sec or 6 sec MLV's.

I have not tried Normal Lossless 24p 3x3 MLV's, to see if it has something to do with the Lossless mode, I will do that next.

I think the mlv_dump I used is the one from the 3rd or 4th of june (I dont know where to see older builds anymore with the changes on download page) , the one that is 759 KB in size.

@a1ex

I always shoot 14 Bit Lossless 60P 1920x800 with Real-Time Preview and I get about 500-700 Frames and with Pre-Recording ON it was down on 350 Max. Pressing and holding Half-Shutter almost always increase the recording time by atleast 2 seconds (real time). I say "almost" because sometimes but rarely the compression goes crazy, I have many times gotten 10-15 Seconds (real time) recording this way, where I just stop the recording because I probably got the burst shot I was aiming for. Like earlier when I was shooting Lens Cap on for the DF Averaging, first frame it crashed saying Compressed size was bigger than uncompressed. A completely black picture on ISO 100 and it could not compress it.

I have never tried 10 bit mode, but what I see from all the examples it is extremely noisy, perhaps that is why your recording times are so low, maybe the compression goes crazy from all the colour casts 10 bit footage introduces. But yeah.. 10 bit vs 14 bit, mathematically you should have higher record times than I am getting. no?

And why would the difference be lower on 113? I thought fir 113 always had higher performance than 123 because of the extra buffer.

But the averaging does not work with the 60P Mode files, I tried it on 24p 3x3 Vanilla MLV files and it worked. When I input a 60P file I get "windows has encountered an error" but Command Prompt does not exit or crash. It just generates an empty averaged.mlv

I have never tried 10 bit mode, but what I see from all the examples it is extremely noisy, perhaps that is why your recording times are so low

I've only used it in order to get constant bitrate. With lossless compression, the recording times will vary a lot with scene contents (so it's harder to benchmark this way; a static scene or a dark frame will make things a bit more repeatable in this case, but I didn't want the extra trouble.)

But yeah.. 10 bit vs 14 bit, mathematically you should have higher record times than I am getting. no?

No.

10 < 1410 > 14 * 60%

Quote

And why would the difference be lower on 113? I thought fir 113 always had higher performance than 123 because of the extra buffer.

The cause is lower CPU usage on both Canon's side (regardless of ML settings), and on ML's side (only when Luma-based zebras are used). Memory bandwidth usage might also be a bit higher on 1.2.3 on Canon's side (didn't check this one, as it's not straightforward to measure). Of course, absolute performance can be higher both with and without global draw, but the difference between these numbers is likely lower (that is, global draw even faster on 1.1.3) iff you use Luma-based zebras (which is the default). I didn't benchmark these cases; it's just what I'd expect to happen.

Yes, now it worked generating averaged.mlv and outputting DNG but it is a very tedious process, having to decompress every file individually, rename it, move it to the appropriate folder... So the next big question, how do I batch process this ? Some command I should know? :)

Or atleast batch the decompression part. After decompression I see all the Metadata, ISO, fstop and so on in the MLV's. That way I can sort the MLV's with the same ISO's to their appropriate folders and do the DFA from there.

What was the point about flat frames? It's just some coding stuff that didn't yet exist as some executable for Windows users? I'm really interesting about vertical stripes problem cause I have horrible ones (at the sky usually)

Please use search function. Flat frame or flatframe.The function exists in mlv_dump which is the same for both windows and mac.Flat frame workflow isn't trivial. I experimented with it some a while ago but there is more to test out.Here is one linkhttp://www.magiclantern.fm/forum/index.php?topic=17795.msg175189#msg175189

1. take a "flat field" video a) put a white shirt over lens b) point to the bright sky c) record using the same settings as before (same exposure time not strictly needed, but would keep the ISO)2. average that video using the -a option (e.g. mlv_dump movie.mlv -a -o flat.mlv)3. process the problematic footage and supply this flat.mlv as flatfield (e.g. mlv_dump in.mlv -t flat.mlv -o out.mlv)

Edit 2: Changed to 45p now worked a bit better. Got 7xx frames before stoped.

Edit 3: You who managed to record 45p and used it in a NLE, how much did you slow it down? Since 48p is really simple with 24p timeline. I tried to slow it down with 53% and it seemed ok. Any thoughts?

Edit 3: You who managed to record 45p and used it in a NLE, how much did you slow it down? Since 48p is really simple with 24p timeline. I tried to slow it down with 53% and it seemed ok. Any thoughts?

I think the easiest (in Premiere / After Effects) would be to conform the clip (using the Interpret/Modify command) to 23.976 or 25p depending on your edit needs. Then you won't have to figure out the exact right clip slow-down rate and it will probably be cleaner looking than slowing the clip back down with the NLE, which runs the risk of blending frames or having to generate new ones if it's not dead-on. Hope that made sense :)

I think the easiest (in Premiere / After Effects) would be to conform the clip (using the Interpret/Modify command) to 23.976 or 25p depending on your edit needs. Then you won't have to figure out the exact right clip slow-down rate and it will probably be cleaner looking than slowing the clip back down with the NLE, which runs the risk of blending frames or having to generate new ones if it's not dead-on. Hope that made sense :)

That made excellent help! Works good in ae also. What shutter speed do you use? At first I left it at 94, but will probably go up to 180 next time? As one can see its quite blurry, also shot from a train...:(http://thumb.ibb.co/doMb55/Comp_1_00884.png) (http://ibb.co/doMb55)

(http://thumb.ibb.co/k5a8JQ/Comp_1_03840.png) (http://ibb.co/k5a8JQ)

Also, anyone notised the black pixel line in bottom and to the left? If so anyone made it correct?

I am thinking if the 60p mode is stressing the sensor so much that it is making the Vertical Stripes more prevalent than 24p 3x3? Is that a possibility? I am saying this because normally I didn't have issues with Vertical Stripes when I was shooting 24p, sometimes I would get some in highlights after heavy grade, but after getting addicted to 60p I am seeing a lot of shots with Vertical Stripes.

And I am not seeing Vertical Stripe fix working properly with 60p footage. These shots have the same exposure and lighting across the entire recording and yet the Vertical Stripes are still there. In the first screenshot its harder to see the the lines, but when playing the recording as this is a Slow pan shot, it becomes very visible. in shot 2 its much more visible, I also zoomed 200%.

Shot 1https://mega.nz/#!QdpETRZR!_y6mdX7ZWGkVisIamJLLAeyss9uzdsEdMtn3TgETlTc - Screenshot from AE - ISO 200 The jpeg compression is hiding it in this shot, look furthest to the left, some remnants of the stripes are there.

FYI: In this case, activating Stripe removal on the DNG's in MLRawViewer removed the stripes, but introduced big chunky vertical lines from all the shadow parts of the image.

This same shot without Darkframe Averaging makes the Vertical Stripes less prevalent, but still there, its as if the deep blacks from Darkframe averaging is enhancing the stripes or the contrast of the stripes.

My point with this post is that I think that mlv_dump stripe removal might need some optimization for 60p mode, I might be wrong and doing something wrong in the process introducing this problem myself, I'd love to be proven so. But its a big issue for the 60p mode.

Just a guess here, but when decompressing, is it also doing vertical stripe fix? and then converting to DNG the same process is done again introducing the stripes again? I don't see any text saying so when decompressing, but just a thought.

Thanks for your answer, but as you can see in my picture : all my DNG sequence are mixed in a same folder; i see you are using windows 7, maybe this issue is only about windows 10.

Ok --I see the issue .So to anyone that want to use the new mlv_dump.exe on Windows platform to batch your UHD/4K/10-12bit & compressed14bit mlv's in to separated file folder.First I thought of making a .bat (win7) & or .cmd (win10) (really the same thing) but I had problems with importing more one file at a time(to much code and getting too complex) . Then I considered building a simple .exe batch program , then it dawn on me why I'm I trying to re-invent the wheel here ::) , so after a quick search I found a Older Win's app that uses mlv_dump.exe . MLVConverter1.9.2.zip (https://www.dropbox.com/s/7qhncp7pi4t268o/MLV%20Converter%201.9.2.zip)MLV Converter 1.9.2 for WINDOWS (https://www.magiclantern.fm/forum/index.php?topic=10198.msg98284#msg98284) You will need some other tool if you want to have it fully functional --Adobe DNG Converter , mlrawviewer (only to view file not working work compressed raw) & IrfanView .All I did was to replace the mlv_dump that there with the new mlv_dump from the experimental download page , installed the latest Adobe DNG Converter (to output compressed dng's)and IrfanView . I tested 14 mlv's files which included 10-12bit , 14bit, UHD(3520x1320) compressed raw 14bit and all exported fine in to the respected folders . The dng's import ok in toAfter Effect CS6 (Wins) didn't try any other app . *Note* in the tools folder you have (3) mlv_dump's (mlv_dump , mlv_dump1 & mlv_dump2) just copy the new mlv_dump and re-name it mlv_dump1 etc. .... that's it should work fully from there with 1080p proxy you can add a customs .xmp file from A.E. and have it export to ffmpeg prores4444 if you wish . If you want to export Dual ISO you need to update cr2hdr.exe to the latest windows version.One thing windows will ask permission to run mlv_dump on every file unless you go the properties and in the general tab check "UnBlock" at least on Win7 , win10 should be the sameHope that helps :DEdit: update exiftool.exe also(https://c1.staticflickr.com/5/4228/35290258021_e3e801ee01.jpg) (https://flic.kr/p/VLtNdv)mlv raw converter1 (https://flic.kr/p/VLtNdv) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr(https://c1.staticflickr.com/5/4282/34610387673_b85bb228b0.jpg) (https://flic.kr/p/UJphrX)mlv raw converter2 (https://flic.kr/p/UJphrX) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr(https://c1.staticflickr.com/5/4260/35290258091_b3a956b54a.jpg) (https://flic.kr/p/VLtNeH)mlv raw converter3 (https://flic.kr/p/VLtNeH) by RedDeerCityTV (https://www.flickr.com/photos/[email protected]/), on Flickr

But... I'm on windows 10 now and mlv_dump processes mlv files in separate folders when I select all and use the "send to" comand found here http://www.magiclantern.fm/forum/index.php?topic=10526.msg102123#msg102123 (http://www.magiclantern.fm/forum/index.php?topic=10526.msg102123#msg102123)

That made excellent help! Works good in ae also. What shutter speed do you use? At first I left it at 94, but will probably go up to 180 next time? As one can see its quite blurry, also shot from a train...:(http://thumb.ibb.co/doMb55/Comp_1_00884.png) (http://ibb.co/doMb55)

(http://thumb.ibb.co/k5a8JQ/Comp_1_03840.png) (http://ibb.co/k5a8JQ)

I usually go by the 180-degree shutter rule of thumb, where the shutter speed is 2x that of your framerate. 24fps = 1/48 (or 1/50), and so on. So if you are shooting 45P, 1/90 (or 1/94 if that's what is available) on the shutter should be a good starting place. If it's still too blurry for your liking, you could keep pushing it further like 1/120 or 1/250 and see if you like that additional clarity.

tried the flat (https://www.magiclantern.fm/forum/index.php?topic=19300.msg185962#msg185962) field (http://www.magiclantern.fm/forum/index.php?topic=17795.msg175189#msg175189) videos to get rid of stripes?tend to be the most reliable ones, as they are not just guessed from one frame and also care for speckles etc

Current conditions are not ideal to make a flat frame. But I will try it out, for sure.

On another note, the --force-stripes has done a terrific job at removing the stripes, there are stripes, but virtually invisible. But it is slow.. Very slow.. Also with no batch process I am seeing a lot of issues ahead. I have a music video coming up, that I wanted to shoot all 60p, but with so many manual steps, decompressing and sorting, converting and having to stay glued at the screen to start next process when one is done, I am feeling like this is way too much to handle. Blaaah..

I hope the jpeg compression will do it enough justice for you to see. I did not do darkframe subtraction on the --forced-stripe footage. Did not look in to how I could do both with the Steroids_dump

Do you think the flat frame would be a quicker step ? It will also require sorting, even more so I assume, each lens to its own? each f-stop to its own ? Darkframe+Flatframe sounds like the ultimate quality we can get with our Canons, but it seems overly complicated for a project shot with multiple lenses at various f-stops and having some manual lenses, it will be hard matching f-stops to the right flatframe and on top of that having to write in the commands for each shot is very tedious. For photography, it seems like a no-brainer though.

Flatframe for each lens is needed. Not sure of the ideal f-stop when it comes to flat frames. I had good results with around f5.6. I used light from a strong in door lamp. Pointing to the sky is probably better.Since you are getting better results with --force stripes with the steroid version it indicates that mlv_dump is either using different code or isn't always on. There are newer and older vertical stripes code so it could be an idea to compare those maybe? I think steroid version is using what is used in mlvfs(older mlv_dump code), havn't checked.

With some trickery I use a HDR AE template in cr2hdr.app which goes through aerender(command line render engine in after effects). In main menu select (p) then (23) for a template. Your MLV files have to be transcoded into folders with dng files through cr2hdr.app.

There is a bug with 1920 50/60 3x3 setting. Once you exit liveview after filming you can´t open liveview again without getting a completely black screen. Exiting the selected mode opening up liveview and reselecting the mode will make it work again.

Tested the 3.5k crop mode 3584x1320 on the lowest compression setting (11-8bit) Lossless, to see how it holds up, pretty pleased with how stable it is, no dropped frames or automatic stoppage's despite shooting some highly detailed scenes, it also preserved excellent colour detail too, fantastic job Alex.

As usual liveview is a challenge to work with, but workable with some extra attention, also there one clip that is standard 1920x804 cuts very well with the crop clips, see if any of you can guess which one it is?

I assume that maps to this PR: https://bitbucket.org/hudson/magic-lantern/pull-requests/837/mlv_snd-and-dual_iso-support-for-mlv_lite ? If so, would have some testing done help at all? Happy compiling and checking it out on the 5d3 if that's of use.

Just tested the June 19th Build, it fixed the ISO 100 Raw Zebras bug and Histogram bug. Thank you very much!

Not that I use the h264 Spotmeter for exposure, but just tested it aswell. Instead of maxing out at 96%, it maxes at 97%. Don't think it matters for the 3x3 60p mode, its Raw only right? just so you know. Personally I don't use h264, ever.

Tested the 3.5k crop mode 3584x1320 on the lowest compression setting (11-8bit) Lossless, to see how it holds up, pretty pleased with how stable it is, no dropped frames or automatic stoppage's despite shooting some highly detailed scenes, it also preserved excellent colour detail too, fantastic job Alex.

As usual liveview is a challenge to work with, but workable with some extra attention, also there one clip that is standard 1920x804 cuts very well with the crop clips, see if any of you can guess which one it is?

That sounds really cool, so if I understand correctly I could measure the Kelvin Temperature instead of exposure? Does any camera do that?, I never even thought about that possibility. Would it measure on a RAW basis or depending on the liveview "colours", could a Picture Profile skew it?

So compiling the PR for audio (with the 11 commits to crop_rec_4k since it was opened), gives me the following on 5D3.113:

- Modules enable fine and I can enable mlv_sound in the sound menu- Sound records in MLV videos in full-sensor mode- Sometimes when hitting stop, there is an error message on screen stating that "sound did not stop: state 5", however I can't reliably recreate it.- There seems to be significantly more write speed required than "advertised" when sound is on - my slower Sandisk Extreme 120 card can't keep up for more than 400 frames with sound, when it can go continuous compressed raw without. Lexar Pro 1066x card is fine at 3k 2.39 though.

I'm a bit unsure of what the outcome should be when recording sound in the new crop_rec mode. There is a WAV output at the end of the run with mlv_dump, but it is only 44 bytes on crop_rec videos. Still new around here - is mlv_dump another compilation target within the magic lantern repo? I'll get googling.

Setup a 4k timeline within FCPX, imported the 3584x1320 MLVs and used Color Finale Pro to add alexa LOG C LUT, from lut utility within color finale, then used the colour wheels and vectors to colour grade.

I was reading in the MLV_Play thread that disabling raw_twk should make playback work, I tried that and it did not work on 60p mode, "No decompression in this mode".http://www.magiclantern.fm/forum/index.php?topic=9062.msg186484;topicseen#msg186484 the thread.

Does raw_twk affect recording in any way? I've just had it on all the time since lossless was released, I forgot about it, does it have any other purpose than speeding up playback? (which was it's original intent, AFAIK)

Since I only use older lenses (nFD glas), around 30-40 years of age. Would it be better to choose a mode that uses the center parts of the sensor more than the higher part of lens glas? In my experience the UHD options render only from the top part of the sensor while center, yeah is center.

Any thoughts on which options is best to use, I mean in terms of lens qualities and DNGs to use?

Option 1: UHD 1:1 3232 x 1376Center crop 3,5k: 3232 x 1376

From a sequence I shot today I experienced a lot of CA and bleeding. Maybe it's just the lens quality?

@bastonford, so glad you're compiling this! Good luck with it. For either run & gun with a Rode VidPro or for dual system audio having the WAV recording in the 5Diii is huge help. It'll add years of usefulness to my camera.

Great to hear you're getting continuous 3k and sound.

I stopped playing with card spanning a long time ago but I always thought it'd be great if the audio could write to the SD card while the video wrote to the CF card. I'm guessing there are underlying Canon issues keeping that from happening or that it doesn't help write speeds at all.

So compiling the PR for audio (with the 11 commits to crop_rec_4k since it was opened), gives me the following on 5D3.113:

- Modules enable fine and I can enable mlv_sound in the sound menu- Sound records in MLV videos in full-sensor mode- Sometimes when hitting stop, there is an error message on screen stating that "sound did not stop: state 5", however I can't reliably recreate it.- There seems to be significantly more write speed required than "advertised" when sound is on - my slower Sandisk Extreme 120 card can't keep up for more than 400 frames with sound, when it can go continuous compressed raw without. Lexar Pro 1066x card is fine at 3k 2.39 though.

I'm a bit unsure of what the outcome should be when recording sound in the new crop_rec mode. There is a WAV output at the end of the run with mlv_dump, but it is only 44 bytes on crop_rec videos. Still new around here - is mlv_dump another compilation target within the magic lantern repo? I'll get googling.

I'm still a little stuck on this - Footage and mlv_dump seem pretty convinced there is sound in there for higher resolutions looking at the metadata, but when exported it's just an empty WAV file. I'm still trying to work out if this an in-camera thing or mlv_dump that isn't doing the right thing.

I have to say though, on the 5d3, the FF 1080p 14bit is stunning quality if you de-bayer it onto a 4k timeline in Resolve, rather than upscaling the video at the end of mastering it. Obviously 4k is way slower to work with, so I tend to edit in on a DCI2k 2.39 timeline, and then just flick the timeline resolution up to the 4k option before delivery. Not saying that higher resolutions aren't necessary, but the most exciting thing for me is 14 bit compressed RAW with audio - a ~45% space saving makes a huge difference on bigger projects!

This is with a Lexar 1066 card, and it seems to run fine for about 8-10 seconds and then bug out with that error dumped to the card. This is mlv_sound module enabled and the module turned on in the audio menu. On the normal crop_rec branch the 3k is continuous without any issues.

I tested this experimental firmware and that's amazing the video I can get from my "old" 5d mark iii, a big thanks to the community and THE developers.

My only concern is about missing mlv_snd module which is for me the reason I won't keep this firmware on my camera right now but I've seen that you are working on a subbranch which add this module, is it already somehow working or blocking issues are still to be fixed ?

I would have love to help (being myself a developer but more web/mobile oriented), I dont have the level I guess to dig into C/Perl code base.

Can you direct me where I can download the update to record sound on the lastest build? I dont know how to work around bitbucket.I basically want to record sound in regular 24p 1080p but I want the 50p 3x3 option from the latest update (I dont need sound for this option).

@The_bald_guy https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip)This is for 5D3 fir 113Oh wait sorry I'm not sure if this version records sound. I missed that you needed that. I think you have to compile it if you want sound.

@whysodifficult @rob_6This video is not for those new to ML. Try this at your own risk. I am not responsible for any damage this video may bring if your camera explodes or bricks. Magic Lantern is not responsible either. It will probably just brick not explode though.Magic Lantern 3072 2.35:1 12 bit Lossless RAW Recording from the CROP_REC_4K build from 4-29-2017Once you have ML installed and the correct build on your SD card, this video should help you set up your 5DIII to record 3K almost continuously depending on the scene. Remember ML on SD and Rec to CF.Remember this build is experimental and not for those that are new to ML. Try at your own risk.

Has anyone noticed bug on x5 crop_mode (3,5k) - sometimes center adjusted to the right. I mean you have an object almost perfectly at center and when you see a footage, it's on the right side of picture. Yet can't figure out what it depend of.

@The_bald_guy https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip (https://builds.magiclantern.fm/jenkins/job/crop_rec_4k/36/artifact/platform/5D3.113/magiclantern-crop_rec_4k.2017Jun19.5D3113.zip)This is for 5D3 fir 113Oh wait sorry I'm not sure if this version records sound. I missed that you needed that. I think you have to compile it if you want sound.

hey @hjfilmspeed is there any thread which shows how to compile a module? I'm looking for sound on the crop rec as well, happy to test out. cheers.

Totally dumb question (but to better understand what is going on): what are the differences between 3.5K 1:1 centered x5 and 3K 1:1 for the end result? Why using the first one over the second or vice versa?I have managed to record 3072x1286 - 2.39:1 files with the two with correct LiveView Preview (Frame mode / lagged) and very close framing (just few pixels differences)...

Stop trolling around, etienne.This project is done by highly skilled people in their spare time not consumed by work, family, friends, household duties and other hobbies. For free, given to you free (as in beer). There is no schedule, time table, milestone to be fulfilled. Top of page -> User Guide -> FAQ -> Troll questions sections

If you can't live with that you are free not to use ML but professional solutions more suitable to your needs.

Hi what CF card speed needed at 5d3 to make 4k video? can use SD cards ?mine is CF 90mb 64gb only and i don't see any crop_rec_mo inside my latest magic lantern? my version is build on 2017-08-07, anyone can anwer me tqvm

Ok Sorry if i did not make myself serious enough. Just want to assure you that my question was definitly not about the when?

but just asking (to whom have taken a look into it) is this a reasonable hope to have? or do there is evidence already that it could turn out to be something like the RT preview with a big question mark on if this is even doabble...

but rest assure i can definitly live without just as i can definitly live without even getting that answer.

I followed your video with the exact settings but encountered 2 errors described below. I also then tried Full-resolution LiveView: 5796x3870 at 7.4 fps and got the same errors. The attached images are the settings for "full-resolution LiveView: 5796x3870 at 7.4 fps"

For both scenarios above the movie recorded and I extracted the DNGS using MLV convertor but could not open the DNGS in Photoshop CC 2017. -The error was "Could not complete your request because the file-format module cannot parse the file." , see attachments- In bridge I can see each DNG file has many dots of different colors. ." , attachments

I followed your video with the exact settings but encountered 2 errors described below. I also then tried Full-resolution LiveView: 5796x3870 at 7.4 fps and got the same errors. The attached images are the settings for "full-resolution LiveView: 5796x3870 at 7.4 fps"

For both scenarios above the movie recorded and I extracted the DNGS using MLV convertor but could not open the DNGS in Photoshop CC 2017. -The error was "Could not complete your request because the file-format module cannot parse the file." , see attachments- In bridge I can see each DNG file has many dots of different colors. ." , attachments

Try to turn off fps override, i've never used it when i shoot in full resolution

I normally use Full RES DNG photo mode for extreme macro to save on my shutter count as some of these stacks take 100's to 1,000's of pics.

I am trying to do the same but using 4k video to save time. The example below is a reduced size macro. The full Res version is 10,467 pics Panorama (16,424 X 18,674 ) using magic lantern and perhaps a few actuations only. So my desire to get the 4k video Full Res working as that took nearly 2 weeks elapsed time in photo mode.

I did as you suggested but nothing occured ie no DNGs created but not sure what I did was correct with your utility.

I copied the utility into the folder with the MLVsI am not sure what to do but at the cmd prompt, I selected 2 numbers, #6 then pressed return, then selected 11 and pressed return key. See the attached black screen image. Then I pressed "R" key and pressed enter... It seemed to do something. See the second screen capture. I copied the original MLVs into a folder called Originals and created 7 folders in the red box that I number as #3 however there are no files in it.

Thanks Danne, I will upload to dropbox, may take .wow, its very slow here, saying 1 hour left

In the meantime, base on the settings I used, is the Aspect ratio correct for the Full Res video of 7.4FPS. Alos I used Frozen LVWhen I hit the play button I cannot see anything in Liveview on the camera, just pink scrambled linesIs that supposed to be the behaviour.

Could be some corrupted MLV file. Let's see. Important though that you're using the latest versions of mlv_dump when processing your files.Gonna be away from my computer for a while. Checking in tonight.

Downloaded the file and run the batch_mlv.cmd command. All files were exported into dng files.

Note! Try all very basic when things are cracking. For example. A more basic folder name might work instead of with spaces. Worth a shot. My script is very basic.*Can confirm spaces in root folder is not working atm with the script. Will check into fixing this.

After double click on batch_mlv.cmd this is my view. Then I just print r and enter and files are sent to a folder.(https://s30.postimg.org/g893xx91d/Screen_Shot_2017-08-13_at_02.35.35.png)

Wohoo Danne, you are a champ. it worked. So if I want to add parameters what is the syntax? at the prompt do I enter "6 11" without the quotes and press return to run parameter 6 and 11)

Once again thank you so much my friend. I am so happy.

With the Full RES video at 7.4fps, I notice that live view is scrambled with pick lines which make it hard for me to focus ( as can be seen in those example DNGs') or compose. What are your recommendations of the settings below for FUll RES? I think I am using 16.9 or 1:1 . Just charging my batteries.

Great :). Made me fix those damn folder space issues so all good.You can add parameters one by one. For instance 1 then enter, then select next setting and so on. When done select r and enter.Just reselect a setting number to erase it.Happy shooting :)

I just tested H264 proxy with sound + 1080p compressed raw and it works perfectly. However h264 are longer. to sync h264 proxy + sound + 1080 raw, I tried to determine the offset. It is around 2s but not regular. Sometimes +1i or 2, sometimes -1i or two. Is there a way to have a stable offset ? If yes, a simple ffmpeg script could sync everything.

I have shot some video on Full Res 7fps and able to extract DNGS ( thanks to Danne) however the liveView of the camera is showing pink and scrambled lines. Any way around this as I cannot focus or compose a shot?

I have shot some video on Full Res 7fps and able to extract DNGS ( thanks to Danne) however the liveView of the camera is showing pink and scrambled lines. Any way around this as I cannot focus or compose a shot?

Hope you're all well! I haven't checked in for a while. Last time I was here, devs were making leaps and bounds - all very exciting. I was just wondering where we're at with the 5D3? I think last time I tried the build 3.5k was almost continuous but without realtime live view or sound recording. Is this still the case?

Apologies, I did try and trawl back through this thread, but it gets quite busy!

I think last time I tried the build 3.5k was almost continuous but without realtime live view or sound recording. Is this still the case?

Mostly. I can do 3.5k 2.67:1 continuous with a fast card in the zoom mode crop. Live view preview is responsive enough for framing that mode, but I think the other hires non-zoom modes still have problems with live view. No sound. Very usable though.

I have tested the 1920x960 @50fps & I came up with a continuous resolution. 1920x818 @ 50fps gives continuous shooting & when you convert the vertical resolution (818) to 1080 it give a film like anamorphic look.Lots of thanks to the Magic Lantern team!!!I'm soon shooting a professional work with it.

Some days ago I took a few hundreds of pictures with the silent picture in Fullres LV mode, saved as lossless DNG. A few of them ended up corrupted - there were a few bytes of garbage data before the JPEG payload (as a result, these DNGs could not be decoded). Unfortunately, I wasn't able to reproduce the issue afterwards - that's where I need your help. The corruption may or may not be related to the silent picture module, or it may also happen with raw recording, or it may have something to do with quick button pressing and switching modes - no idea.

I've tried to record a timelapse of Fullres LV silent pictures - did not trigger the issue.

To spot the bug, I've added an assertion in 96c5204 (latest build). If you can trigger this assertion (easy to spot - it will fill the screen with a message about lossless.c:338), please try to find a way to reproduce.

Got some other bug. Havn´t been testing silent DNG almost at all but pretty impressive with lossless DNG. Scrolling between lossless DNG files with file manager is also great feature. Anyway. Tried producing errors and got this. A fake sized DNG. Assert.log included.https://bitbucket.org/Dannephoto/magic-lantern/downloads/Lossless_DNG_error.zip

Can be reproduced like this:1 - Activate silent.mo - open up live view in photo mode - shoot some lossless DNG files2 - Switch over to movie mode, shoot a CR23 - Switch over to photo mode again and start shooting some lossless DNG files. After a while it should produce the error. Maybe after 3-4 lossless DNG files.

I also had mlv_lite and RAW video set to on, EXPO. Override - Global Draw OFF. The other error code mentioned I still havn´t found.

Thanks Danne - taking a still photo (CR2) in movie mode was the key to reproducing the issue.

With silent pics (silent.mo):- start the camera in movie mode, make sure silent pics are disabled- take a still photo (CR2, not JPG)- enable silent pics (simple, lossless DNG)- take 2 silent pics using half-shutter- the second one will trigger the error in about 90% of cases

The error does not happen if the above sequence is performed in photo mode.

With raw recording (mlv_lite.mo):- start the camera in movie mode (any, even plain 1080p)- take a still photo (CR2, not JPG)- enable raw video if it isn't already, with lossless compression- start recording => kaboom :)

Was the second sequence working in any of the previous builds? (back to April 1st 3rd - older builds here (https://builds.magiclantern.fm/jenkins/view/Experiments/job/crop_rec_4k/))

Reproduced the second sequence (raw recording opening up straight into movie mode) with the latest build.Then jumped to test april 10th build and the same test is working without errors (lossless 14-bit 1920x1080) opening up straight into movie mode. Will try and see what build starts to fail.Did the test wrong. Let me check again.

Nikfreak, you are being so super nice! As soon as I have a free moment I will try to do some testing using an external monitor while shooting "4K" in my mind, using an external monitor should help while shooting with frozen liveview.. but again, I don't trust what my mind thinks

Took a closer look at the recent error messages. These problems were there in previous builds as well; they were just silent (in other words, I was unaware of the issues). Most of them appeared when estimating the compression ratio during standby - in this case, I've been calling the lossless compression routine in the so-called "dummy mode" (without image output - just to have it report the compression ratio). Turns out, when the planets were aligned, this method resulted in some left-overs on the EDMAC write channel (the one that writes the compressed output in memory); as a result, the first frame was sometimes invalid.

As I couldn't figure out how to clean the EDMAC channel, the easiest way was to avoid calling the lossless compression routines in "dummy mode" - that is, always provide a valid output buffer. If I go that far, I might as well refactor the memory management, so the raw recording buffers are always allocated during standby (and freed when exiting LiveView or when going to photo mode). It was a bit of work, but that's what I did.

You may ask, why?

Allocating memory for video buffers is slow (takes a few seconds to allocate the entire shooting memory) and complicated (see exmem.c and RscMgr thread). So, having the memory allocated in advance means recording can start right away (already discussed a few pages earlier (http://www.magiclantern.fm/forum/index.php?topic=19300.msg185903#msg185903)).

By contrast, creating the MLV file is very fast, so there's very little to gain by opening it in advance.

Minor: the status bar also shows how many buffers we have allocated, and their size.

Regarding 1080p48 - it was pushed a bit over the safe limit, so I've reduced its resolution to 1040. You still have the option to go back to 1080 in the crop_rec submenu (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183399#msg183399); however, you'll get a lot of errors. With some fiddling, you can probably get 1060 (e.g. set Target YRES to 1060 and Delta HEAD3 to -20), but I wouldn't trust it yet.

Also found a way to sync the H.264 proxy with the raw stream - by making the extra frames black. The first non-black frame from H.264 will match the first frame from MLV. Easy.

I've also integrated the thread-safety annotations (using clang) and fine-tuned the dynamic menus to make them halfway usable.

Overall, the build is still too buggy for my taste, but at least it's a small step forward.

Magic Lantern version : crop_rec_4k.2017Aug26.700D114Mercurial changeset : cffeb4898221 (crop_rec_4k) tipBuilt on 2017-08-26 21:27:15 UTC by [email protected]Free Memory : 128K + 2524KThe EOSM didn't save a core dump but created this assert log when recording 10-bit lossless with H.264 Proxy (the only way to get mv1080 on the M):

Aren´t mlv_rec and mlv_snd deliberately out of the equation on these builds? Wouldn´t mind them getting back of course :P.

Just recorded a few takes with H.264 proxy and 14bit compressed. The black frames seems to work as advertised which is friggin´ great. Back to script automation. Start by matching feature:MLV:Time: 08:29:45 (GMT+0)H.264:Create Date : 2017:08:27 08:29:43

[blackdetect @ 0x7f8b08d1d600] black_start:0 black_end:1.37637 black_duration:1.37637Output blackframes longer than 0.5 seconds and pixel threshold set to 0.01. If set to 0.00 it won´t work. Next find a way to interpret numbers to frames and search only beginning of the file to save time.Probably just cut off 0-1.37637 and export this example file to a new MOV should do the trick. Will check some more into this tonight.

Also found a way to sync the H.264 proxy with the raw stream - by making the extra frames black. The first non-black frame from H.264 will match the first frame from MLV. Easy.

Overall, the build is still too buggy for my taste, but at least it's a small step forward.

Thank you Alex, h264 proxy is a very good and complete feature now ! I use it to playback easily (of course) but also to use and sync the sound from H264 proxy with the raw stream. It solves temporarily the lack of mlv sound with comressed raw.

Aren´t mlv_rec and mlv_snd deliberately out of the equation on these builds? Wouldn´t mind them getting back of course :P.

Just recorded a few takes with H.264 proxy and 14bit compressed. The black frames seems to work as advertised which is friggin´ great. Back to script automation. Start by matching feature:MLV:Time: 08:29:45 (GMT+0)H.264:Create Date : 2017:08:27 08:29:43

[blackdetect @ 0x7f8b08d1d600] black_start:0 black_end:1.37637 black_duration:1.37637Output blackframes longer than 0.5 seconds and pixel threshold set to 0.01. If set to 0.00 it won´t work. Next find a way to interpret numbers to frames and search only beginning of the file to save time.Probably just cut off 0-1.37637 and export this example file to a new MOV should do the trick. Will check some more into this tonight.

Tested some more around ffmpeg and blackdetect filter. I´m sure there are more ways to go on about this but this seems to work as a oneliner in bash to cut of the black frames from the MOV file. MOV file should this way match start of MLV file and after this extracting the wav file is easy. Will work on automation details in Switch. FFmpeg needs to be installed if anyone wants to test.

How this monkeywrench oneliner is working:The INPUT.MOV is your H.264 proxy and OUTPUT.MOV will be the shortened MOV file without the blackframes. To speed up processing I do "killall ffmpeg" after one second of processing to get the the black frame detected. It then proceeds with copying out the "cleaned" OUTPUT.MOV file.

Here is a for loop which will trim and "clean" beginning and end of the H.264 proxy file. It needs some testing so if anyone could check if the new files match the MLV file it would be nice to know. If running into issues please upload the LOG.txt file.

*Not really working with longer files regarding chopping of the ending. Start of files works with any file with the former for loop. Gonna check further when I have time...

do you think it might be possible to include resolution settings for anamorphic? I already experimented with the EOS 50D and anamorphic but it is limited to lower resolutions. With the 5D3 it might be possible to go higher and according to the first crop_rec on steroids posts it's not unrealistic that it might work at continous recording.

The aspect ratio does not necessarily have to be 4:3. The new Arri Alexa models allow for a new 6:5 ratio. With 2x anamorphics that leads to 2.4 : 1 and is perfect for anamorphic shooters. http://www.arri.com/de/camera/alexa/kameras/kamera_details/alexa-sxt-plus/subsection/new_sxt_features/

A ML resolution of 2048 horizontal and 1716 vertical would be the best option. With 2x anamorphics that would be 4096 : 1716 after desqueeze just like the new Alexa SXT 6:5 ProRes 4K cine anamorphic mode :). If you have the time it would be great if you could add some other resolution options in that area in order to find what works at continous recording. 1920 horizontal and 1600 vertical would also be great for 3840 : 1600 desqueezed.

# with -ss $start after -i, it also trims the first good frame, why?# in this case, we have to use -ss start -i input.mov -to end-start# with short clips, bc outputs .1234 rather than 0.1234, but ffmpeg doesn't like it# also, without trickery, it may leave one extra black frametrimmed=$(echo $end - $start - 0.01 | bc)ffmpeg -ss 0$start -i "$1" -to 0$trimmed -vcodec copy -acodec copy new_"$1" -loglevel quiet

Only tested with 2 very short clips.

For long clips, you may want to scan only the first and the last 2-3 seconds. With some trickery, the analysis can probably be done in one single ffmpeg command (see e.g. https://superuser.com/a/898765 ), if you know the clip duration in advance ( http://ffmpeg.gusari.org/viewtopic.php?f=11&t=1284 ).

Nice. Works good on Ubuntu. Shortly tested on a virtual box set up with limited drive etc. Mac cries about grep syntax as usual but with some tweaking it will work on mac too. The beginning frames are easier to remove. I have some ideas about the ending when it comes to longer files to avoid long scan delays.

*All filters(-vf) seems to require re-encoding which most likely will delay the trim filter solution(yet to be verified). That is why my reverse effect filter won´t work since it seems to encode audio first then video which will delay processing a lot. What works is -ss and -t with the lossless -copy function splitting files at the end on longer files for example. Also putting the -ss before the -i input file as suggested in above script.

readarray blacks < <(ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")I simply kill the process after one second of running and it would probably work with the array as well.

On 5D3 123. Everytime I Q button (sub menu) anything in the menu, after that Shutter Scroll wheel and Apertures wheels are Yielding "Busy" when moved. The Aperture I can change in ML Menu, shutter speed will change in ML menu, but the busy sign comes on and it does not change to the shutter speed after all.

If I don't touch anything in the menus, I can keep using the Shtt wheel and Ap wheel.

Also Shutter Fine Tuning was behaving very weirdly, but that most likely has something to do with this.

Everytime I Q button (sub menu) anything in the menu, after that Shutter Scroll wheel and Apertures wheels are Yielding "Busy" when moved. The Aperture I can change in ML Menu, shutter speed will change in ML menu, but the busy sign comes on and it does not change to the shutter speed after all.

Reproduced - the BUSY message appears when pressing half-shutter (e.g. to close ML menu), but it's not related to submenus. Workaround: press Q twice outside ML menu. Or, enter ML menu and exit it by pressing DELETE rather than half-shutter. Related bug: ETTR sometimes locks up for the same reason.

The "fix" would be to re-enable the SRM use-after-free (not straightforward, but not hard either). Reason: keeping SRM memory allocated causes Canon code to show the BUSY message (and I don't know where/how to patch it).

However, using unallocated memory is not exactly a good practice. Therefore, if one can find a solution to keep the SRM memory allocated (marked as used, so Canon won't overwrite it, no matter what), but without the BUSY message, that would be the best way to solve this.

Btw!! another tip for people with Manual Lenses. When starting the camera in crop mode with a manual lens attached, liveview wont start, the camera does not detect a lens and therefore does not turn on the liveview. Instead of entering photo to movie mode and turning off and On crop mode etc. A workaround: when you turn on Camera with a manual lens after 1-2 secs the message "Ensure a lens is attached..." It blinks quickly about 2-3 times, for the duration this message is blinking, you can press "Start/Stop" button and it will force in to liveview in Crop Mode :) boy did i save a couple thousand shutter actuations after learning this ;)

Sometimes you might have to go in to Canon Menu and back for the shutter speeds to sync.

With a manual lens, without pressing anything, LiveView starts directly into crop mode. With a Canon lens, it flickers a bit (starts in regular mode, then switches to crop mode after 1-2 seconds). Not sure what the issue is...

edit: sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).

Just tried some really dark shots and ffmpeg correctly did not detect them as blackframes. I think we´re cool with black.Updated my script with some analysis. Having some difficulties comprehending A1ex advanced pieces but I think I got something similar in another way. bc causes parse errors as expected when missing a zero. It works anyway even if it looks shitty in terminal:

The leading zero from the second example is harmless - both values are accepted by ffmpeg.

Now the conditionals: I've looked through the man page of bc (it's the first time I use it) for some conditional operator, or min/max functions, but couldn't find any. As I didn't want to figure out the advanced syntax, I've stumbled upon the relational operators - for example (1 < 2) evaluates as 1 and (1 == 2) evaluates as 0. That makes it easy to build conditional expressions without knowing the if/then syntax.

After a second look over the man page, the if/then syntax no longer looks that hard:

With a manual lens, without pressing anything, LiveView starts directly into crop mode. With a Canon lens, it flickers a bit (starts in regular mode, then switches to crop mode after 1-2 seconds). Not sure what the issue is...

edit: sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).

Okey, my experience with manual lenses without electronics is that the liveview wont start. I'll say for the slow mo crop modes atleast. I have not used the high res modes for quite some time.

Did some minor changes which now seems to yield perfect cuts at start and ending in the script. Since we now how to delimit reencoding by using -to I tried the reverse filter again to achieve blackframe detection of the ending part.

This isn´t an original canon H.264 MOV file but prores file right? Think you want to run original H264 files for this workflow. I also noted that your black sequence is pretty long at the end. Probably more than three seconds, is that right?

@Lars SteenhoffTested some more and I managed to clean your prores file as well. I had to change threshold settings and also black length estimation prolonged to 5 seconds from earlier 3. Now it needs to be tested on darker footage so it doesn´t break elsewhere. Seemed to work on my test files over here. Corrected an error as well.

ffmpeg -i INPUT.MOV -c copy -map 0:a OUTPUT.WAVNext I try to extract the wav file, and want to change the script to process all the .mov files in the folder and make the wav file with the corresponding names.

to allow Davinci Resolve to import the audio and raw file sync up.

Here is the corresponding MLV for testinghttps://www.dropbox.com/s/hhnn68a24mer6q6/953A9793.MLV?dl=0

And all the files togetherhttps://www.dropbox.com/sh/a3a5wtyppgddnt4/AABh0Snrmb3fJC4T4CqrvTVra?dl=0

I will put davinci resolve compliance and automation workflows in Switch eventually. For dng folder embedding you´ll need some additional wav metadata. You can inject the data with bwfmetaedit.You can check row 190 and onwards in here:https://bitbucket.org/Dannephoto/switch/src/fa38c2375f43b051bf54268e3912dd6c946321a9/source_code/mlv_dump_01.txt?at=default&fileviewer=file-view-default

If you want to get the wav files from the script you can run this right away next to your MOV files:

I'm not sure if its something todo with All-I vs IPBI had the camera on IPB I don't know if thats affects anything for the black frames.

Because the prores does it well with the same settings. ( prores is All-I )and because we don't recompress the .MOV file maybe there are only certain frames that are allowed to be cut. Purely speculating.

Actually, I think it´s the floating point numbers. FFmpeg can´t cut the files with that precision on the frames so it does some crude cutting instead(Now this is speculation) However. Here´s a version which simply shaves off the the floating point leaving us with integer values. Didn´t test very much but it cuts you example H.264 file. Probably would be better if we could round numbers.By the way. I actually managed to trick a dark shot for blackframe with the latest changes that worked on your prores file.-vf "blackdetect=d=0.1:pix_th=0.09"If you´re working with in cam MOV files I suggest you set the script to:-vf "blackdetect=d=0.1:pix_th=0.01" in both places in the script.

Thanks for pm with files. I guess it will be pretty hard to match exactly all the time, especially the ending part. I see to it that wav will match and work in Switch. Easier to cut wav when you extracted the amount of dng files. Thanks for testing and feedback.

Found a way to allocate the entire SRM memory without locking the exposure controls... sort of:

- When allocating the entire SRM memory (normally used for still photos, for holding uncompressed 14-bit raw data), Canon code locks the shutter button and shows the BUSY screen (to prevent you from taking any more pictures until some buffer becomes free again).- Unfortunately, they also lock the exposure controls after you press the shutter halfway.- I've tried a bunch of tricks, such as calling their unlock routines used in this allocator (ClearBusy (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189251#msg189251), RealClearBusy); they did unlock the GUI, but pressing the shutter fully (by mistake or for whatever reason) gave ERR80. Unlocking the GUI for a short time and re-locking it again did not re-activate the exposure controls.- Low-level note: RealSetBusy/RealClearBusy only changes a GPIO; the MPU does react to this change.- There is another way to lock the GUI - using PROP_ICU_UILOCK (our wrapper is gui_uilock). That one lets you lock only the shutter button (UILOCK_SHUTTER), but does not lock the exposure controls.- Guess what: having both locks enabled (UILOCK_SHUTTER and Canon's SRM lock) works and DOES NOT LOCK THE EXPOSURE CONTROLS!!!

Therefore, one possibility would be to avoid the UILOCK_NONE state as long as the entire SRM memory is allocated.

However, this has another issue: half-shutter events are no longer present. Not sure how to solve it...

The MPU does send a message that tells the main CPU to show the BUSY screen. Maybe I should just interpret that as half-shutter press. I believe the message is some part of:

If anyone is still interested there is a way to get rid of that last black frame by doing multiple runs on stubborn files. By putting in a while loop we can keep recopy the file until it chops the very last blackframe out. Some talk about drop frames and black frames over here.https://video.stackexchange.com/questions/10383/remove-black-frames-from-vob-files-with-ffmpeg

Hello,When I use the proxy mode, H264 files get hot pixels. Is there a tool to remove it on H264 files ? By the way, when using 10/12 compressed mode, I get pink zone in the highlight. Is it removable ? the goal is not to correct it for master file but to have a clean proxy for offline editing.

I have been using Full-resolution LiveView: 5796x3870 at 7.4 fps (128ms rolling shutter) - continuous*) at 5 FPS! with success to capture still macro.

Given the low frame rate (5fps) has anyone used this , if so what is the recommend shutter speed for such slow rate under ML for a stationary subject/object. I see that "they" state to double the shutter speed of the FPS. which means if its 5fps then my speed should be 1/10. Given the subject is stationery and there is no movement is there any advantage of increasing the shutter speed. I have done some tests and cannot tell the difference other then loss of light.

Better: count the frames in the "sidecar" MLV and trim the MOV to the same number of frames.

This is what I do with the wav file. Seems the way to go for proxy files as well. Just tested and verified a perfect cut from the problematic file from Lars Steenhoff. And it´s fast. Since first cut always seems correctly detected, ending part can be calculated via mlv_dump and the MLV file.

Needs later version of mlv_dump to work and the corresponding MLV next to the proxy file.

frct=$(mlv_dump *"$MLV" | awk '/Processed/ { print $2; }')Grabbing the Processed number is a bit slow on bigger MLV files since it will process the whole file before giving you the information. There is a Frames Video: metadata tag which could be used but it´s never exact so not useful here.I will rewrite the processing in Switch maybe tonight...

I think there is a bug in the builds since 19th August with 50/60p 3x3.

I am getting some "Frame Jumps" I don't know how to explain it, but its like a frame will repeat it self 5-10 frames later. If that makes sense? At first I thought it was slow external drive causing it but I see it also after transcoding. It might happens only once in an MLV and sometimes 2-3 times. This bug might have been there all along, I think I have seen it "here and there" before, but very random, I will have to have a look at older takes from this summer. But since 19th almost every MLV has this problem.

I have not tried yet to cut the frames out, but not sure if this repeat frame overrides the one that "should have been in its place".

But then again it could also be the Converter, Danne's Batch script, I don't know. I cant see the footage until I have converted the MLV's to DNG.

At the moment, I am on crappy satellite internet again.. This Sat is extra bad, there will be no uploading, but might have some proper internet tomorrow and I will upload a MLV.

But I thought I atleast mention it now in case someone else experienced this and could chime in.

5D3 123. Aug 20th build. The shots I noticed this most on were shot a week ago. I have since then updated to 26th, but have not looked at anything I shot with 26th build. (no time)

I get the "busy" message while the 5x crop and raw rec is on an (can't change apperture and shutter speed). In my memory it didn't happen on previous builds. I'm on the latest available Aug 26 5DIII123.

* Tried applying the SRM_ClearBusy hack and manually block the full shutter event. Result: the camera ignores the event and stays in LiveView for about 2 seconds, then the MPU throws ERR80.

* Tried decoding the BUSY message (the one from UILOCK_SHUTTER) - it's complicated, they are different in each video mode. After getting it somewhat working, I've noticed it also blocks autofocus. Go figure...

* In the end, I've noticed that freeing one SRM buffer (and keeping the others allocated) appears to be a good compromise (even though it's still a hackish solution): - no BUSY screen - with N-1 buffers allocated from SRM, and nothing from Shoot memory, Canon firmware: - will still record H.264 (it will use the last buffer in this case) - will take a picture, then ERR70 (because it expects the SRM buffers to be freed in the same order as allocated) - will go outside LiveView and back, but will show BUSY on the screen - therefore, in this case, using the last SRM buffer is NOT safe - with the entire Shoot memory is allocated, and then N-1 buffers from SRM, Canon firmware is very unlikely to use the last buffer: - will not start recording H.264 (if the allocation was done in standby) -> ERR70 (timeout allocating memory). - will not stop recording H.264 (if the allocation was done while recording) -> lockup, ERR70 after 10-20 seconds (similar) - will not finish taking a picture (BUSY screen after capturing the exposure; will overwrite the additional SRM buffer) - will not enter H.264 playback mode from LiveView (black screen, Cannot playback movie after 10-20 seconds) - will enter CR2 playback mode from LiveView - will exit LiveView (to regular photo mode) - will enter Canon menu - all the failed actions from above will succeed as soon as we will deallocate our Shoot and SRM memory buffers (if done before the timeout) - with the exception of still photo capture, none of the above actions resulted in overwriting the additional SRM buffer - therefore, in this case, I believe using the last SRM buffer during LiveView is *probably* safe.

In any case, this method should be a lot safer than using the entire SRM memory after free (as in previous builds without the BUSY screen).

I also believe mlv_rec (in its current state) should now be usable on the current codebase (except maybe for full-res LiveView), as the memory management trickery has been moved into the memory backend.

BTW - if any of you has any clue on how to reproduce this bug (http://www.magiclantern.fm/forum/index.php?topic=19300.msg189513#msg189513), it will be very appreciated.

Forgot to mention - 700D and EOS M will need an extra constant (easy to find).

Just tested latest crop_rec_4k build. The busy signal is gone! What an achievement. The camera is set free again :).So, what did I test. Decided to run a few MLV takes in 14bit-lossless 24fps and (5D mark III 1.1.3) with proxy recording on. This combo works great and continuously. My findings:Short files are ok regarding metadata. Then I recorded a 3 minute MLV and suddenly metadata for processed frames weren´t to be found. I got this instead. Check the end. Anything obvious here?:

Reached end of chunk 1/3 after 2309 blocksGot a new RAWI, throwing away average buffers etc.Got a new RAWI, throwing away previous frame buffers etc.mlv_dump(12500,0xa533e1c0) malloc: *** error for object 0x887000: pointer being realloc'd was not allocated*** set a breakpoint in malloc_error_break to debugAbort trap: 6With srm OFF:

Reached end of chunk 1/2 after 2308 blocksGot a new RAWI, throwing away average buffers etc.Got a new RAWI, throwing away previous frame buffers etc.mlv_dump(11467,0xa533e1c0) malloc: *** error for object 0x895000: pointer being realloc'd was not allocated*** set a breakpoint in malloc_er