Hi, i have an EGL image, how can i pass it to the OpenMax encoder? i see that it supports this format as input port of the encoder ( OMX_COLOR_FormatBRCMEGL). How can i populate the the pBuffer and nFilledLen of OMX_BUFFERHEADERTYPE (http://maemo.org/api_refs/5.0/beta/libo ... y_p_e.html) with the EGL image? Thanks.

TBH I have no idea whether parsing EGL images will work or not. I have recollections of it being used within Android, but that imposed a set of restrictions on the buffers in itself.
When set to OMX_COLOR_FormatBRCMEGL it expects to find a OMX_BRCMVEGLIMAGETYPE structure in the buffer. Most of the fields are obvious. nUmemHandle is the memory handle (as reported by "sudo vcdbg reloc") for the final EGL image, with nUmemOffset being any offset into that buffer.
It looks like it has a chance of doing something sensible if you feed it that, but I'll give no guarantees.

Set the port definition with eCompressionFormat = OMX_VIDEO_CodingUnused and eColorFormat = OMX_COLOR_FormatBRCMEGL. I would expect nBufferSize to be sizeof(OMX_BRCMVEGLIMAGETYPE), or 24 bytes.

Cast the buffer pointer to be a OMX_BRCMVEGLIMAGETYPE and populate the fields. As previously stated, they're all obvious apart from nUmemHandle. I don't know how you get to that value from EGL. You can use "sudo vcdbg reloc" to list the relocatable heap handles, and it needs to match the one for your output object. I can ask a colleague if he remembers how to get hold of that handle.

Use OMX_EmptyThisBuffer as you are passing a buffer which references an EGL buffer, not an EGL handle.

Discussion had with said colleague.
Create your EGL target buffer with EGL_IMAGE_BRCM_VCSM and you can get the relevant handle back - see vcsm_square in raspicam for an example.
There are a number of provisos that have to be followed, but it does appear to have a good chance of working.

the vcsm_buffer is the handler of the buffer to link to nUmemHandle of OMX_BRCMVEGLIMAGETYPE ? Or vcsm_buffer contain the egl_image pixel data in RGBA32 ? or none of the two?
P.S. using eCompressionFormat= OMX_VIDEO_CodingUnused and eColorFormat= OMX_COLOR_FormatBRCMEGL gives 15360 as nBufferSize.

vcsm_info.vcsm_handle is the memory handle for the pixel data and vcsm_buffer is the CPU pointer to the pixels. The pointer is obtained by this line of code
vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type);

EGL images in the driver are containers that store the pixel format, size and other data plus generally a mem-handle to the pixel data. Using VCSM should make it easier for you do do the video-encode bit because since you allocate the VCSM buffer you already know the mem-handle and therefore don't need to find a way to extract the mem-handle to the pixel data from the GL driver.

It is a demo based on RaspiStill and RaspiTex (i simply modify them)..
I added the video encoder, the associated buffer and other encoder's stuff in RASPITEX_STATE.
The Encoder Initializatione is in raspitex_init through int Create_ilclient_encoder(ILCLIENT_T **client,COMPONENT_T **video_encode, int width_enc,int height_enc, int bitrate, int fps, int color_format,RASPITEX_STATE* state). In RaspiTex.c i added the EmptyBufferDone and FillBufferDone callback.
During the vcsm_square_redraw function in vcsm_square.c i send the buffer to the encoder.
The output video is saved in a file called test.h264.
The execution command is this one: /raspistill -n -w 1280 -h 960 -g -t 0 -v -gw 0,0,640,480 -gs vcsm_square

That runs and displays an image. video_encode is being given frames and encoding them. You've butchered the system with where buffers point etc, and it appears to be saving data to test.h264.
The recorded image is corrupted though as you've put in the OMX_BRCMVEGLIMAGETYPE that the image is 640x480 instead of 1024x1024. "./raspistill -n -w 1280 -h 960 -g -t 0 -v -gw 0,0,1024,1024 -gs vcsm_square" resolves that. Do note the requirement on RSO buffers to be a power of two in width.

You have then hit a firmware bug - I recently changed the image conversion routines at the front of video_encode to use the ISP instead of the VPU. Those routines don't have config for RGBX32. Either get a firmware from before July 9th, or wait for me to get the fix out.

With that I am recording the GL image correctly, although there is obviously some timing issue as there is significant tearing across the image. It looks like you're passing the image to video_encode before you've called vcsm_square_draw_pattern which adds the vertical bar to the image. In which case you definitely have a race condition as to which processes the pixels first, and a caching issue. Make the call to vcsm_square_draw_pattern and call vcsm_unlock_ptr (which will flush the ARM cache) BEFORE you pass the buffer to video_encode.
You are also running at a pretty low bitrate (500kbit/s), so the image quality isn't great. 4Mbit/s looks significantly better. I am getting what appear to be buffer skips, probably because critical threads are getting blocked whilst you write the data out. You'll need to refactor your code to reduce those.

Thanks!!!! you're my saviour!! It works (rollback the firmware to the 3 of july)!!
Of course i use this code only as base/educational-example on how can encode EGLimage. If i make my own application i design it with the apporpiate accuracy.
I'm only a raspberry GPU-beginner proggrammer, this forum and code study is the only way to undestrand the things.
Just for curiosity what does the define OMX_SKIP64BIT ?? I always use OpenMax without define it.
And what does vcsm_vc_hdl_from_hdl (suppose perform some type of handler conversion) ? i cannot find its definition userland/interface/mmal/vc/.

Thanks!!!! you're my saviour!! It works (rollback the firmware to the 3 of july)!!
Of course i use this code only as base/educational-example on how can encode EGLimage. If i make my own application i design it with the apporpiate accuracy.

I must confess to getting lost in your various hacks as to where the output buffers were being returned. I recognise that you were just trying things out, but it did all get a little messy in there. Overriding the FillBufferDone callback totally threw me! I'd also recommend that you don't do too much work in that callback context - I think you are blocking all IL communications at that point if you do significant amounts of work there. MMAL is a tad cleaner there in that the buffer callbacks are in a worker thread context.

longo92 wrote:I'm only a raspberry GPU-beginner proggrammer, this forum and code study is the only way to undestrand the things.
Just for curiosity what does the define OMX_SKIP64BIT ?? I always use OpenMax without define it.

When they wrote the IL spec they blundered pretty badly.
Not all compilers support 64 bit variables, or they don't require 64 bit alignment on 64 bit variables. The latter is the case with the VideoCore firmware.

If they had thought about it and made sure there was padding or other fields to make nTimeStamp naturally aligned to a 64 bit boundary then there would be no problem ever.
If we'd noticed the issue early on then we could have forced the firmware to adopt 64bit alignment on nTimeStamp to work around it. Unfortunately we haven't found a way to retrospectively work around it without breaking all existing applications that (correctly) set OMX_SKIP64BIT. TBH IL is such a pain in the neck to work with that I think the preference is to deprecate it instead! Everything you've done should also work equally with MMAL which is generally easier to work with.

longo92 wrote:And what does vcsm_vc_hdl_from_hdl (suppose perform some type of handler conversion) ? i cannot find its definition userland/interface/mmal/vc/.

https://github.com/raspberrypi/userland ... csm.c#L649
vcsm in the kernel uses its own internal handles (not file descriptors for reasons best known to itself) and does things like locking the handle to processes and the like. The firmware doesn't know or care about Linux processes, and has a simpler handler system.
If you cat /sys/kernel/debug/vc-smem/state (may need sudo) then you'll see the mappings, such as

I must confess to getting lost in your various hacks as to where the output buffers were being returned. I recognise that you were just trying things out, but it did all get a little messy in there. Overriding the FillBufferDone callback totally threw me! I'd also recommend that you don't do too much work in that callback context - I think you are blocking all IL communications at that point if you do significant amounts of work there. MMAL is a tad cleaner there in that the buffer callbacks are in a worker thread context.

Yes, future steps is to use the equivalent mmal encoder instead of openMax encoder.

I have another question there is a workaround on this requirement

Do note the requirement on RSO buffers to be a power of two in width.

? Because it limits my resolution only to the ones having power of two.

All Clear.
Another cuorisity question: could i pass the vcsm_buffer retrieve with vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type) to the pBuffer and the proper size to the EmptyThisBuffer? Of course the encoder input port is set with OMX_COLOR_Format32bitABGR8888 (in one of my previouse experiment, when i encode using the glReadPixels, i notice that GL_RBA = OMX_COLOR_Format32bitABGR8888). Thanks

All Clear.
Another cuorisity question: could i pass the vcsm_buffer retrieve with vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type) to the pBuffer and the proper size to the EmptyThisBuffer? Of course the encoder input port is set with OMX_COLOR_Format32bitABGR8888 (in one of my previouse experiment, when i encode using the glReadPixels, i notice that GL_RBA = OMX_COLOR_Format32bitABGR8888). Thanks

VCHIQ (the main IPC mechanism) is likely to complain if you do as it gets confused based on a set of Linux kernel page table flags. It'll be comparable to https://github.com/raspberrypi/userland/issues/386
But also with IL there is an inherent copy involved in passing buffers from ARM to VideoCore, so performance will suffer. That's probably why the BRCM EGL format was created in the first place.

If you switch to MMAL, then setting MMAL_PARAMETER_ZERO_COPY on the buffer and populating buffer->data with the vcsm vc handle, and you should get the equivalent zero copy option as BRCMEGL.
It's almost the same technique I use in yavta where I take a dmabuf for a CMA allocation from V4L2, use vcsm_import to import it into vcsm and get a vc mem_handle for it, and then pass those buffers into MMAL. Likewise drm_mmal for video_decode into DRM (Linux kernel rendering manager).

Ok, this is the encoding format. But OMX_BRCMVEGLIMAGETYPE is the data struct that i pass to the encoder (through the pBuffer of OMX_BUFFERHEADERTYPE), there is an equivalent struct in MMAL? Or pass directly the vcsm handle to Data field of MMAL_BUFFER_HEADER_T ?

Ok, this is the encoding format. But OMX_BRCMVEGLIMAGETYPE is the data struct that i pass to the encoder (through the pBuffer of OMX_BUFFERHEADERTYPE), there is an equivalent struct in MMAL? Or pass directly the vcsm handle to Data field of MMAL_BUFFER_HEADER_T ?

Sorry, misread what you were looking for.
I suspect this path has never been used under MMAL. MMAL and IL share the same components with just the frameworks above changing, so the component is still looking for a buffer with the same contents as OMX_BRCMVEGLIMAGETYPE. Either use OMX_BRCMVEGLIMAGETYPE, or recreate your own version for MMAL (pull requests welcome).

However, as you say, if you have already set up the MMAL port format to match what EGL is creating, then you can set MMAL_PARAMETER_ZERO_COPY on the port, allocate a mmal_pool with buffers having a size of 0, and then put the vc_handle into buf->data field and set buf->length and buf->alloc_size appropriately.
There is nothing magical about the buffers that EGL creates via VCSM, so they can be treated as standard pixel buffers.