Tile format

For the tile format specific to this video engine, V4l2 doesn't have (yet) this pixel format. Until then, as in v4l2, the pixel formats are represented as fourcc identifiers in an u32 value. It will be sufficient to define this custom tile format in the driver and user headers. Example.

256MiB limit

This video engine requires contiguous physical memory buffers to be located in the lower 256MiB of memory.

ideally if possible instead of having a fixed memory region, all should be available for allocation

videobuf2-dma-contig, allocates physical contiguous buffers, but to restrit to low 256M, requires dma_declare_coherent_memory to be called. Example. The declared memory region must be reserved using the generic /reserved-memory node.

No parsing in kernel

this video engine is a fixed function engine, this is a advantage by its simplicity, but in other ways this means that bitstream parsing can't be done by a firmware. Bitstream parsing in the kernel is not allowed.

13: Hugues Fruchet: Video codecs
There's a need to parse bitstream fields for those codecs, but that requires complex code (10K lines). Moving it to kernel could make it unstable, as it is harsh to write those parsers without any risk of causing crashes.
It seems to be better to put those parsers inside libv4l, using an open source license.
Results:
Drivers that require proprietary user space components should stay out of mainline
Multi-format buffers could be useful here
The hardware/firmware needs a lot of data extracted from the bitstream next to the bitstream itself. This is a custom format, so it is OK to add a new pixelformat for each of those formats. Such complex parsing should be done in userspace in libv4l2.
If very little parsing is required (MPEG), then that can be done in the kernel instead.
Recommendation is to start simple with e.g. just an MPEG implementation.

Request API, is a newer still experimental addition to V4L2 and Media Controller framework. In resume, any number of controls (configuration data) can be packed in a object called request, and if attached to a buffer, the driver can apply this configuration to the hardware when processing said buffer.

Way to go forward.

initial we can ignore that bitstream parsing is been done in the kernel, and first aim to have a working driver.

other option can be to split the driver in a common part that can be mainlined, and for each codec do as a submodule that can compiled out of tree. This also allows the distributions to choose which codecs to include.

this means that initially we will not worry about working for the inclusion in the mainline kernel (because the bitstream parsing will be rejected.)

using libv4l2 as suggested above to do the bitstream parsing in user space.

in this paragraph is explained that libv4l2 can transparently convert between formats, when the requested format mismatch the formats supported by the hardware driver. As in V4L2 a codec bitstream is considered as in equal mode to an image format, it should be possible to also convert bitstream format to a pre-parsed bitstream format compatible with the specificities of this video engine.

there is also libv4l2 plugins, in which could offer a possible way to do this transparent bitstream conversion.

perpendicular discussion that reaffirms the place for bitstream parsers.

Progress status

Detail of the conclusion of each step for each target kernel version (sunxi-3.4 / distro kernel / mainline).

User Land

Will aim to preserve the compatibility with similar users of v4l2 codec interfaces. Within its limits.

By the motive of the numerous video codecs apis in existence and equal mode the number of media players, the implementation of the support is outside the scope of this effort.

Only will be written simple programs for testing and example in how to use.