RichShumaker wrote:As I always seem to start these thread posts THANKS 6by9 for all you do.

Now I have a simple question, does the HDMI to CSI workflow work currently?
I read thru a lot of the posts and I was just looking for a simple, "yes it works do this" and I was having a hard time finding that.
Is there a Wiki or place with more details on setup and usage?

Thanks everyone for all the hard work getting this going.
I had dropped External Video Inputs into the Pi using the GPU several years ago so I am excited that it has been active for as long as it has.

Yes it works for most of the resolutions tried between [email protected] to [email protected], although some only in UYVY mode and some only in RGB24 mode. It's all related to the HDMI vs CSI timings. Please remember that interlaced resolutions are not supported.
I have it working in the same way as half of raspi_tc358743.c in that it can route the images through to the display. For the sake of completeness I'm just adding H264 encoding as well so it is the same as raspi_tc358743.c, as well as handling some of the V4L2 setup to make it a little more user friendly.

No, I haven't written up a how-to because it is still in a state of flux as it is being reviewed upstream. If you want to be involved in testing it then please email me and I'll add you to the list and send out instructions. As mentioned in my response to auvidea, things vary based on which Pi variant you are using, and I haven't finnessed the nice way of handling that.

Perfect and Thanks.
I am not a good beta tester as I am an advanced user.
Also I don't own the product yet as it is pricey for something that I can't plug and play with.
I just bought a USB3 HDMi video encoder for less than the cost of the HDMI 103 board.
Which is basically how I would use this as a video encoder for production.

I appreciate and have benefited greatly from all the work you do and all the people in the community do.
I have used the RasPi cameras since it was first released as a User.
That is why I gravitated towards RPi_Cam_Web_Interface as it works for me and is easy to deploy over and over.
That is what I would use with this as well unless it didn't work for some reason.
I love testing weird formats with RPiCWI, like 42 frames per second. I seem to love that frame rate.
I have a V1 camera and I know the limitations of the GPU to 1080p encoding.

Here is a video of me testing different cameras, 3 Pi's and 1 NDI from an Android Cell Phone.https://youtu.be/_vzl0lU5k5I
My goal has always been to use the Pi as a Motion Capture Rig or as I call it Reverse 360 cam.
I built a 360 cam with Pi's and then figured out I don't want that.
I need to film the subject in 360 not the world in 360 and I think it is the greatest downfall of the current state of 360 video recording / VR stuff.

Side note NDI is a NewTek open standard for using Ethernet to send a Broadcast Video & Audio signal.
It is amazingly fast and works really really well. Don't let my crappy video fool you it is awesome.
The ARM SDK is not a full blown SDK from what I know. Not sure if you could adapt it to a Pi.
I know that the GPU has a blob with codecs(which is what I consider NDI a Compression Decompression Format) that can be unlocked BUT this codec is too new and I know it is not in the blob.
I would ask to add it except I know that is not really an option.

Thanks again and have an awesome day.
I have a feeling I will be purchasing one of these later this year to experiment with.

I've been trying to use a B101 board from Auvidea connected to a GoPro Hero 5 to get the video feed on a Raspberry Pi 3 without success. The raspivid command works OK when the B101 is connected to my laptop via HDMI but does not when connected to the camera.

I don't know if it's a problem with the camera not supporting the EDID provided by the B101 board.

First I am new to this so I don't know the in's and out's of the software.
I do know AV troubleshooting.

From an AV troubleshooting point of view I would change the settings on the GoPro and see if 'any' resolutions work like 720p or Standard Def.
Also I would try a converter box not to make a conversion but to see if I could 'inject' what the signal needs to be recognized.

I have had 'issues' with many devices not synchronizing because the signal was 'off' even if it was ever so slightly.
Specifically my Black Magic Design Capture Card would not recognize my GoPro via HDMI, then I sent it through an HDMI to SDI box as the card also has SDI.
Then it worked no problem.

carlos.rodriguez wrote:What can be causing this assertion failure in mmal_queue_sanity_check?

A mmal_queue uses a singlely linked list to queue up buffers. mmal_queue_sanity_check checks that the buffer about to be added to the queue isn't already there, as adding it would create a circular reference in the list, and "bad things" then happen.

Your buffer sizes look a bit odd - 27 bytes is the H264 header bytes, 263 bytes with flags 0x0C is the I-frame, and then multiple 36 byte packets as P-frames - those are tiny for encoded data. Has the encode bitrate got turned right down somehow? I can't remember whether that is on the command line or not. Have a quick look at the source code as it should be obvious.

carlos.rodriguez wrote:Thanks Rich Shumaker. I am trying to find a video receiver with only 720p but all my TVs here have at least 1080p so I can't test if the GoPro is able to ouput 720p video.

Thanks,

Carlos

6 by 9 already answered with something much better.
I just wanted to let you know that most modern screens that do 1080p also do 720p. You should be able to set your GoPro to 720p and then direct connect to a 1080p screen and it will see the signal and show you 720p.

It is the reverse that seldom happens that a Standard Definition Screen can show a 1080p signal.

6by9 wrote:Your buffer sizes look a bit odd - 27 bytes is the H264 header bytes, 263 bytes with flags 0x0C is the I-frame, and then multiple 36 byte packets as P-frames - those are tiny for encoded data. Has the encode bitrate got turned right down somehow? I can't remember whether that is on the command line or not. Have a quick look at the source code as it should be obvious.

Thanks, I tried modifying raspi_tc358743.c doubling the bitrate value (from 17000000 to 34000000) and the errors are gone! I get 1080p25 from my laptop but a few glitches appear in the video from time to time.

RichShumaker wrote:
I just wanted to let you know that most modern screens that do 1080p also do 720p. You should be able to set your GoPro to 720p and then direct connect to a 1080p screen and it will see the signal and show you 720p.

The problem is that the GoPro Hero 5 has no setting to change the output video format. It is supposed to be selected automatically based on the EDID received. Even if I set to record in 720p30 I still get an HDMI out signal at 1080p60. I will try and contact GoPro support for more information.

6by9 wrote:Your buffer sizes look a bit odd - 27 bytes is the H264 header bytes, 263 bytes with flags 0x0C is the I-frame, and then multiple 36 byte packets as P-frames - those are tiny for encoded data. Has the encode bitrate got turned right down somehow? I can't remember whether that is on the command line or not. Have a quick look at the source code as it should be obvious.

Thanks, I tried modifying raspi_tc358743.c doubling the bitrate value (from 17000000 to 34000000) and the errors are gone! I get 1080p25 from my laptop but a few glitches appear in the video from time to time.

That's a bit bizarre. It possibly means that the frame rate value being set is wrong. If given a framerate then bitrate control assumes the frame rate is as defined and divides up bits accordingly. If fps is set to 0 then it works off the frame timestamps (which should all be valid and correct).
It'd be useful to know what bitrate is actually being produced with your 34Mbit/s value (which technically is beyond what the codec can produce!)

carlos.rodriguez wrote:

RichShumaker wrote:
I just wanted to let you know that most modern screens that do 1080p also do 720p. You should be able to set your GoPro to 720p and then direct connect to a 1080p screen and it will see the signal and show you 720p.

The problem is that the GoPro Hero 5 has no setting to change the output video format. It is supposed to be selected automatically based on the EDID received. Even if I set to record in 720p30 I still get an HDMI out signal at 1080p60. I will try and contact GoPro support for more information.

The EDID advertises the supported resolutions and framerates, and raspi_tc358743 goes up to 1080P30 with a native/recommended resolution of 720P60. The GoPro should be reading that and selecting an appropriate mode. Your logging would imply that that is working based on the logging:

6by9 wrote:The EDID advertises the supported resolutions and framerates, and raspi_tc358743 goes up to 1080P30 with a native/recommended resolution of 720P60. The GoPro should be reading that and selecting an appropriate mode. Your logging would imply that that is working based on the logging:

6by9 wrote:The EDID advertises the supported resolutions and framerates, and raspi_tc358743 goes up to 1080P30 with a native/recommended resolution of 720P60. The GoPro should be reading that and selecting an appropriate mode. Your logging would imply that that is working based on the logging:

That log was when connecting to my laptop. When connected to the GoPro I get no screen and the raspi_tc358743 command returns the following:

Ah, OK.
raspi_tc358743 gives up after 5 seconds of waiting for a signal and just tried to run (it's not meant to be a polished app!).
Whilst the data sheet isn't public, http://elixir.free-electrons.com/linux/ ... egs.h#L313 gives the relevant bit meanings. It got copied for https://github.com/6by9/userland/blob/h ... 743_regs.h
Repeatedly getting 0x01 just DDC5V means that the GoPro isn't really attempting to connect. It's just possible that the GoPro is trying to enforce HDCP when it isn't supported but either the chip or code, but I don't know.

RichShumaker wrote:
I just wanted to let you know that most modern screens that do 1080p also do 720p. You should be able to set your GoPro to 720p and then direct connect to a 1080p screen and it will see the signal and show you 720p.

The problem is that the GoPro Hero 5 has no setting to change the output video format. It is supposed to be selected automatically based on the EDID received. Even if I set to record in 720p30 I still get an HDMI out signal at 1080p60. I will try and contact GoPro support for more information.

That is weird for sure.
The issue you describe definitely seems like you are trying to give the RasPi a signal it doesn't like.
I have had similar issue with Capture devices. If I try to feed X to certain devices it doesn't like it. X being a resolution usually 60P and not 30P(or 60i).
GoPro has a LOT of resolutions and here is the chart - https://gopro.com/help/articles/Block/A ... O5-Session
It seems weird that the record resolution is different then the resolution out the HDMI and that is something I would say GoPro would need to answer.
Mic Bergsma is my Go to Guy for GoPro Tips and Tricks here is a video of his that might help as well, https://youtu.be/QYzvQpnYSK0
I also found this GoPro thread that discusses HMDI out on the Hero 5 - https://community.gopro.com/t5/Cameras/ ... 227/page/2#
It is your exact issue. If the GoPro is sending a 60p signal and not a 30p or 60i signal the RasPi won't lock on.
60p is considered 3G and 60i is considered 1.5G in Broadcast terms. The RasPi is only rated to 30p(60i).
I am not sure if the RasPi can handle the interlace but I don't see why it couldn't. I just know that 60p is beyond the capabilities of the RasPi at this time.

The only other option and it is silly is to convert the signal with a signal converter(downscaler or Scan Converter I think it what they are called) that will cost more than the RasPi with the add on board though. They run between $300 and $3000 depending on the features you need.

Good Luck and I hope it works because I want to use a RasPi Capture device with my GoPro someday.

RichShumaker wrote:It is your exact issue. If the GoPro is sending a 60p signal and not a 30p or 60i signal the RasPi won't lock on.
60p is considered 3G and 60i is considered 1.5G in Broadcast terms. The RasPi is only rated to 30p(60i).
I am not sure if the RasPi can handle the interlace but I don't see why it couldn't. I just know that 60p is beyond the capabilities of the RasPi at this time.

The Toshiba TC358743 HDMI to CSI-2 chip will cope up to 1080P60, but 2 lanes of CSI-2 won't.

Interlaced video has multiple issues:
- The two fields come in with different packet IDs, but handling that nicely to a user app with the correct signalling is not trivial. Obviously getting fields reversed is next to useless.
- The H264 encoder is progressive only, up to 1080P30 (a bit higher with overclocking).
- The TC358743 can only handle interlaced as RGB888 or YUV444, so 50% extra data over YUV422.

Last edited by 6by9 on Fri Jun 30, 2017 3:51 pm, edited 1 time in total.

RichShumaker wrote:It is your exact issue. If the GoPro is sending a 60p signal and not a 30p or 60i signal the RasPi won't lock on.
60p is considered 3G and 60i is considered 1.5G in Broadcast terms. The RasPi is only rated to 30p(60i).
I am not sure if the RasPi can handle the interlace but I don't see why it couldn't. I just know that 60p is beyond the capabilities of the RasPi at this time.

The Toshiba TC358743 HDMI to CSI-2 chip will cope up to 1080P60, but 2 lanes of CSI-2 won't.

Interlaced video has multiple issues:
- The two fields come in with different packet IDs, but handling that nicely to a user app with the correct signalling is not trivial. Obviously getting fields reversed is next to useless. - The H264 encoder is progressive only, up to 1080P30 (a bit higher with overclocking).
- The TC358743 can only handle interlaced as RGB888 or YUV444, so 50% extra data over YUV422.

I apologize as I understand interlace all too jaggedly well and I should have known it was not 'simple'.
That is good to hear the RasPi can handle up to 60p with the 4 lane CSI. Can the GPU encode 60p?
This leads us back to the start which is I think the GoPro is sending 60p and the RasPi wants 30p and therefore can't lock onto the signal.

As I said I would attempt to lower the GoPro signal as low as it will go, so 480p I think is what it is called. Basically either Standard Definition PAL/NTSC signal size.
If the Raspberry Pi works with that then I would slowly raise it to 720p then 1080p - The P stands for Progressive and it is necessary for the Pi to see the signal properly based on what 6 by 9 just said.

RichShumaker wrote:That is good to hear the RasPi can handle up to 60p with the 4 lane CSI. Can the GPU encode 60p?

Not 1080P60. I'd already stated "The H264 encoder is progressive only, up to 1080P30 (a bit higher with overclocking)."

RichShumaker wrote:This leads us back to the start which is I think the GoPro is sending 60p and the RasPi wants 30p and therefore can't lock onto the signal.

No, it seems to be sending nothing.
The raspi_tc358743 app starts by polling the TC358743 (which can support 1080P60) for the HDMI timing and mode information for the source. The TC358743 is reporting back "no sync", so this is all well before the Pi has got involved in trying to set up receivers.
The only bit the Pi has got involved in is sending the TC358743 an EDID to use, and that is only advertising modes

If a device sends no EDID can the signal be 'locked' on'?
The reason I ask is that you can strip the EDID out.
So what happens if a signal that is not proper is sent to the RasPi?
Let's say a 4K signal is sent to it?

Oh and if it is the case that the RasPi sees nothing in this situation that would lead me to say bad cable.
Have you attempted to use a known good cable? Meaning testing the cable and GoPro outside the RasPi setup?

RichShumaker wrote:
Oh and if it is the case that the RasPi sees nothing in this situation that would lead me to say bad cable.
Have you attempted to use a known good cable? Meaning testing the cable and GoPro outside the RasPi setup?

Rich Shumaker

Oh and I should have also asked have you gotten this unit to work with any other inputs besides the GoPro?
From my experiences with the camera when I have seated my CSI cable incorrectly it has caused errors when you try to view the live image.
I am guessing we would be seeing errors but I was not sure as I stated I don't own one of these yet.

RichShumaker wrote:I wonder how high up the resolution chart it can go in the real world?

Tested on rpi3 for now but can it can work with other other rpis (including the pi zeros) by tweaking a config file (more about this will probably be written in a tutorial). Any resolution (with auto scale) any encoding (planar yuvs, packed rgbs, mjpeg, h264) any fps (depending on the source) up to the hardware limits is supported by the driver.

RichShumaker wrote:I wonder how high up the resolution chart it can go in the real world?

Tested on rpi3 for now but can it can work with other other rpis (including the pi zeros) by tweaking a config file (more about this will probably be written in a tutorial). Any resolution (with auto scale) any encoding (planar yuvs, packed rgbs, mjpeg, h264) any fps (depending on the source) up to the hardware limits is supported by the driver.

Two Words, WOOHOO & AWESOME. I know you are probably thinking WooHoo isn't that actually 2 words, heheh.
Thanks for that awesome news it made my day.

RichShumaker wrote:I wonder how high up the resolution chart it can go in the real world?

Tested on rpi3 for now but can it can work with other other rpis (including the pi zeros) by tweaking a config file (more about this will probably be written in a tutorial). Any resolution (with auto scale) any encoding (planar yuvs, packed rgbs, mjpeg, h264) any fps (depending on the source) up to the hardware limits is supported by the driver.

Two Words, WOOHOO & AWESOME. I know you are probably thinking WooHoo isn't that actually 2 words, heheh.
Thanks for that awesome news it made my day.

No disrespect to UV4L2, but this is NOT using any upstream kernel code. It is a closed source userspace library.
It's also using the rawcam component that is known to have a bug that leads to locking up the Gpu.

6by9 wrote:
No disrespect to UV4L2, but this is NOT using any upstream kernel code. It is a closed source userspace library.
It's also using the rawcam component that is known to have a bug that leads to locking up the Gpu.

Dagnabit I was doing a happy dance.
I do have to say THANK YOU to 6by9 for pointing this out as it would have made me mad to run into that Bug when trying this out.

So basic question, what is the best way to 'stream' the TC358743 using the Pi?

6by9 wrote:No disrespect to UV4L2, but this is NOT using any upstream kernel code. It is a closed source userspace library.
It's also using the rawcam component that is known to have a bug that leads to locking up the Gpu.

As far as I can tell, the firmware code is as closed as UV4L. You are right in saying that it is not using the upstream kernel code. In facts, almost everything runs in userspace, accessing the GPU whenever possible. UV4L is NOT a library. It consists of a series of Video4Linux2 userspace drivers (you can NOT tell there is a difference from kernel drivers from the user perspective) that you can use with any third-party applications and other extensions. We always promptly fix any problems that the users may find in any module of UV4L. As to the rawcam component locking up the GPU, we possibly understood when this problem would happen and eventually found a working solution (indeed it was a nightmare to get the pipelines up and running properly). The hope is that any known problems with the GPU will also be fixed as soon as possible or at least that the rawcam component won't change for the worse in the future, which should be the case according to what you said in another post (point 3):

rawcam will remain supported as it is a useful tool in avoiding having to push everything into the kernel. (Fixing the double-free issue is on my list too).

First of all, I've "solved" my issue with the GoPro Hero 5 and the B101 board. I don't know the reason but if I connect an HDMI switch between the GoPro and the board everything works great. (link to the switch used: http://www.tecknet.co.uk/hdmi03.html). No need to power the switch with the MicroUSB included.
I've been sniffing EDID data and the switch does not change it so the issue is not there. I also tried with a GoPro Hero 3 and it works fine without the need of the switch. Anyway, I can live with that.

The problem I am facing now is with 1080p25 resolution. I get a lot of glitches in the video even in the preview window (which I guess is in raw format because when the encoding format is wrong I still get the preview to show up). I've uploaded a screenshot of the issue: https://ibb.co/krJg2v . I've tried at different bitrates and fps. Has any of you managed to get 1080p25 video out of the B101?