Raspberry pi: Video and Audio recording and streaming guide

I’ve spent some time on setting up audio and video streaming on my raspberry pi (mostly used as a baby monitor right now). While there are great ressources out there, it took me a lot of effort to find them and put everything together. So here’s a handy list of instructions if you want to:

Record and stream audio

Record and stream both audio and video into one file

Are just looking for an introduction into the topic.

Limitations

There are a few things I need to put up front here:

I use a Raspberry Pi 2 with Raspbian on it, with the PiNoir camera module and a cheap USB microphone connected to it. The same instructions probably apply for all Pi models. If you use a different OS than Raspbian, you’re probably far more advanced than I am and know how to adopt this stuff yourself. If you use a normal webcam instead of the Pi camera modules, most of my ressources will not be helpful either.

I only cover streaming on a local network and will not adress streaming over the internet. Most importantly, I’m an absolute beginner regarding Pi and linux things and can not give any security-related advice (e.g. how to setup an encrypted internet stream). So if you need these instructions and do anything security-related with them (like adding internet access to potentially private video streams), you’re completely insane.

This should set you up for everything more advanced. Additionally, use the “top” command when connected with SSH to see all running processes (use “k” to kill a process and STRG+c to exit top) and append ” &” to a command to start it in the background (so you dont have to open several SSH windows at once to start several processes).

Streaming video

For streaming video, there are basically the possibilities of using GStreamer, VLC, ffmpeg, netcat and mjpeg.

There is a very good blogpost here that describes streaming video and audio with GStreamer (install it withsudo apt-get install gstreamer1.0) and also covers the setup for an usb microphone. For me, the stream worked very well and without any noticable latency (i’d say below 100ms). Downsides are mainly that GStreamer is required on the receiving end – with OSX and Linux this works perfectly fine, for Windows this tutorial here may help (although I haven’t tried it). If you want to connect with multiple devices including Smartphones, this will probably not be your preferred option, but if you have one fixed machine as receiver, this should work fine.

For the other options, there is a decent article here describing each approach. It ultimately recommends using mjpeg. This appears to be the best solution for web-based access with smartphones and also seems to have low latency (I haven’t tried it) but probably won’t work well combined with audio or with high-resolution streams.

I’ve tried GStreamer (see above), VLC and Netcat. VLC produced mediocre stream quality with a lot of lag, Netcat was very simple to set up but wouldn’t work on Android since you have to add “:demux h264” on the receiving end as an option. If you want to try it, use these commands:

I haven’t tried FFmpeg since I didn’t feel like compiling it myself (~2h duration) and there appears to be no clear benefit. Mainly I wanted to have both video and audio in my stream – which brings us first to network protocols and then to picam.

Network protocols and streaming audio

Again, I’m not an expert in any of this, but it’s particularly bad for network protocols. So read ahead with care.

It appears there are two protocols for sending data over a network: TCP and UDP – see here, here or here for an explanation. Basically, TCP checks that no packets are lost (slow and reliable) while UDP just pushes packets over the network without any checks (fast but unreliable). So for videostreaming UDP is mostly the better option (potentially less lag) but TCP is used for http, appears to be easier to set up and is supported by more devices (e.g. every standard webbrowser). The simple streaming example with netcat above uses TCP.

Then there is the question of where to send your stream to. You can either send your stream directly to a certain IP (which means you need to tell your pi where the stream is headed to) or send it to a multicast adress in the range 224.x.x.x – 239.x.x.x that makes it available on the complete network (and, depending on your router configuration, can be forwarded to other LANs that are connected). Or you can set up an rtsp server that will process incoming requests for streams. To give you an idea for volume considerations: raspicam has a standard bitrate of 5 Mbit/s, a standard wifi card with 802.11g allows up to 54 Mbit/s. The recommendation I have seen is that for a up to ~3 users and 3-4 seconds lag, you should be fine with a TCP/http stream, for more receiving devices or less lag, you should use UDP / RTP.

Below, I’ll show the different possibilities with VLC and an audio stream from the microphone. This can also be adopted for a video stream (see the links above). Setting up the usb microphone is well described in the article already mentioned above.

Combine video and audio with picam

Now that we have set up both video and audio recording – how can we combine them in one file? It turns out this is not as easy as it should be. The raspivid-command clearly does not accept audio input itself, so we need some other program to combine both.

After spending a lot of time on optimizing VLC, I’ve found the program picam that does exactly what I’ve been looking for. So if you want to either record or stream both video and audio on your pi, I’d highly recommend picam. The instructions for installing picam are very clear and have worked for me without problems.

Picam will be started with (assuming the USB microphone is on hw:1,0):

cd ~/picam
./picam --alsadev hw:1,0 &

To start/stop recording use:

touch hooks/start_record
touch hooks/stop_record

Here are a few examples for setting up picam for different recording types:

Setting up a network stream with picam and node-rtsp-rtmp-server:

Picam has support for http streaming built into it and the instructions page also adds details to passing the stream to different programs. The easiest option is to use the node-rtsp-rtmp-server written by the same author which works very well. While Node.js is already included in Raspbian, I needed to install the npm and coffee tools manually:

sudo apt-get npm
sudo npm install coffee-script -g

With this, I was able to install the node-rtsp-rtmp-server as documented:

Automatically starting a stream upon boot

There appear to be several possibilities to automatically start a program after a reboot, documented here. I use cron to start a script called autostart.sh that itself starts either audiostream.sh or videostream.sh (I haven’t figured out a way yet to start both at the same time but toggle the # depending on which one I currently do not need).

run “crontab -e” and add:
@reboot /home/pi/autostart.sh

These are the scripts I use (remember to make them executable with “chmod +x <scriptname>.sh”):

Sidenotes: It appears to be helpful to add a short delay after boot to make sure everything is set up. Also, the home directory seems to be not yet available so I use /home/pi instead of ~. In any case, the “> log_autostart.txt” part in the first script leads to all output being written to the text file to give an indication on what went possibly wrong.

Optimizing latency and recording streams in VLC

When I first ran the code given above, I still had ~2 seconds of latency in my stream. Since the rtsp-solution used above is supposed to be much faster, I had a look at VLC on the receiving end and found a simple improvement: At Tools > Preferences > input/codecs there’s a network option where you can set the caching from standard to lowest latency. With this, I’m down to roughly 300-500 ms of latency in my stream.

Also, VLC offers the possibility to record your current stream. You can either find it in the menu or add it to the playback buttons. In my case, the video is recorded to my local videos folder.

Motion detection and other ressources

That’s it, we’re at the end of the now very long blog post. Since I’ve also had a look at motion detection (regarding the possibility to use the Pi as a simple home alarm system) and this relates to the video streaming topic, I’ll also put links in here as well.

Hi Jon,
as said in the beginning, these ressources apply mostly for the pi camera modules – in the case of picam, it appears to be module only. For your USB webcam, UV4L might be a good solution worth investigating – see http://www.linux-projects.org/uv4l/ – I think they also have tutorials on their site.
Otherwise, try googeling the topic. I’ve seen solutions with motion ( see e.g. http://www.instructables.com/id/Raspberry-Pi-remote-webcam/ – not sure if sound is supported) and with ffmpeg (you may have to compile it yourself, but I believe there are precompiled binaries available these days).
Hope this helps.
Cheers, Daniel

I’ve been using motion for a while with webcams, very happy with it. I see a config file setting for using either a USB cam or the Pi cam. Is it safe to assume the latest version of motion supports the Pi cam and the “enhanced” version is no longer required?

Ah, okay.
Your blog saved me countless hours 🙂 loads of thanks for this detailed write-up.
I have PiCam with Node RTSP-RTPM-SERVER running on RPi Zero W, and it works quite fine though due to H/W constraints sometimes it just crashes 😦 intermittently

I need some more advice from you. I need to stream audio/video to a browser not using vlc player/plugin, so far I see that Picam does support HLS, however node-rtsp-rtmp-server does not support HLS publishing. Is my understanding correct?

Hey Imran,
I can’t really help you here, I haven’t tried anything beyond the article and have no experience with http servers and streaming. Depending on the effort you want to put into this, using Youtube for the stream distribution might be a conventient option. I’m not that sure if you can use PiCam to connect to Youtube (assuming you want to include audio), so you have to figure that out yourself.
See these articles as a starting point:

Imran, an absolutely superb write up and piece of software integration. I have one question. Is it possible to use a port other than 80 for the rtsp stream? If so in which configuration file would I need to make a change. Many thanks in advance.

Thanks for the input but changing the port from 80 in the config.coffee file just killed the stream altogether. It’s a great shame because this does everything I need with the exception of the rtsp port. I need to get it to a 55nn range. But thanks again for the input.