Experiments with FFmpeg Filters and Frei0r Plugin Effects

Alexey Suhoviy

Web Developer

SHARE

4

This is my third article about video processing with Rails. Here I’m going to show examples of FFmpeg video and audio filters and Frei0r plugin effects. All these features are being added to the app I described in my previous articles:

All of the additional parameters I added to the code above might not be needed for your current project. But I’ve included almost all possible parameters to avoid issues when something doesn’t work because some extension isn’t installed.

Fix possible issues with the Frei0r plugin

Frei0r is a cross-platform framework for video effects. With Frei0r minimalistic plugin API, you can avoid issues with recurring re-implementation of standard effects and easily implement filters and mixers provided by the plugin’s API.

When I installed FFmpeg with Frei0r, the Frei0r effects didn’t work at first. To check if you have such an issue, run FFmpeg and try to add Frei0r effects to a video:

If you run the ls -l /usr/local/lib/frei0r-1/ command you’ll see that the plugins are installed with the .so extension.To solve this problem, on my machine (macOS 10.12.5, ffmpeg 3.3.2, frei0r-1.6.1) I copied the .so extensions to .dlib:

You also have to set this environment variable with the path to the folder where the .dylib files are stored:

export FREI0R_PATH=/usr/local/Cellar/frei0r/1.6.1/lib/frei0r-1

This solution looks like some strange hack. Nevertheless, I finally got Frei0r to work well.

FFmpeg and Frei0r filter processing examples

FFmpeg is a video and audio converter that can also process live audio and video. Moreover, you can resize video on the fly with high-quality results. You can find more detailed information about FFmpeg in my previous articles and in the FFmpeg documentation.

For a description of all available FFmpeg filters, check the filters section of the documentation.

Video and audio effects

FFmpeg with the Frei0r plugin is used for video effects. The options -vf and -af specify simple video and audio filters. The -filter_complex option specifies complex filters.

To avoid excessive audio processing, you should copy the audio stream unchanged from your input source file with the option -c:a copy when using all video filters.

For all audio effects, we can copy the video steam unchanged with the -c:v copy option.

The -c:v copy option isn’t applied for reverse, slow down, and speed up effects because they alter both audio and video streams.

By default, FFmpeg selects the best input stream of each kind (video, audio, subtitles, etc.) Let’s assume that for our audio effects we want to convert all audio input streams, however (maybe we have the same stream in different languages, for example). You can do this with the -map 0 option.

The official Frei0r page doesn’t contain any documentation on how this option should be used. That’s why I had to conduct my own research. During my research, I found a helpful resource that describes Frei0r filters along with some other plugins. With the help of this resource, I learned about all available filters and their parameters.

Here’s the original video that I used for my video effects processing demonstration:

And here’s the original video I used for my audio effects demonstration:

Zero saturation with the hue filter produces a black and white output video.

-vf hue=s=0

#3 Vertigo effect

The vertigo effect performs alpha blending with zoomed and rotated images. For my example, I set the phase increment parameter to 0.2 and kept the zoomrate parameter with its default value to modify the video.

-vf frei0r=vertigo:0.2

#4 Vignette effect

The vignette effect makes your film look like it was shot with a vignetting lens. I used the default settings.

-vf frei0r=vignette

#5 Sobel effect

The Sobel effect is used in image processing and computer vision, particularly within edge detection algorithms where it creates an image emphasizing edges. The Sobel effect doesn’t have any parameters you can change.

-vf frei0r=sobel

#6 Pixelizor effect

The pixelizor effect is also applied with default settings and creates a pixelated picture.

The slow down effect uses the setpts filter to double the Presentation Time Stamp (PTS) of the original film. As a result, you get a video that’s half the speed of the original.

The atempo filter is used to halve the speed of audio with the 0.5 (50% of the original tempo) setting.

-filter:v setpts=2.0*PTS -filter:a atempo=0.5

#15 Speed up effect

The speed up filter works with the same approach as slow down effect, only in the opposite direction. It takes half of the original video Presentation Time Stamp and makes it twice as fast.

-filter:v setpts=0.5*PTS -filter:a atempo=2.0

#16 Echo audio effect

The aecho audio filter reflects your audio stream to create an echo effect just as if you were in the mountains.

-af aecho=0.8:0.9:1000|500:0.7|0.5

The first parameter is the input gain of the reflected signal (0.8). The second parameter is the output gain of the reflected signal (0.9). Next there’s a list of delays from the original signal in milliseconds, separated by the pipe | symbol. After this parameter goes the list of decays for each delay in the previous list. The colon : separates items in lists.

The f and d parameters perform the same functions as in the tremolo filter.

#19 Chorus audio effect

The chorus filter resembles an echo effect with a short delay. The difference is that the delay in the echo filter is constant, whereas in the chorus filter it varies using sinusoidal or triangular modulation.

-af chorus=0.5:0.9:50|60|40:0.4|0.32|0.3:0.25|0.4|0.3:2|2.3|1.3

The parameters in the chorus filter are ordered as follows: input gain, output gain, delays list in ms, decays list, speeds list, depths list. Parameters are separated by the pipe and items in lists are separated by the colon.

#20 Watermark

The watermark feature is implemented in the carrierwave-video gem. It uses FFmpeg’s overlay filter.

On the video below, you can see the combination of the vertigo, vignette, RGBnoise, distorter, glow, reverse, echo, tremolo, chorus, and watermark visual effects.

The watermark on this video is a transparent png image with the red text FFmpeg TV:

In this article, we’ve covered implementation of FFmpeg video and audio filters and Frei0r plugin effects and our media experiments with them. FFmpeg and Frei0r are powerful tools. The filters described here mostly add some additional distortions to video. But FFmpeg can also be used to perform video and audio enhancements such as lens correction, sound level normalization, noise and shake removal, video streaming, and much more.

4.1/ 5.0

Article rating

17

Reviews

Remember those Facebook reactions? Well, we aren't Facebook but we love reactions too. They can give us valuable insights on how to improve what we're doing. Would you tell us how you feel about this article?