Video analysis

BBT platform provides a set of algorithms for picture and video analysis, which can be used for both live and offline processing. This section lists available algorithms and describes problems they are aimed to detect, as well as use-cases in which algorithms may be applied.

BBT platform contains a set of video analysis algorithms, which are able to analyze video contents and find common problems in them. The video analysis, i.e. video quality assessment, is performed in real time. BBT platform provides live video analysis, real-time monitoring, warning and tracking of measured results of video sequences.

BBT platform provides several algorithms for video analysis, which are presented and described in this section.

Blocking is the image artifact which occurs as a result of lossy compression of image data due to a low bandwidth. Lossy data compression allows a large image to be stored in a smaller amount of memory and transmitted using less bandwidth, but at the cost of losing some information about image pixels. When the compressed image is restored to its original format, not every pixel can be restored and blocks of approximate pixel values appear on the restored image. Blocking detection algorithm analyzes video frames by searching for these blocks.

Analysis is performed within the single frame (intra-frame). It is possible to measure it in real time up to 1080i60.

Freezing artifact occurs then the transmission of video is interrupted and the user does not receive all video frames. Instead, the last received frame is repeated or no frame is shown. At the user side, it is manifested as a black screen or still image for a certain amount of time. Freezing detection analyzes a video stream for periods of black screen or still frame lasting longer than a specified threshold.

Black screen is a type of freezing artifact in which transmission of the video stream is stopped for a certain amount of time. It is simpler than general freezing since it is manifested as a black screen for a certain amount of time. Black screen detection searches for black frames in a video stream and calculates how long they last.

Analysis is performed inter-frame to measure the length of consecutive black frames, although decision whether a frame is absent is performed intra-frame.

Packet loss happens when transmission of video stream is interrupted in such way that incomplete frames are received (each frame is sent as multiple packets and losing some packets results in incomplete data about the frame being received). It is manifested as blocking artifacts, but they do not represent approximate pixel values as in the blocking artifact from lossy compression. Instead, they represent missing or wrong pixel values.

Noise in the video frame occurs when there is a random shift in pixel value from the original. It is manifested as individual pixels of slightly different color from the pixels in its neighborhood. It leads to a high variance in pixel values. Detection is helped by a median filter which averages values of pixels in a certain block of image – if the pixel value is very different from the average of its neighborhood, it is detected as noise.

When analyzing quality of a video stream, it has expected values of brightness and colors. Brightness is measured on a Y component (luminance, i.e. gray component of an image) and colors are measured on U and V component (chrominance, i.e. color components). If the measured value of brightness or colors is different significantly from the expected values, this anomaly is detected.

Image jitter occurs when pixels or whole lines in a frame from a video stream are randomly displaced due to interference in video transmission. It is an anomaly of analog nature, i.e. it appears if the video source is analog. It is caused by mechanical disturbances, supply voltage problems or imperfect synchronization.

Blurring and ringing artifacts occur in compressed image or video due to rough quantization of the high frequency image components. This results in image with reduced sharpness and deformations of edges. It can be detected as high frequency noise near edges.

Blurring is manifested as a smooth blur resembling that of viewing the image through a translucent screen. It is sometimes intentionally added to the image as an artistic effect, but in normal video streaming it is a sign of a defect stream.

Ringing artifact is manifested as “ghostly” artifact near edges and sharp transitions in a frame, making them appear blurred and less sharp.

Besides measuring the quality of individual frames and the complete video stream, as performed in previously mentioned algorithms, correctness of a device operation (e.g. television sets, set-top boxes, DVD players, etc.) can be tested by comparing its output to the referent image, i.e. expected output. BBT contains multiple approaches to picture comparison which has a goal of deciding whether two images are perceptually same without measuring their quality. Picture comparison solutions in BBT are:

Video decoding check (without reference video) – checking the quality of the video stream as produced by the device under test or received from the broadcast. Video stream is decoded and analyzed frame-by-frame by the desired algorithms.

Zapping time measure – checking the delay time of the device response, e.g. from the moment a user sends the command from a remote controller to the moment the device responds to a command by a certain content on the screen.

Interfaces operation check – checking the functionality of input and output interfaces on the device under test (television set, set-top box) by comparing the images produced by the device on its output interfaces with the expected content using one of the picture block compare algorithms.

Screen operation check – checking the functionality of the device as a whole by observing the content on its output screen and comparing the content with the expected one. Content can be gathered by grabbing the image from the internal memory of the device screen or by capturing the actual content of the screen using camera. Comparison with the referent content is performed by one of the picture block compare algorithms.