Computer Vision

Final Project: Enhanced Augmented Reality

For this project, students were required to create a simple markerless augmented reality system, similar to the assignment below, but without the
corner markers and produce a white paper detailing our experiences with the technique.

To accomplish this task, I used an image tracker system which uses a combination of image recognition to identify a target
and then search through input frames for positive identification.

Each frame, a number of steps are completed including feature extraction and matching, homography estimation and refinement, and pose estimation
based on a known camera calibration.

A few of the computer vision methods used in the project are ORB feature detection, KNN feature matching using a brute force matcher with cross correlation,
homography estimation using RANSAC, SolvePNP() with Levenberg-Marquardt optimization, and conversion of the rotation matrix to a vector using
Rodrigues.

Augmented Reality

In this project, students were required to build a basic augmented reality system using corner markers and handle some real life cases such as
camera rotation/skew as well as video artifacts and noise.

To start, we injected basic still frames into the video with a calculated homography to ensure the process worked correctly but the
extra credit version extended this technique by interleaving frames from a video into the marker locations.

To achieve this, I used non-local means denoising to remove any static noise and then Gaussian blurred the result to remove circle artifacts.

Harris corners were used to estimate marker locations and those results were fed into KMeans with a K = 4 to identify the corners.

These four points were then used to compute a homography matrix using least squares and the injected image was then inverse warped into
the final frame to avoid holes or strange artifacts in the output.

Traffic Lights and Signs

For the traffic light project, students were tasked with detecting traffic lights and signs in simulated images. Later, these detectors
were used with real images to evaulate their efficacy in real world scenarios.