Methods:
A fast algorithm was developed based on row projections in a sliding window, providing initial borders (“Comb” method). These borders were then used by a slower but more precise variational algorithm that iteratively refines the borders (“Complex” method). Mouse eye retina images acquired by a custom-built spectral-domain OCT device were manually segmented by two independent experts, the test data set containing 20 images. The vitreo-retinal border (VRB), the inner and outer borders for the retinal pigment epithelium (RPEi and RPEo, respectively) were segmented. Then, first the Comb method was used for the segmentation of the same dataset, followed by the Complex method, including the variational refinement. The results of the two methods were compared to the manual segmentation by the mean absolute deviation in pixels from the manual segmentation by unpaired t-test.

Results:
The highest number of errors was observed for the segmentation of the RPEo. The Complex method provided significantly less errors in the segmentation compared to the Comb method alone (3.75 vs 2.23, 2.41 vs 1.86 and 3.89 vs 2.86 mean deviation for pixels Comb vs. Complex for the VRB, RPEi and RPEo, respectively, p<0.05 for all comparisons).

Conclusions:
There was a small difference observed between the manual and automated segmentation methods used in this study. Active contour detection further enhanced by a variational algorithm seems to be a promising segmentation tool that may be used both in animal and human OCT image segmentation.