Jason Best

Director of Biodiversity Informatics

BRIT’s Computer Vision(aries)

August 31, 2016

Our interns and volunteers are invaluable at BRIT. In this series, they discuss their experiences with us.

This summer, four high school students from Trinity Valley School interned at BRIT through our Junior Volunteer/Intern program. These students were given the task of applying their computer science background to the challenge of helping BRIT create a quick and easy way to determine the fullness of our herbarium cabinets. By better understanding the details of the capacity of the cabinets, BRIT will be able to strategically plan for future growth and management of the herbarium collections.

I worked with students Grace Beasley, Jacob Haydel, Kevin James, and Ashia White to explore the process of using computer vision technology to analyze images of the open cabinets. We set out to extract details of each cabinet’s structure and the height of each stack of specimens within the cubbies of the cabinet to determine how much space is available for use in the cabinets. The students also created a powerful graphical interface for visualizing the cabinet data once they are generated.

The students already had experience with Processing, a software platform which facilitates the creation of software with rich and dynamic user interfaces, so we chose this as our primary tool for the project. Fortunately, Processing also has a number of additional modules which helped the team manipulate and analyze the images. In particular we used two modules that are focused on computer vision technology: OpenCV and BoofCV.

Computer vision technologies are used for many applications ranging from self-driving cars, to robotics, to entertainment, and much more. For our particular challenge, we used computer vision techniques to analyze the images of open cabinets to first find the boundaries of the cabinets and the cubbies, then to identify and measure the spaces that were filled with specimens. The computer vision modules provided all the tools we needed to do this, but we still had to experiment a lot to find the right combination of tools and methods to get consistent and accurate results. We haven’t yet found the perfect formula, but we are very close to a solution that we hope to start using soon.

To perform a full herbarium capacity analysis, we’ll need to capture an image of every cabinet in the herbarium (something we hope to recruit our loyal volunteers to help with), then process them with our computer vision solution. In the end, we’ll have a dataset with a record for each cubby and the percentage fullness of that cubby, as well as a total fullness percentage of each cabinet. With those data, we’ll have a much better idea of how much room we have to grow and how to strategically organize our collection to more easily adopt orphaned collections. To better understand this huge amount of data, the team also created a graphical interface that helps visualize the data in a more intuitive format. In particular, we created a herbarium “heat-map” using Processing which can read all the capacity data and superimpose color-coded cabinets on a herbarium map (pictured below). The interface also lets you click on a cabinet and see the details of each cubby and eventually we’ll combine this with the herbarium inventory data to let you see what species are in the cabinet.

The project still has a ways to go before we have a final, polished tool ready for use by the herbarium, but several of the students are eager to continue their efforts beyond the end of their summer break. Below, Jacob and Kevin share their summer experiences.

Jacob Haydel

Going into this project for BRIT, I didn’t really know what to expect. I hadn’t really done any work with computer vision, but I was eager to get started. The main thing I have learned from the project is that programing as a team is very different from working on an individual project. When working as a team you have to make sure everyone is on the same page and agrees on the direction the project should go. I also learned a lot about the problems that face BRIT in the digitization process. Taking a written format and converting it to a digital one can be a very tedious process; however there are huge benefits to digitization.

Working on the program, at first we had a really simple goal, to figure out how full the cabinets were. As we progressed we slowly started adding other goals to our original goal and expanded our idea of what this program could do. Despite this we were able to incorporate most of the ideas we came up with, and I think when the final program is done it will be much better than we initially imagined. I have really enjoyed working on what I would consider my first real world project in programing. It has definitely solidified my desire to pursue computer science as a career.

Kevin James

Going into this project, I wasn’t too sure how we were going to approach a project on such a large scale as this one. To begin, we learned the basics of computer vision and data visualization. Over the summer, we mainly worked on the data visualization aspect of the project, using fake data to test our program. We now have a heat map of the BRIT Herbarium, as well as a program that lets us view the fullness of individual cabinets. We hope to continue to expand this program, and we also hope to find a reliable method to obtain accurate data from the herbarium.

Our group faced the challenges that come with working on such large code as a team. This included changing specific parts of the code without unintentionally altering the entire program. We eventually learned how to work together more efficiently, and we managed to make a working program to display our herbarium data. This project was a very rewarding experience for me because I got an opportunity to use my computer skills in the real world.

My Summer Education: The Microscopic World

This post was written by Vanessa Marshall, 2017 summer intern and student at The University of Alabama. BRIT has been amazing – a catalyst that has unlocked the doors to the natural world and shown me wonders that I had never fully appreciated. I have always loved hiking, climbing trees, being outside in general, but my connection to the surrounding plant-life was distant, similar to the relationship between a homeowner and the trees that form the hardwood floors. I appreciated plants’ beauty and enjoyed the shade, but now there is a definite connection between my (limited) scientific knowledge and the physical plants. It means so much more to know the scientific name of a tree you just ran past, or to recognize Vitis mustangensis and know that you can eat the wild grapes growing on the vi...

A Day as a Botanical Illustrator

Article written by Isabella Wu, 2018 BRIT Herbarium and Research Intern and student at Emory University. Clear skies heralded a warm afternoon with no cover from the blazing sun. I was on my way to the LBJ Grasslands an hour away from the Botanical Research Institute of Texas (more commonly referred to as “BRIT”) where this whole herbarium adventure began. Kim Taylor, who is a rare plants expert at BRIT, had planned an outing to assess a new species first published two years ago in 2016. Memory’s Rose, latin name Rosa memoryae, sounded so good and beautiful. But there was a real possibility that it might not actually be a new species but rather a variation of the abundant Rosa foliolosa. We intended to find out. More specifically, we were going to collect the plant and record the number an...

A Summer Deep in the Weeds: Surprises Found Along the Way

Article written by Kelly Carroll, 2018 BRIT Herbarium and Research Intern and student at Trinity University. The property doesn’t look like much, driving by – a sea of seemingly uniform brown grass studded with mesquite trees, bordered by development and what looks to be a small-scale hackberry forest with a dry creek-bed running through it. Of course, everything looks different when you get close enough to see detail. Still, I didn’t have very high expectations when I arrived with Dan Caudle , a BRIT Resident Research Associate and grass and prairie expert, in May of 2018 to do a survey of the vegetation cover and biodiversity on the property ahead of construction that would ravage a good-sized section. When the summer started, I had no real experience in identifying anything beyond being...