The role of image modality and visual characteristics in archiving biomedical images.

Imaging in biomedicine has seen an explosive growth in recent decades. Clinicians can offer better diagnosis, and scientists and the lay public often better understand complex biomedical concepts through visual means. Typically, patient identifiers are used for archiving and indexing images’ metadata in the clinical setting, and bibliographic citation data are used in library collections, such as the open access biomedical research articles from the U.S. National Library of Medicine’s (NLM) PubMed Central® (PMC) repository. Automatically detected image modality and visual image characteristics can offer valuable addition to these traditional textual metadata for archiving and indexing visual material. Example image modalities include computerized tomography (CT), X-ray, magnetic resonance imaging (MRI), ultrasound, photographs, illustrations, charts, graphs, and sketches. The open-access biomedical literature in PMC is a source for OpenI (pronounced Open “eye”), a multimodal biomedical information retrieval system developed at the NLM that enables users to search for and retrieve relevant images and text. Our methods were evaluated in an international benchmarking forum in which we achieve a classification accuracy exceeding 90% at the highest level of a hierarchically organized image modality taxonomy, and 63.2% at the leaf level.