Combine unsupervised learning and heuristic rules to annotate organism morphological descriptions

Abstract

Abstract Biodiversity literature is a comprehensive compilation of information on living organisms and fossils. Rich factual information on characteristics of organisms is presented in narrative form, hence limiting its repurpose and reuse. Transforming narrative information into atomic forms has been of special concern to informatics researchers and biological researchers alike. Research done previously shows similar results but lacks a detailed, scientific evaluation that would help illuminate the problem and eventually lead to a higher performance approach. Due to the sublanguage nature of morphological descriptions, it is thought that general‐purpose nature language processing (NLP) tools are not effective in this application. A heuristic‐based approach has been suggested in the literature. In this paper, we report our experiments with such an approach, where a set of simple, intuitive heuristic rules, informed by results of an unsupervised learning algorithm, is used to segment taxonomic descriptions and identify the organs along with their associated character/value pairs ( color=white, shape=ovoid ). This model system allows us to investigate the character annotation problem further, study the characteristics of morphological descriptions, identify the areas where the system fails, and suggest ways to address those failures. One such suggestion is to make use of general‐purpose syntactic parsers in a controlled manner.