Using Linear Algebra for Intelligent Information Retrieval

Abstract

Abstract Currently most approaches to retrieving textual materials from scientic databases depend on a lexical match between words in users requests and those in or assigned to documents in a database Because of the tremendous diversity in the words people use to describe the same document lexical methods are necessarily incomplete and imprecise Using the singular value decomposition SVD one can take advantage of the implicit higherorder structure in the association of terms with documents by determining the SVD of large sparse term by document matrices Terms and documents represented by \t of the largest singular vectors are then matched against user queries We call this retrieval method Latent Semantic Indexing LSI because the subspace represents important associative relationships between terms and documents that are not evident in individual documents LSI is a completely automatic yet intelligent indexing method widely applicable and a promising way to improve users access to many kinds of textual materials or to documents and services for which textual descriptions are available A survey of the computational requirements for managing LSIencoded databases as well as current and future applications of LSI is presented Key words indexing information latent matrices retrieval semantic singular value decomposition sparse updating AMSMOS subject classications A