The MARIE Project 

Principal investigator: Prof. Neil C. Rowe

MARIE is research on using captions to find multimedia, especially images and especially those on the World Wide Web. Work has been supported by DARPA, the U.S. Army, and the Naval Postgraduate School. The main aspects of the project are: (1) finding captions; (2) natural language processing; (3) image processing; (4) probabilistic reasoning; and (5) learning from experience.

The original testbed was a 1991 snapshot of the unclassified portion of the Photo Library of NAWC-WD, China Lake, California, a naval air test facility (available below). We have supplemented this with additional pictures and captions from the NAWC-WD Web pages. MARIE-1 covered 220 captions, and MARIE-2 covered 615 captions with relatively deep semantic processing. MARIE-3 was experimental. MARIE-4 is a Web crawler that searches the Web using Java servlet technology, and indexes all the captions it finds using a set of heuristics; it has been used to create indexes to millions of images, but its understanding of the captions is shallower than that of MARIE-2. Other aspects of the project have developed analysis methods for the regions and subjects of images, and important efforts have been devoted to making the processing efficient using ideas of optimal information filtering.

2005 book chapter, surveying caption extraction on the Web
Paper assessing the importance of context in finding Web captions
1999 overview paper on MARIE
Paper on MARIE-4 at its World Wide Web image-caption extraction
Short paper on the extension of the previous to audio and video
Short paper on automatically finding advertisements on Web pages
Paper on automatically finding and labeling the subject on a captioned imageIn PDF format
Paper on MARIE-3's World Wide Web caption extraction
Earlier overview paper on MARIE
Paper on image-region classification of MARIE
Paper on natural-language processing of MARIE-2
Paper by Eugene Guglielmo on natural-language processing of MARIE-1
Earlier paper on natural-language processing of MARIE
Paper on MARIE software handling abbreviations and misspellings
1997 paper on finding captions for Web images
1998 paper on mapping between image regions and caption concepts
Paper on reference resolution in image captions
Paper on use of caption understanding for decision support in command-and-control
1995 paper on processing challenges in MARIE
Paper on optimization of information filtering in MARIE
Paper on load balancing in parallel implementations of MARIEIn PDF format
Prof. Rowe's main page
Go to NPS CS Home Page