An experiment in machine learning from the University of Washington hopes to teach itself “everything about anything,” at least where images are concerned.
Dubbed LEVAN (Learning Everything about Anything), the algorithm’s goal is to take a concept and construct a visual archive of as many permutations as it can. Think of it as an improved version of Google’s image search, broken down by the different permutations of a category, rather than what Google thinks you might be interested in.
“It is all about discovering associations between textual and visual data,” said Ali Farhadi, a UW assistant professor of computer science and engineering, in a statement. “The program learns to tightly couple rich sets of phrases with pixels in images. This means that it can recognize instances of specific concepts when it sees them.”
The “concepts” LEVAN archive include a list of random nouns, 175 at present, ranging from “aeroplane” through “buckinghampalace” to “eating,” “Obama,” and “walking”. LEVAN searches millions of texts on Google Books, looking for modifiers to its concepts, and only keeping the visual ones: “my dog” is discarded, but “down dog” and “newfoundland dog” are kept. Then the algorithm goes out and searches the available image repositories until it finds metadata that matches what it seeks.
The goal is to help make it easier to curate image data, not necessarily for search engines but the average joe.
“Major information resources such as dictionaries and encyclopedias are moving toward the direction of showing users visual information because it is easier to comprehend and much faster to browse through concepts,” Santosh Divvala, a research scientist at the Allen Institute for Artificial Intelligence and an affiliate scientist at UW in computer science and engineering said in a statement. “However, they have limited coverage as they are often manually curated. The new program needs no human supervision, and thus can automatically learn the visual knowledge for any concept.”
Eventually, the researchers say, they hope to develop an app that will help quickly tag images for archiving. For now, the LEVAN repository will remain as an archive, while the technology itself will be presented at the Computer Vision and Pattern Recognition annual conference in Columbus, Ohio.
If you’re looking for a specific image of a specific thing, it’s hard to top Google or Bing. Both Google and Bing have some useful features that LEVAN lacks—such as sorting by size or by Creative Commons licenses, for example. But if you’re searching for an image—of an unknown breed of dog, for example—and don’t quite know what it is, LEVAN may be of greater use.
Unfortunately, given the feverish demand for computer science engineers, it’s likely the technology behind LEVAN will be absorbed through an “acqui-hire” of the team’s talent. But for now, if you’re stuck looking for an image, click on over.
As PCWorld's senior editor, Mark focuses on Microsoft news and chip technology, among other beats. He has formerly written for PCMag, BYTE, Slashdot, eWEEK, and ReadWrite.