Images to be alongside text in future search engines


Next generation search engines may work not only with text but also with images thanks to new technologies for visual content retrieval developed by Vitalas, a European Commission-funded project whose prototype system will be tested from next week by Exalead, the fourth biggest search engine in the world.

The new application is currently being tested by news agencies such as AFP and Belga for the automatic annotation and retrieval of pictures from their archives. “Often journalists are not able to pick the right image, not because it is not present in their archives but because it is impossible or too time-consuming to find it”, explained Alexis Joly, a researcher at Inria, one of the institutes behind the development of the new technology. With a visual content retrieval system, this problem will disappear, he told a conference in Lille on 10 December.

Applications for the new developing technology range from biology to security issues. “After having tried it, the French judiciary police showed enough interest to buy 200 licenses of the prototype”, said Joly, explaining that the software has helped localise paedophile networks around the world through the identification of particular objects in the background of pictures uploaded for illegal use. The prototype is able to match features, like a chair in a room, within very large databases of images.

The software developed by the Vitalas project only focuses on “rigid details” and not on elements subject to change. This means that is not able to identify faces. Facial recognition is indeed the other big branch of visual content retrieval. “They are two different families of software”, pointed out Nozha Boujemaa, scientific coordinator of the Vitalas project and director of research at Inria, speaking at the Lille conference celebrating 40 years of Inria  (Institute National de Recherche en Informatique et en Automatique).

Facial recognition is currently used by law enforcement and border control agencies, particularly at airports. In addition, the software has already found a use on the internet, since search engines such as Google and Exalead are currently offering facial detection applications.

However, facial detection is a step behind facial recognition because it solely involves spotting a face in an image and not its identification. “The application of facial recognition software on the internet is not for today neither for tomorrow”, underlined Boujemaa, explaining that the technology is not yet advanced enough to be used on a large scale. Moreover, “while the technology is quite ahead in terms of frontal face recognition, there are still relevant problems in the recognition of a profile”, added Boujemaa.

Although far from being applied on a large scale, these innovative technologies already pose real threats to privacy and security, warned the EU agency for network and information security, Enisa. In a recent report, Enisa pointed out that “while face recognition allows the linking of profile data involving the person’s physical body, content-based image retrieval allows the linking of location data through the recognition of common objects in images. It opens up the possibility of deducing location data from apparently anonymous profiles containing images of users’ homes. This can lead to stalking, unwanted marketing, blackmail and all the other threats associated with unwanted disclosure of location data”.

Subscribe to our newsletters