ONLINE, September 2001
Copyright © 2001 Information Today, Inc.
As the amount of digital content around the world grows each year, there is an increasing need for tools to help users control and manage this content. Content too is changing; more and more audio-visual content is being indexed, stored, and streamed over networks where bandwidth continues to expand. As digital libraries become more common, audio-visual content will likewise gain prominence. How do we search and browse this rich content efficiently and effectively? What technology can help us manage rich content that goes beyond the hitherto standard text-based query systems? MPEG-7, making its debut in September 2001, is ISO's newest standard. It specifically addresses these questions, and its impact will be felt in many multimedia application domains where content management and retrieval are important. Increasingly, this includes libraries and information departments.
According to University of California at Berkeley professors Peter Lyman and Hal R. Varian ("How Much Information"; http://www.sims.Berkeley.edu/how-much-info), "the world's total yearly production of print, film, optical, and magnetic content would require roughly 1.5 billion gigabytes (1.5 exabytes) of storage. This is the equivalent of 250 megabytes per person for each man, woman, and child on earth." Lyman and Varian claim that 93% of all information produced worldwide is stored in digital formhard drives in standalone PCs account for 55% of total storage shipped each year.
They go on to say that "published" individual information is 600 times larger than traditional published information. There are 80 billion new photographic images per year and, from 1990 to 1995, UNESCO reports there were 4,250 films produced annually worldwide. Consider that it takes approximately two gigabytes to store an hour of motion-picture images in digital form using the MPEG-2 compression standard. If the images in 4,500 full-length movies produced each year were converted into bits, the world's annual original cinematic production would, therefore, consume about 16 terabytes. Much of the non-textual digital information is part of the invisible, or deep, Web.
It is clear that not only is information growing hugely each year, but also that digitization has become the preferred means of storage. With this growth of digital contentparticularly multimedia contenta need for more sophisticated content management applications has developed that will facilitate effective and efficient management, searching, and browsing of the content. MPEG-7 provides the tools needed to create these applications in an open and non-proprietary environment, and is the gold standard for content management interoperability, not just for entertainment companies, but for every company, every industry, everywhere.
MPEG-7 is a standard for describing features of multimedia content that provides the world's most comprehensive set of audio-visual descriptions. These descriptions are based on catalog (title, creator, rights); semantic (the who, what, when, and where information about objects and events); and structural (the color histogram, measurement of the amount of color associated with an image, or the timbre of a recorded instrument) features of the AV content and leverages on AV data representation defined by MPEG-1, 2, and 4. MPEG-7 also uses XML Schema as the language of choice for content description, and will be interoperable with other leading standards such as Dublin Core (http://www.dublincore.org) and TV Anytime (http://www.tvanytime.org).
MPEG-7, formally named "Multimedia Content Description Interface," is the standard that describes multimedia content so users can search, browse, and retrieve that content more efficiently and effectively than they could using today's mainly text-based search engines. It's a standard for describing the features of multimedia content. However, MPEG-7 will not standardize the (automatic) extraction of AV descriptions/features. Nor will it specify the search engine (or any other program) that can make use of the description. It will be left to the creativity and innovation of search engine companies to manipulate and massage the MPEG-7-described content into search indices that can be used by their browser and retrieval tools.
It is important to note that MPEG-7 addresses many different applications in many different environments, which means that it needs to provide a flexible and extensible framework for describing audio-visual data. Therefore, MPEG-7 will define a multimedia library of methods and tools. It will standardize:
All domains making use of multimedia will benefit from MPEG-7 including:
Typical applications enabled by MPEG-7 technology could include:
There are several possible ways to search for visual content using the inherent structural features of an image. In this example, there are four image features detailed. The color histogram feature of an image (1) allows me to search for images that have the same color. Note: the position of the colors is not important but rather the amount of similar color in the image. The next feature, spatial color distribution (2), allows me to search for images where the location of the same color is important. You can see that the added object in the right-bottom flag does not affect this type of search. You can additionally search for images that have a similar edge or contour profile as in the spatial edge distribution search technique (3). Note that color does not make a difference to this type of search. Finally, you can see an example of searching by object shape (4). Here, the color and edge profiles are not important.
This example of MPEG-7 technology is demonstrable today by visual search applications developed by ETRI of South Korea and Mitsubishi Electric (U.K.). More information about these and other MPEG-7 applications can be reviewed on the MPEG-7 Industry Focus Group Web site (http://www. mpeg-7.com).
We live in the age of convergence, from the level of production through to distribution and consumption. The technical hardware and communication infrastructure is evolving and will soon reach the point where computing and communications will become embedded in everyday objects and environments. Media will also then become ubiquitous. Ubiquitous media will create a huge demand for new content, and meeting this demand must involve fundamental changes to all stages of media production, management, and delivery.
Media archives will become vast and interconnected pools of content, too large to be managed manually. Customization of content within programssubstitution of structural elements (characters, music, voices) according to viewer desires, content scaling for PDA, cell phoneswill not only be possible, but also easy and pleasant. MPEG-7 will enable the creation of tools (through its structured combination of low-level features and high-level metadata) for coping with this "outbreak" of generic content. One benefit of MPEG-7 is interoperability, the rapid uptake of MPEG-7, because it is built on enabling technologies and standards. MPEG-7 is harmonizing with other standards that have demonstrated success and acceptance in both traditional media and new media businesses, e.g., W3C (XML, XML Schema), Dublin Core, and TV Anytime, etc. This will allow rapid integration into your company's products.
MPEG-7 will enable the content management system at one subsidiary to leverage the content of another subsidiary. For example, AOL, CNN, and Warner Bros. could categorize, exchange, process, and manage assets across boundaries or along an entire supply chain, possibly from production house to advertising agency to CNN.
MPEG-7 uniquely provides comprehensive standardized multimedia description tools for content. Descrip- tions for the catalogue level (e.g. title); the semantic level (who, what, when, where); and the structural level (spatio-temporal region, color histogram, timbre, texture) will provide tools for creative developers to generate new waves of multimedia applications. With MPEG-7's sister standard, MPEG-4 (streaming), an ideal combination is made for solutions that require efficient streaming of content, content manipulation, and indexing and retrieval of that content. In particular, mobile application developers (NTTDocomo) have already begun to adopt these two standards, and the trend is set to hugely increase as the demand for visual and audio information services continues to grow.
MPEG works closely with representatives of the creative industries to ensure that the best possible protection of the rights of stakeholders is maintained both in content and in metadata.
Stored audio-visual content, gathered over the years by broadcasters, libraries, and publishers, becomes more valuable because, with MPEG-7 indexing technology, more comprehensive methods are available for users to access and retrieve more detailed descriptions of that content.
Information more focused to industry, including a publicly accessible news mailing list, is also available at the MPEG-7 Industry Focus Group Web site (http://www.mpeg-industry.com).
|Feature Extraction||MPEG-7 Scope||Search Engine|
Content analysis (D, DS)
Feature extraction (D, DS)
Annotation Tools (DS)
Description Schemes (DSs)
RefL MPEG-7 Concepts
Searching & Filtering
Neil Day ( email@example.com) is chair of the MPEG-7 Applications and Promotions to Industry Ad Hoc Group and the MPEG-7 Industry Focus Group, while working for Digital Garage Inc. Strategic Research & Development Department (www.garage.co.jp).
Comments? Email letters to the Editor at firstname.lastname@example.org.
Copyright © 2001, Information Today, Inc. All rights reserved.