Digital multimedia
Neil C. Rowe
U.S. Naval
Postgraduate School
Multimedia data can be important assets of government computer systems.� Multimedia data can be documents, statistics, photographs and graphics, presentations, video and audio of events, and software.� Examples include maps, video of meetings, slide presentations by consultants and vendors, graphs of budgets, and text of regulations.� Video of meetings of legislatures and other government organizations is particularly valuable as it makes government processes more visible to citizens and can encourage trust in government.� Multimedia is also particularly valuable in presenting geographical information (Gant & Ijams, 2004; Greene, 2001), a concern of all governments.� Added multimedia can also be used to more effectively deliver information to people, as with films, animations, sound effects, and motivational materials.
Multimedia information is important for digital government because it is often a more natural communication mode for people than text.� It is thus important that government be responsive to the needs and desires of citizens by providing it.� Much of the world is illiterate, and the ubiquity of television means even the literate often prefer watching video to reading text.� Some citizens have special needs: Blind people need audio, and deaf people need images.� Video and audio also convey information beyond text: A video of a legislature meeting contains subtleties not apparent from its transcript.�� Research has shown that multimedia is especially good at conveying explanatory information about functional relationships in organizations (Lim & Benbasat, 2002).� Research has also shown that people learn better from multimedia presentations than from conventional classroom instruction, and the multimedia provides a consistent experience available at any time unlike human instructors (Wright, 1993).
This article is to appear in Anttiroiko, A.-V., & Malkia, M. (Eds.), Encylopedia of Digital Government, Hershey, PA, USA: The Idea Group, 2006.
Management of multimedia data entails considerations not encountered with data that is solely text (Vaughan, 2003).� The main problem is data volume: A typical report can be stored in 20,000 bytes, but an typical 20 cm. square image requires 500,000 bytes to represent adequately, an audio clip that is one minute long requires around 1,000,000 bytes, and a typical 3 cm. square video clip that is one minute long requires around 50,000,000 bytes.� Compression techniques can reduce storage requirements somewhat; however, media that can be compressed significantly tend to be merely decorative and not very useful for digital government.� Multimedia size is especially a problem when transferring media between computers, especially with the limited data rates of conventional telephone lines and modems (Rao, Bojkovic, & Milovanovic, 2002).� So since digital government cannot be sure what technology its citizens have, it must be conservative in its use of multimedia.�
Distributed database technology (Grosky, 1997) can help manage multimedia data efficiently.� However, the human side of multimedia management requires a different set of skills than those of most computer support staff.� One needs "media specialists" familiar with the problems of the technology, including some staff with art training to address the aesthetic issues that arise.� Much multimedia management is time-consuming, so adequate personnel must be available.� Government can also choose to actively encourage development of a multimedia-supporting infrastructure by its industries (Mohan, Omar, & Aziz, 2002).
A first issue in using multimedia data is finding it.� Citizens often want to retrieve quite specific multimedia objects to meet their needs, and they can do this with a browser or a search engine (Kherfi, Ziou, & Bernardi, 2004).� This requires "metadata" describing the media objects such as size, date, source, format, and descriptive keywords.� A browser can provide a hierarchy of media objects that users can navigate.� This works best when media objects can be described in just a few words, or are characterized along a single dimension like date or place.� Otherwise, a keyword-based search engine is necessary, such as that provided by commercial services like Google but adapted to search only the government data.� Accommodating a broad range of citizens means keeping extensive synonym lists for keyword lookup, so that many possible ways of specifying the same thing will work.� In some cases a graphical specification may be a good way for the public to specify what they are looking for, such as a visual timeline or geographical display on which they click at the location they want.
�
Unfortunately, it is difficult to index and search non-text media for its contents.� Segmentation as by identifying shapes within images can be tried, but it is time-consuming and often unreliable.� So captions on media objects (text that describes them) are valuable (Rowe, 2005).� They can be directly attached to a media object or located near it in a document.� Captions directly attached include the name of the media file, descriptive keywords, annotations describing the object like the "alt" string associated with Web media, the text of clickable links on the Web that retrieve the object, text directly embedded in the media like characters drawn on an image, and data in different channels of the media like narration or "closed captions" for video.� Captions indirectly attached include text centered or in a special font above or below the media, titles and section headings of the document containing the media, special linguistic structures like ?The photo shows??, and paragraphs above or below the media.� Caption text can be indexed for a more precise keyword search than that obtained by just indexing all the words of the enclosing document (Arms, 1999).� This is what the ?media search engines? such as the image searchers of Google and AltaVista do, though their success rate at finding images is not as good as their success rate at text search.� Media retrieval is however an active area of research and new developments are appearing frequently.
Multimedia can enhance documents in many ways.� It can enliven information, and this is helpful for the often-unexciting information provided by governments.� But the primary concern of government must be for media that convey important information of their own.� Mostly this means delivery of multimedia information from the government to its citizens, though there are also issues in collection of information by government (Cheng et al, 2001) and communications within government.
Broadcast technology is the traditional method for a government to disseminate media, through newspapers, radio, and television.� Broadcast is a one-way technology.� This is fine for announcements and authoritarian governments, but interactivity is very important to a responsive and effective government.� So the Internet and especially the World Wide Web are increasingly preferred to deliver user-selected information and permit completion of forms.� The Web is well suited for multimedia.� It permits embedding of pointers to multimedia content in documents with much the same ease as text.� Web multimedia can range from informal illustrations to media retrieved from structured databases in response to queries entered on dynamic Web pages.� Media is particularly helpful for the illiterate, as graphical interfaces can enable access to the full power of computers without the necessity of words.
A key issue for digital government is the choice of media formats.� Government information systems intended for only internal use can follow a very few mandated formats for interoperability.� But much multimedia is for the public, and the public uses a diversity of computers, software, and networking services, and accessibility to all citizens is important.� So copies of important multimedia in different formats are essential.� Web images are currently mostly JPEG and GIF format with some PNG format.� Audio and video are more diverse: currently popular audio formats are WAV, RAM, MID, and ASX, and currently popular video formats are MPEG, SWF, MOV, FLI, AVI, and MP3.� Multimedia can also be software in various formats.� Not all these formats are supported by all Web browsers, so it is important for government to provide free viewer software for downloading so citizens can view any of its multimedia.� This generally restricts governments to using formats that have free open-source software for reading or viewing them.
Multimedia software of particular interest to government organizations is groupware, supporting collaborative activities of groups.� It can be used to run meetings of people at widely scattered locations so participants can see, hear, and read comments by other participants.� Groupware requires transmission of video, audio, and graphics between sites (Klockner et al, 1997).
Because of its bulkiness, multimedia is often best stored at a few centralized sites and retrieved from there.� That can entail logistical problems since video and audio in particular need to be delivered at a certain minimum speed to be viewable or listenable (Smith, Mohan, & Li, 1999).� Video or audio can be fully downloaded in advance of playing, but this entails a time delay and most citizens prefer real-time delivery (?streaming?).� Important applications with streaming are being accomplished including video meetings, video medicine, distance learning, multimedia mail, and interactive television.� Streaming is simplified if it is delivered by one-way broadcast, and this works well for standardized content such as training materials.� Traditional technology like television can also be effective in streaming of government media (Morisse, Cortes, & Luling, 1999) but requires citizens to access it at a particular time.� Another alternative is to supply citizens with an optical disk (CD-ROM or DVD) containing the media.
The biggest challenge with streaming is ensuring adequate bandwidth (rate of data transmission).� A single MPEG-1 compressed video of standard television-picture quality needs around 2 megabits per second (though videoconferencing and speeches can be adequate with less), and music audio requires 1 megabit per second.� Standard (ISDN) telephone lines provide 0.064 megabits/second, inadequate for both.� T-1, T-2, etc. lines can improve this to theoretically 1.5 megabits/second, but that is still inadequate for video.� "Digital subscriber lines" using cable television technology can provide higher bandwidths using network technologies such as ATM, but even those can be pressed to produce real-time video.� Data-compression techniques can reduce bandwidth somewhat; typical maximum compression ratios range from 2:1 for audio to 20:1 for images and 50:1 for video.� Other tricks with noncritical video are to periodically skip frames or decrease the size of the images.
Transmission bandwidth is also bounded by the delivery rate of the media from an archive.� Live video may be able to bypass storage and go directly onto the network. But in general, multimedia data must be archived in blocks for efficient memory management, though the blocks can be larger than those typical for text.� Magnetic tape and optical disks are less flexible in manipulating blocks than magnetic disks, so the latter is preferable for multimedia.� But it does take time for a disk head to go between blocks, so successive blocks can be put on different disks so that a block from one disk can be transmitted while the other block is being readied ("striping").
Besides bandwidth limits, networks can also have transmission delays ("latency"), which affects real-time interactive applications like videoconferencing.� Delays can be minimized by good network routing (Ali & Ghafoor, 2000; Gao, Zhang, & Towsley, 2003).� Using different paths to relay different parts of multimedia data from source to destination can reduce the effect of any one bottleneck.� If data loss is especially important to avoid, redundant data can be sent over the multiple paths.� But much video playback can tolerate occasional data loss.
Another transmission issue is the evenness of the arrival of multimedia data at a destination site ("burstiness" or ?jitter?), since unevenness can cause unacceptable starts and stops in audio or video.� This can happen when other network traffic suddenly increases or decreases significantly.� Usually video delays cannot exceed 0.1 seconds and music-audio delays 0.0001 seconds, so this is a key ?quality-of-service? issue.� Caching of data in storage buffers at the delivery site reduces the problem but effective multimedia buffers must be big.� Transmission by multiple routes will also help.
Multimedia delivery is still more difficult when the destination device is a small handheld one, since these are limited in memory, processor speed, and networking capabilities.� Although multimedia is better displayed on conventional computer hardware, many users prefer such small devices to access the Internet while engaged in other activities.� Then streaming is necessary and must be done with significant bandwidth and screen-size limits.� Managing the display of information on handheld devices is called "content repurposing" and is an active area of research (Alwan et al, 1996; Singh, 2004).
Other important technical issues in streaming include (Jadav & Choudhary, 1995):
� System architectures should be chosen for fast input and output; parallel ports are desirable.
� "Star" and fully-connected network topologies are desirable.� That may only be feasible with local networks for many applications.
� Switches should be preferred to routers on network connections since the routers have lower bandwidths and higher delays.
� Experimentation with the packet size for multimedia data may improve performance since the best size is hard to predict.
� Caching of frequently-used data can help efficiency, since some media objects will be much more popular than the others.
Thus without careful design there can be serious problems in multimedia delivery.� These problems can be negotiated between senders and receivers, either beforehand or at the time of transmission.� Generally speaking, worst-case or deterministic metrics for quality of service are more important than average-case or statistical ones for real-time multimedia since users have limits beyond which performance in unacceptable in speed, delay, or jitter.
Governments must archive many important records and this includes multimedia information.� The size of media objects necessitates high storage costs and slow access.� While the costs of storage media continue to decrease, now more data is being created in the first place.
A problem for archiving is the diversity of storage devices and media formats.� New ones arise continually, so archiving requires either archiving the hardware and software that can read the stored media even when the hardware, software, and formats are no longer being used for new media, or else periodically recopying the data into new devices and formats (Friedlander, 2002).� For instance, the Library of Congress of the United States still has audio stored on wire media, a technology long obsolete.� Video may be a particular problem in the future as there are several competing formats today as well as an ongoing shift to high-definition images.� Unfortunately, digital media is different from old books in that it can be virtually undecipherable without the right hardware and software to read it.� So governments must continually invest to support their media archives.
Another problem for digital government is to decide what to archive (Liu & Hseng, 2004).� Legal requirements may specify particular archiving, but governments have a responsibility to anticipate future data needs as well.� Copyright issues can actually simplify as media ages and enters the public domain.� But when money for archiving is limited, conflicting interests within the public may disagree as to what information to keep, and politics may needed to resolve this.
The increasing speed of computers and networks and the decreasing cost of digital storage will enable increased use of multimedia in computer systems in the future.� This will enable governments to store and offer increasing amounts of multimedia data for their citizens.� Media like video of public meetings, census maps, and graphics documenting government practices will become routinely available so that all citizens can access them without needing to be physically present at government facilities.� The main challenges are indexing all this media so citizens can find it, and delivering it (particularly video) across a network fast enough to be useful.� Speed and cost improvements alone will not solve all the technical problems, as other issues discussed above must be addressed too.� It will take a number of years to reach levels of adequate government media service even in technologically advanced countries.
Multimedia is a natural way for people to communicate with computers, more natural than text, and should become an increasingly common mode for people to learn about and participate in their governments.� Good planning is necessary, however, because the data sizes of multimedia objects can be significantly larger than those of text data.� This means that media delivery can be unacceptably slow or uneven with current technology, and this limits what can be offered to citizens.� Nonetheless, the technology to support media access is improving, and governments will be able to exploit this.
Ali, Z., & Ghafoor, A. (2000, November).� Synchronized delivery of multimedia information over ATM networks.� Communications of the ACM, 43 (11), 239-248.
Alwan, A., Bagrodia, R., Bambos, N., Gerla, M., Kleinrock, L., Short, J., & Villasenor, J. (1996, April).� Adaptive mobile multimedia networks.� IEEE Personal Communications, 3 (2), 34-51.
Arms,
L. (1999, Fall).� Getting the picture:
observations from the Library of Congress on providing access to pictorial
images.� Library Trends, 48(2), 379-409.
Cheng,
W., Chou, C., Golubchik, L., Khuller, S., & Samet, H. (2001, May).� Scalable data collection for Internet-based
digital government applications.� Proc. First National Conference of Digital
Government Research, Los Angeles, CA, 108-113.
Friedlander, A. (2002, April).� The National Digital Information Infrastructure Preservation Program.������ D-Lib Magazine, 8 (4).� Retrieved January 13, 2006 from www.dlib.org/dlib/april02/friedlander/04friedlander.html.
Gao, L., Zhang, Z.-L., & Towsley, D. (2003, December).� Proxy-assisted techniques for delivering continuous multimedia streams.�� IEEE/ACM Transactions on Networking, 11� (6), 884-894.
Greene, R. (2001).� Opening access: GIS in e-government.� Redlands, CA: ESRI (Environmental Systems Research Institute) Press.
Grosky, W. (1997, December).� Managing multimedia information in database systems.� Communications of the ACM, 43 (12), 72-80.
Jadav, D., & Choudhary, A. (1995, Summer).� Designing and implementing high-performance media-on-demand servers.� IEEE Parallel and Distributed Technology, 3 (2), 29-39.
Kherfi, M., Ziou, D., & Bernardi, M. (2004, March).� Image retrieval from the World Wide Web: issues, techniques, and systems.� ACM Computing Surveys, 36 (1), 35-67.
Klockner, K., Mambrey, P.,
Printz, W., & Schlenkamp, M. (1997, September).� Multimedia groupware design for a distributed government.� Proc.
23rd EUROMICRO Conf., 144-149.
Lim, K., & Benbasat, I.
(2002, Summer).� The influence of
multimedia on improving the comprehension of organizational information.� Journal of Management Information
Systems, 19 (1), 99-127.
Liu, J.-S., & Hseng,
M.-H. (2004).� Mediating team work for
digital heritage archiving.� Proc. Intl. Conf. on Digital Libraries,
Tucson, AZ, 259-268.
Mohan, A., Omar, A., & Aziz, K. (2002, December).� Malaysia's multimedia Super Corridor Cluster: communication linkages among stakeholders in a national system of innovation.� IEEE Transactions on Professional Communications, 45 (4), 265-275.
Morisse, K., Cortes, F.,
& Luling, R. (1999, June). �Broadband multimedia information service for European
parliaments.� Proc. Intl. Conf. on Multimedia Computing and Systems, II:
1072-1074.
Rao, K., Bojkovic, Z., &
Milovanovic, D. (2002).� Multimedia communication systems:
techniques, standards, and networks.�
Englewood Cliffs, NJ: Prentice-Hall.
Rowe, N.
(2005).� Exploiting captions for Web
data mining.� Chapter 6 in Web Mining: Applications and Techniques,
ed. A. Scime, The Idea Group, pp. 119-144.
Singh, G. (2004,
January-March).� Content
repurposing.� IEEE Multimedia, 11 (1), 20-21.
Smith, J., Mohan, R., &
Li, C.-S. (1999, October).� Scalable
multimedia delivery for pervasive computing.�
Proc. 7th ACM Conf. on
Multimedia, Orlando, FL, I: 131-140.
Vaughan, T. (2003).� Multimedia: making it work, sixth edition.� New York: McGraw-Hill Osborne Media.
Wright, E. (1993, Winter).� Making the multimedia decision: strategies for success.� Journal of Instructional Delivery Systems, 7 (1), 15-22.
bandwidth: Amount of data (measured in bits per second) that can be transmitted across a network.
broadcast: Transmission of some data in one direction only to many recipients.
caption: Text describing a media object.
data compression: Transforming of data so that it requires fewer bits to store.
groupware: Software to support collaborative work between remotely located people.
jitter: Uneveness in transmission of data, an important problem for streaming video.
media search engine: A Web search engine designed to find media (usually images) on the Web.
metadata: Information describing another data object such as its size or caption.
multimedia: Data that includes images, audio, video, or software.
quality of service: The quality of data transmission across a computer network as a composite of several factors.
streaming: Video or audio sent in real time from a source, thereby reducing storage needs.