virtual reality transfer protocol (vrtp)

vrtp is being developed to provide client, server, multicast streaming & network-monitoring capabilities
in support of internetworked 3D graphics and large-scale virtual environments (LSVEs).

Status	Software	Charter	Why	Motivation	What	Components	Details	References

Status

This is a research working group in an early stage of development and experimentation. You have been warned!

Since 1995 we have actively collaborated with a number of researchers on the specifics of vrtp architectural design. Currently four Ph.D. students and several master's students are working on components expected to become part of vrtp.

We applied to become a Web 3D Consortium Working Group in January 1998, and our charter was approved in February 1998. We have established the vrtp mail list and hypermail archive at the www.web3D.org site.

Our primary motivation to open up participation in vrtp development is to execute vrtp implementation quickly and thoroughly. We want to to maximize the network abilities of shared 3D worlds. We will also want to keep pace with the subsequent explosion of network demand by large-scale interconnected 3D virtual environments.

Software

The following distributions are individually available. They are structured to be mutually compatible in the vrtp directory structure.

vrtp-related software
vrtp working group Web site (this page) Interim update page vrtp.zip, vrtp.tar.gz
DIS-Java-VRML Web site relative link (if installed locally) Interim update page Most recent download
Real-time Transport Protocol (RTP) Monitor Web site relative link (if installed locally) Interim update page Most recent downloads
rtpMonitor.zip, rtpMonitor.tar.gz
Dial a Behavior Protocol (DABP) Available soon
Recursive Ray Acoustics (RRA) Sonar Algorithm Web site relative link (if installed locally) No interim updates Most recent download

**vrtp-related software**
vrtp working group	Web site	(this page)	Interim update page	vrtp.zip, vrtp.tar.gz
DIS-Java-VRML	Web site	relative link (if installed locally)	Interim update page	Most recent download
Real-time Transport Protocol (RTP) Monitor	Web site	relative link (if installed locally)	Interim update page	Most recent downloads rtpMonitor.zip, rtpMonitor.tar.gz
Dial a Behavior Protocol (DABP)	Available soon
Recursive Ray Acoustics (RRA) Sonar Algorithm	Web site	relative link (if installed locally)	No interim updates	Most recent download

The Bamboo plugin architecture is not yet integrated in any way, but is our goal componentization architecture. Bamboo is available via the Bamboo web site.

Concurrent related work: Extensible 3D (X3D) Specification Task Group. It has delayed some of our efforts in vrtp, but promises even greater eventual impact. Both efforts are proceeding in parallel with excellent progress.

Charter

Our charter to start a virtual reality transfer protocol (vrtp) working group in the Web 3D Consortium has been approved. It remains available for comment. The VRML Review Board (VRB) had the following comments, which need to be integrated into our work plans.

the positioning of vrtp with respect to higher-level architectures such as LivingWorlds (it shares many features with transport protocols, but also provides special treatment for things such as behaviors)
the relationship between vrtp, DIS and HLA, and whether they are intended to be decoupled
the position with respect to other network protocol layers, e.g. reliable UDP-based protocols or streaming protocols, and whether vrtp is transport independent
the use of vrtp as a protocol vs. its client/server/ monitoring components -- in some ways vrtp seems like a protocol, in others it seems like a framework of APIs

Answering these important questions will first be addressed via vrtp mail list discussion.

Why vrtp?

The capabilities of the Virtual Reality Modeling Language (VRML) permit building large-scale virtual environments using the Internet and the World Wide Web. However the underlying network support provided by the hypertext transfer protocol (http) is insufficient for large-scale virtual environments. Additional capabilities for many-to-many peer-to-peer communications plus network monitoring need to be combined with the client-server capabilities of http. To accomplish this task, we present a detailed design rationale for the virtual reality transfer protocol (vrtp). vrtp is designed to support interlinked VRML worlds in the same manner as http was designed to support interlinked HTML pages. vrtp will be optimized in two ways: on individual desktops and across the Internet. vrtp appears to be a necessary next step in the deployment of all-encompassing interactive internetworked 3D worlds.

What does success look like?

vrtp will be public-domain software that will be available for all desktop computers. Installing vrtp will provide any machine with client, server, peer-to-peer and network monitoring capabilities. People will use vrtp with 3D browsers to navigate and join large, interactive, fully internetworked, VRML worlds. Because the scene graph paradigm is common across a number of Web-based 3D programming methodologies, vrtp interfaces will also be developed that are adaptable to other application programming interfaces (APIs) such as Java3D and VRML-NG.

Basically we are building network connectivity for cyberspace, optimized for live 3D and scalable with the Web.

Caveat surfer: many parts of this effort have a long way to go! There is a lot of heavy-duty detail and behind-the-scenes work going on. It is not a good project to join in order to do a little quick hacking (at least not yet). It is a good project to join if you are interested in contributing to a serious long-term effort that integrates 3D graphics with internetworking. You might take a look at the DIS-Java-VRML working group if you want to see how these efforts can grow.

Motivation

The following book chapter is an in-depth examination of the interrelated networking and 3D graphics forces which motivate this work.

Graphics Internetworking: Bottlenecks and Breakthroughs

Don Brutzman
Chapter four, Digital Illusions, Clark Dodsworth editor,
Addison-Wesley, Reading Massachusetts, August 1997.
www.stl.nps.navy.mil/~brutzman/vrml/breakthroughs.html

Although networking is considered to be "different" than computer graphics, network considerations are integral to large-scale interactive three-dimensional (3D) graphics. Graphics and networks are now two interlocking halves of a greater whole: distributed virtual environments. New capabilities, new applications and new ideas abound in this rich intersection of critical technologies. Our ultimate goal is to use networked interactive 3D graphics to take full advantage of all computation, content and people resources available on the Internet.

Network breakthroughs repeatedly remove bottlenecks and provide new opportunities. A pattern appears as we attempt to scale up in capability and capacity without limit: every old bottleneck broken reveals another. Understanding the bottlenecks, the corresponding solutions and potential upper bounds to growth permits us to develop effective networked graphics. When we overcome current bottlenecks, "effectively networked graphics" will simply mean "applications."

Internetworked graphics can be examined from the perspectives of connectivity, content, interaction, economics, applications and personal impacts. Internetworking refers to the ability to seamlessly interconnect multiple dissimilar networks globally. Connectivity has numerous dimensions including capacity, bandwidth, protocols and the many-to-many capabilities of multicasting. Content equals the World Wide Web and includes any type of information, dataset or data stream that might be used in the graphics environment. Interaction implies minimal latency, a sense of presence, and the ability to both access and modify content. The economics of networked graphics environments is developing rapidly and principal forces can be identified. Applications drive infrastructure development and are the most exciting part of networked graphics. Finally, the personal impacts which accompany these developments range from trivial to profound as high-quality interactive internetworked computer graphics become the norm on all computers.

Where does vrtp live?

Just as the functionality of ftp, telnet and gopher were integrated in http to support HTML, we are integrating and optimizing protocols for client, server, peer-to-peer and network monitoring in vrtp to support VRML.

Combining and optimizing existing protocols is a well-understood challenge, not a leap of faith. It is also becoming clear that such functionality can be used for other 3D scene-graph APIs in addition to VRML.

diagram: HTML needed http, now VRML needs vrtp

For a long time, many of our conversations with people about VE networking ended up in the same frustrating question: "are you discussing client-server OR peer-to-peer?
We now realize this is a false dichotomy. A full spectrum of functionality exists between client-server and peer-to-peer. vrtp must support this entire spectrum well.

vrtp Components

We have done a lot of work implementing large-scale virtual environments over the network. Most people have huge laundry lists on what functionality is required for a scalable network architecture. The real challenge is squeezing down on all these issues and understanding the core requirements. The design rationale explains in detail why these key components are needed to support the full variety of network needs in virtual environments.

vrtp Components
Client	looking at someone else's live 3D world
Server	showing other people your live 3D world
Peer-to-Peer	for scalable behavior interactions between numerous interacting worlds
Monitoring	so that everything just works. why? because nobody really knows the state of the Internet
Plugin Kernel	Bamboo plugin architecture for Java and C++ dynamic class loading

More details please...

Well, the best answer for technical details is an implementation, but we don't have one yet. Currently we are working on a variety of components. Initial implementation is planned for mid 1999. The design goal attributes for vrtp follow.

vrtp is:	vrtp is not:
a framework for best-of-breed protocols a combination of existing software a way to give user scenes easy access to a full spectrum of network capabilities support for multi-user and avatar interactions vrtp:// URL & browser API extensions for client OR server OR multicast capability easy to use (otherwise we're not done yet) all about simplification & streamlining	possible using http yet another transport protocol, or a reliable multicast protocol a competitor to existing protocols a multi-user or avatar specification a step in an untested direction hard for 3D-content authors to understand about adding complexity

Here are a matched set of simple and detailed architecture diagrams which show how the various vrtp components fit together.

Primary paths for dataflow are mostly vertical, up & down the columns in the diagram.

3D client functionality is in the orange box for a variety of 3D scene graphs. Multicast streaming, server and monitoring components are in the yellow, green and blue box columns.

The light-blue "universal platform" box indicates that vrtp will be implemented in both Java and C++ (via Bamboo plugins), enabling interoperability with all platforms of interest.

(Larger diagrams can also be seen side by side.)

A major piece of work accomplished in 1998 was the design of the vrtp streaming behaviors component.

From the bottom up: behaviors arrive as multicast packets over the wire, are subscribed to via the Area of Interest Manager (AOIM), are buffered in a separate thread, have Real-time Transport Protocol headers examined for synchronization, packet contents are decoded using the XML-based dial-a-behavior protocol, then the entity dispatcher sends events to the 3D scene graph.

simple diagram: vrtp streaming behavior stack

detailed diagram: vrtp streaming behavior stack

Prototype implementation of this functionality has been produced during implementation of the DIS-Java-VRML software distribution.

We intend to use Bamboo as the plugin architecture which will selectively load & dynamically connect vrtp components on the desktop at runtime.

Bamboo is a collection of the core mechanisms common to networked VEs. It is an API toolkit for serious programmers. It has a dynamically extendible runtime environment. Bamboo is based on OpenGL++, which implies OpenGL, C++, and STL (standard template library). OpenGL++ is a multi-platform (SGI, PC, Macintosh, etc.) visualization toolkit that contains the best of Performer and Inventor. Bamboo sits along-side rather than on top of OpenGL++. Bamboo is designed based on plug-ins.

The Bamboo web site has additional detail, mailing list information and the latest set of distributions.

Annotated References

vrtp design rationale. This is the project manifesto. It describes the network requirements driving Large-Scale Virtual Environments (LSVEs), why desktops need client, server, peer-to-peer capabability, why network monitoring is the essential missing element, and how we intend to build vrtp.
Brutzman, Don, Zyda, Mike, Watsen, Kent and Macedonia, Mike, "virtual reality transfer protocol (vrtp) Design Rationale," Workshops on Enabling Technology: Infrastructure for Collaborative Enterprises (WET ICE): Sharing a Distributed Virtual Reality, Massachusetts Institute of Technology, Cambridge Massachusetts, June 18-20 1997. Available in .pdf, along with the WET ICE workshop presentation slides in .pdf and PowerPoint.
Streaming behaviors prototype: DIS-Java-VRML Working Group. The IEEE Distributed Interactive Simulation (DIS) Protocol is used to communicate state information (such as position, orientation, velocities and accelerations) among multiple entities participating in a shared network environment. Java is a portable networked programming language that can interoperate on any computer which includes a Web browser. The Virtual Reality Modeling Language (VRML) enables platform-independent interactive three-dimensional (3D) graphics across the Internet, and can be used to compose sophisticated 3D virtual environments.
The DIS-Java-VRML Working Group is developing a free software library, written in Java and interoperable with both DIS and VRML. This has been our prototype for developing vrtp streaming stack functionality. Primary code contributors are Don McGregor and Don Brutzman of NPS. It solves a number of important technical tasks, including multicast connectivity into a web browser.
The DIS-Java-VRML Annotated References document provides an extensive list of additional technical references.
Background work: the "big picture." The book chapter motivating the vrtp work is synopsized near the top of this page.
Brutzman, Don, "Graphics Internetworking: Bottlenecks and Breakthroughs," chapter, Digital Illusions, Clark Dodsworth editor, Addison-Wesley, Reading Massachusetts, to appear 1997. Available at www.stl.nps.navy.mil/~brutzman/vrml/breakthroughs.html
Tutorial on internetworked 3D graphics. An overview of network infrastructure issues relating to interactive 3D graphics. Relevant networking capabilities are demonstrated in real time. Topics include: how Multicast Backbone (MBone) functions support real-time video and audio; collaborative work tools that functionally incorporate computer graphics across the Internet; and the relationship of OpenGL, VRML, Java, and HTML to Internet, streaming, and distributed interactive simulation protocols. This knowledge helps 3D graphics developers and users in building and integrating interactive applications that use the Internet effectively. These tutorials for SIGGRAPH/SIGCOMM 97, SIGGRAPH 98 and SIGGRAPH 99 are intended to bridge the distance separating researchers in the two premiere professional societies in graphics and networking: ACM Special Interest Group on Graphics (SIGGRAPH) and ACM Special Interest Group on Data Communications (SIGCOMM).
Rhyne, Theresa-Marie, Barton, Bob, Macedonia, Mike and Brutzman, Don, "Beyond Bottlenecks and Roadblocks: Internetworked Computer Graphics," SIGGRAPH/SIGCOMM 97 and SIGGRAPH 98 tutorial. Available at www.crcg.edu/events/sig98/course.html
Related work: internetworked graphics tutorials. Available at www.crcg.edu/events/sig98/course.html
Related work: Dial-a-Behavior Protocol project. Existing behavior protocols for entity-entity interactions are either rigid & heavyweight (e.g. IEEE Distributed Interactive Simulation - DIS protocol) or proprietary & nongeneral (e.g. networked game protocols). A key limitation in scaling up to many participants in large-scale virtual environments (LSVEs) is inter-entity communications. We will build a dynamically modifiable behavior network protocol for virtual environments, i.e. Dial-a-Behavior Protocol (DBP). DBP will permit protocol designers to define both syntactic and semantic packet contents, and also provide tools for modifying protocol formats on-the-fly without stopping a networked exercise. This approach will allow experimental evaluation of protocol performance to replace the ad hoc design approaches currently in use. A series of applications will be demonstrated to validate DBP functionality.
Don McGregor's dial-a-behavior protocol (DBP) alpha implementation will be publicly available in February 1999.
Related work: CBone. In the 1995 report entitled Virtual Reality: Scientific and Technological Challenges, the National Research Council recommends establishment of a nationwide, open virtual environment research network. The purpose of that network is to allow universities, academia and government agencies to examine the technological issues involved with networking large-scale virtual environments (LSVEs). Proposed work includes the study of design issues behind virtual environment protocols, the development of tools for rapid development of virtual environment protocols, and the development of networked software architectures capable of supporting LSVEs. We intend to accomplish the goals laid out in the NRC report. There are two major components to this effort. One component is the establishment of the high-speed OpenVE network. We call this network the Cyberspace Backbone (CBone) in homage to the Multicast Backbone (MBone) currently used for VE network testing. The second component is study and development of a virtual reality transfer protocol (vrtp), an applications-layer protocol to be used for communicating peer-peer and client-server state information among participants in internetworked LSVEs. We are examining a gamut of issues including tools for rapid protocol specification/reconfiguration, DIS, dial-a-protocol DIS and scalable networked software architectures. This paper outlines the design rationale for establishing the CBone.
Brutzman, Don, Zyda, Mike and Macedonia, Mike, "Cyberspace Backbone (CBone) Design Rationale," 15th DIS Workshop on Standards for the Interoperability of Distributed Simulations, Institute for Simulation and Training, Orlando Florida, September 16-20 1996, paper 96-15-99. Available in PostScript with presentation slides in PostScript and PowerPoint.
In December 1998 we commenced global multicast behavior streams as part of DIS-Java-VRML MBone testing. We are establishing this continuous test program as a preliminary CBone.
Background work: scaling up behavior streams via Area-of-Interest Management (AOIM). Mike Macedonia's CG&A paper and dissertation make fundamental contributions on how many interacting entities can effectively communicate using multicast groups and area-of-interest management. Spatial, temporal and functional communication partitions that exist in the real world can be used to structure inter-entity communications in virtual worlds. Mike shows how use of multicast groups over local-area and wide-area networks provide a way to scale up live behaviors.
Macedonia, Michael R., Zyda, Michael J., Pratt, David R., Brutzman, Donald P., and Barham, Paul T., "Exploiting Reality with Multicast Groups," IEEE Computer Graphics and Applications, vol. 15 no. 5, September 1995, pp. 38-45. Available at www.npsnet.nps.navy.mil/npsnet/publications/IEEECGA.ps.Z
Macedonia, Michael R. "A Network Software Architecture for Large Scale Virtual Environments" Ph.D. Disertation, Naval Postgraduate School, Monterey, California, June 1995. (PDF)
Book. Modeling and Simulation: Linking Entertainment & Defense, Editors: Michael Zyda and Jerry Sheehan, National Academy Press, September 1997, ISBN 0-309-05842-2, 181 pages. www.npsnet.nps.navy.mil/zyda/NRC.html
Book. Virtual Reality: Scientific and Technological Challenges, Editors: Nathaniel I. Durlach and Anne S. Mavor, Committee on Virtual Reality Research and Development, National Research Council, National Academy of Sciences Press, Washington, DC 1995. Sections written or with major contributions by Mike Zyda, chair of the NPSNET research group: Chapters - "Executive Summary", "Overview", "Computer Hardware and Software for the Generation of Virtual Environments", and "Networking and Communications", ISBN 0-309-05135-5. www.nap.edu/readingroom/reader.cgi?auth=free&label=ul.book.0309051355 (or if that link changes) www.nap.edu/readingroom/ (then select Computer Science, then select Virtual Reality).
Background work: mobile agent networking considerations. Stone, Steve, Zyda, Mike, Brutzman, Don and Falby, John, "Mobile Agents and Smart Networks for Distributed Simulations," 14th DIS Workshop on Standards for the Interoperability of Distributed Simulations, Institute for Simulation and Training, Orlando Florida, March 11-15 1996, paper 96-14-133, pp. 909-917. Available at www.npsnet.nps.navy.mil/npsnet/publications/Mobile.Agents.and.Smart.Networks.for.Distributed.Simulations.pdf
Background work: internetwork infrastructure for virtual environments. Our original papers describing the basis for vrtp. Yes, we've been working on these problems for a while.
Brutzman, Donald P., Macedonia, Michael R. and Zyda, Michael J., "Internetwork Infrastructure Requirements for Virtual Environments," First Annual Symposium of the Virtual Reality Modeling Language (VRML 95), sponsored by ACM SIGGRAPH, San Diego California, December 14-15, 1995. Available at www.stl.nps.navy.mil/~brutzman/vrml/vrml_95.ps with slides at www.stl.nps.navy.mil/~brutzman/vrml/vrml_95.slides.ps
Brutzman, Donald P., Macedonia, Michael R. and Zyda, Michael J., "Internetwork Infrastructure Requirements for Virtual Environments," NII 2000 Forum of the Computer Science and Telecommunications Board, National Research Council, Washington DC, May 23-24 1995. Unpublished invitational workshop. Available at ftp://taurus.cs.nps.navy.mil/pub/auv/brutzman/nii_2000.ps.Z or nii_2000.txt
Background work: development of VRML. The meteoric development of the Virtual Reality Modeling Language (VRML) was one of the most important standardsdevelopments in graphics and internetworking. An informal, intense, open and collaborative design process has worked. VRML version one defined a concise and workable scene description language. Version two refined this language to incorporate behavioral animation mechanisms compatible with the current World Wide Web. Networked interoperable interactive 3D graphics are now feasible for everyone's computer. This panel presentation examined lessons learned over the past two years, where VRML is going, and how VRML is triggering fundamental changes in the economics, mindset and membership of the graphics community.
Brutzman, Don, Pesce, Mark, Bell, Gavin, van Dam, Andy and AbiEzzi, Salim, "VRML: Prelude and Future," Proceedings, Association for Computing Machinery (ACM) Special Interest Group on Computer Graphics (SIGGRAPH 96), New Orleans Lousiana, August 4-9 1996. Available at www.stl.nps.navy.mil/~brutzman/vrml/siggraph96panel.html
As of 1999, VRML is 3D on the Web. Here is Don Brutzman's VRML page.
More Web 3D technologies based on scene graphs are also appearing, including Java3D, Fahrenheit and VRML-NG. The vrtp project will track and support them. Hopefully, interoperability of all these important efforts can be maximized through Web 3D Consortium coordination efforts.

The Uniform Resource Locator (URL) for this home page is http://www.web3D.org/WorkingGroups/vrtp

Contact information: Don Brutzman (brutzman@nps.navy.mil)
(4 September 1999) (official NPS disclaimer)