Mixed-Reality Training Simulation

mrtMixed-reality training simulations combine real people and physical environments with virtual people and places that are either controlled by humans or by artificial intelligence. These simulations are used by the military to conduct training exercises in an efficient, flexible and inexpensive environment.

Running mixed-reality simulations is complicated. They involve many systems including multiple video game engines, motion trackers, projectors and computers, all of which communicate with each other in real-time. The initialization, running, debugging and maintenance of these systems are extremely difficult tasks that, if not carefully performed, can use up the resource savings that mixed-reality simulations are designed to achieve.

These issues are addressed through a set of computer applications known as the Mixed Reality Toolbox – or, MRT. These applications help run and monitor various aspects of the simulations. MRT has four separate components: MetaTracker, Gadget Probe, Overview, and Launcher.

MetaTracker consolidates data from the simulations’ multiple tracking systems while Gadget Probe makes sure all of the gadgets are working properly. For example, if the trigger from a training gun in a military simulation is malfunctioning, Gadget Probe will display this error on one of the monitoring computers.

Overview then combines the information from Gadget Probe and MetaTracker to create a birds-eye-view, 3D visualization of the simulation that is helpful for monitoring the entire simulation from a computer screen.

With these three tools the user can troubleshoot issues that arise much more efficiently and simply than before.

The final component is Launcher. Before Launcher, all of the simulations’ run time parameters were done through a computer’s command line where a slight misspelling or an incorrect capitalization could cause the simulation to not function properly or run at all. Now, just by checking or unchecking boxes on a computer screen, a user can control many of the system’s operations including switching between the various simulations and deciding what computer runs the simulations.

After the simulation is over, MRT also helps by providing log files that make it easy to see and debug any problems that might have occurred.

The end result of MRT is that much of the headache of running and managing complex mixed-reality simulations is eliminated. MRT decreases the simulations’ runtime while also increasing the usability and monitoring abilities of the user.