Calian Develops and Delivers Virtual Reality Trainer with Speech Recognition for Joint Terminal Attack Controller (JTAC) Training

3D rendering virtual reality trainer used by Canadian Armed Forces with speech recognition for joint terminal attack controller.

The Customer

The Canadian Armed Forces (CAF) deploys forces around the world, combining army, navy, air force and special operations forces to meet the government’s needs. On land, joint terminal attack controllers (JTAC) play a vital role on the front lines, coordinating fires from any source. The National Research Council (NRC) funded a research project to combine speech recognition technology with virtual reality technology to develop an immersive training solution to train JTACs for the CAF.

The Challenge

Training JTACs is a complex undertaking. JTACs are expected to be able to call for fire from any source— conventional tube artillery, mortars, rocket artillery, fixed-wing aircraft, attack helicopters and naval assets using guns or missiles. The JTAC needs to learn a complex lexicon of terms and a specific phrasing terminology to coordinate fire on the right target at the right time while ensuring the safety of friendly forces.

The challenge was to develop a new immersive training solution that integrates automated voice recognition technology into a virtual reality experience, where the trainee also manipulates tools. The trainee uses tools and equipment such as maps, binoculars, radios and a laser range finder to determine the exact location of targets and to designate the target in the case of precision-guided munitions. The trainee also must memorize complex radio procedure as part of the callfor-fire process.

Military defence speech recognition training equipment.

The Solution

The JTAC trainer that Calian developed uses a virtual reality (VR) device to immerse the trainee in the virtual environment. VR is combined with Calian’s training augmented by speech recognition (TASR), which provides audio cues to the trainee and recognizes the trainee’s spoken responses.

The JTAC trainer guides the trainee through the scenario, including detailed synthetic terrain, an overview of the enemy situation, proximity of friendly forces and civilians, and a summary of the weapons systems available for the mission. With audio and visual briefing, the trainee establishes a clear sense of the context.

Military defence speech recognition training.

With the context established, the trainee moves to their observation position. The trainee scans the target area of interest to determine if the rules of engagement (ROE) allow for weapons to be used on the target. This tests the trainee’s judgement and decision-making on assessing the proximity of friendly forces, the risk of collateral damage and selecting the most appropriate weapons system to achieve the desired effect on the target.

The trainee then communicates with friendly forces and airspace coordination to initiate the process of calling for fire. The calling-for-fire process has specific lexicon and phrasing terminology. TASR understands human speech and provides automated responses to the trainee. This simulates conversation between the trainee and fire coordination personnel, from air traffic control and attack aircraft pilots for air strikes.

The Results

The trainer proved the effectiveness of the TASR technology, allowing trainees to practice voice procedure, terminology and lexicon. The VR solution successfully tested the trainees judgement and decision-making.
With failure states included in the trainer, when trainees make mistakes on voice procedure or errors in judgement they will be notified. This is vital to showing trainees where they made either procedural mistakes or where they made errors in judgement. This real-time feedback gives trainees positive reinforcement when they deliver fire according to procedures and where they make mistakes. This rapid feedback provides an immediate opportunity to halt the scenario and have the instructor guide the trainee where they made mistakes so they can learn from them.

The JTAC trainer approach and technology can be re-purposed for any individual training scenario. No matter the occupation and role that a military customer is preparing for, Calian’s VR solution with TASR voice recognition provides a rich training experience for voice procedure, judgement and decision-making for the application of weapons systems.

Our training and simulation experts are standing by

Speak with an expert today