Logo: University of Southern California

Fast Arabic Tutor

June 21, 2004 —

Mission to Arabic: It's Not Your Father’s Language Lab

 To teach soldiers basic Arabic quickly, University of Southern California computer scientists are developing a system that merges artificial intelligence with computer game techniques.

The Rapid Tactical Language Training System created by the USC Viterbi School of Engineering's Center for Research in Technology for Education (CARTE) and partners test solder students with videogame missions into animated virtual environments where soldier students have to successfully phrase questions and understand answers in Arabic to pass.
Special Forces troopers will be using the 80-hour system now being finished at CARTE's headquarters at the Information Sciences Institute.
"Most adults find it extremely difficult to acquire even a rudimentary knowledge of a language, particularly in a short time,” said CARTE director W. Lewis Johnson. "We're trying to build an improved model of instruction, one that can be closely tailored to both the needs and the abilities of each individual student."
Cadets at the U.S. Military Academy at West Point studying Arabic triede an early version of the system in October 2003, and offered suggestions. December trials at Ft. Bragg by enlisted personnel were encouraging and led to further guidance on making the material accessible.
Johnson leads a six-person CARTE team that is spearheading the effort. The Defense Advanced Projects Agency (DARPA) and the Office of Naval Research are funding the work. The Rapid Tactical Language Training System is one of several instructional programs in a DARPA initiative aimed at developing a heterogenous set of active learning tools.
Part of the system, the "Mission Skill Builder," resembles an intensive version of the language laboratory programs that have been in use for generations, in which students imitate and practice words and phrases pronounced by native speakers.
"While it's similar to drill-and-practice language programs that have been in use for some time, the Skill Builder incorporates some important innovations," Johnson elaborated.
These include:
  • Speech recognition technology tailored for language learner speech, that is able to evaluate learner speech and detect common errors;
  • Pedagogical agent technology that provides the learner with tailored feedback on their performance; and
  • A learner model that dynamically keeps track of what aspects of the language the learner has mastered and what areas the learner is deficient in.
Along with linguistic skills, this section of the program also instructs students in non-linguistic cultural matters of importance in communication. "People don't just communicate with words," said Johnson.
"In face-to-face conversation, nonverbal behavior such as gesture, posture, gaze, head movements and facial expression play an important role in coordinating a successful exchange," explained ISI research scientist Hannes Hogni Vilhjalmsson, a specialist in modeling human non-verbal communication. "Wrong interpretation of nonverbal cues or the wrong nonverbal responses can lead to serious misunderstanding and escalate hostility. Since these cues can depend on culture, it is important to include all of these behaviors when teaching conversation skills in a foreign language. "
Vilhjalmsson says that "by exposing learners to realistic face-to-face situations and by training them to be culturally sensitive, we prepare them to become effective social players as well as speakers in the new language."
Points covered in culture training include:
  • social skills necessary to build rapport with local people;
  • appropriate degrees of politeness to use in different social situations;
  • how to disagree with someone without offending;
  • and how to respond to offers to hospitality.
Gesture training includes common Arabic gestures that a Westerner might misinterpret (for example, Arabs may roll their eyes to mean "no") and American gestures (such as thumbs-up) that an Arab might misinterpret.
The examination or application part of the training system, the "Mission Practice Environment," is still more innovative. It is designed to give students an unscripted, unpredictable, and therefore challenging test of their mastery of these elements.
In this segment, students wearing earphones and microphones control a uniformed figure moving through a Lebanese village, complete with outdoor coffee bar. They meet animated Arabic speakers, who (thanks to artificial intelligence driven voice recognition programs) can carry on free-form conversations.
"These AI figures can understand what the students say, if it's said correctly - or won't, if it isn't. And they will respond appropriately," said Johnson.
In the exercise, after exchanging greetings the student learns the names of locals, the name of the place, the identity of the local headman and the location of his house, and must follow these directions through the game interface to get there.
"In typical videogame fashion, the idea is to get to the next level," said Johnson. "In this game, in order to get to the next level, the learner has to master the linguistic skills."
The program already has features to adapt it to each individual user, noting consistent errors or difficulties, which can be targeted for extensive or remedial practice.
So far, researchers have completed approximately seven hours of the program. The full program will have about 80 hours of instruction, and introduce perhaps 500 carefully chosen words of the "Levantine" Arabic spoken in Lebanon to learners. If all goes as planned, the system may be deployed next year.
"We here in the Department of Foreign Languages are very excited about the Tactical Language Training System and the new capabilities that it can provide to military language learners, including our cadets," said Colonel Stephen LaRocca of the Department of Foreign Languages at West Point.
"This system allows learners to rehearse real-world tasks in the most realistic environment technology can provide," LaRocca continued. "It will be available wherever and whenever the learner is available, and activities can be repeated as often as the learner desires. ... It has the potential to greatly expand speaking opportunities in a meaningful, motivating context."
Working with CARTE on the project are the USC Integrated Media Systems Center; UCLA's Center for Research on Evaluation, Standards, and Student Testing; and the Micro Analysis & Design Inc. firm of Boulder, CO.
CARTE is headquartered at the Information Sciences Institute (ISI), which is part of the USC Viterbi School of Engineering. Besides Johnson and Vilhjalmsson, the ISI researchers involved include Stacey Marsella, Catherine M. LaBore, Dimitra Papachristou, Carole Beal, Nicolaus Mote, Shumin Wu, Hartmut Neven, Ulf Hermjakob, Mei Si, Nadim Daher, and Gladys Saroyan.
For more information see  the Tactical Language Project Home Page:
- Eric Mankin