June 21, 2004 —
Mission to Arabic: It's Not Your Father’s Language Lab
To teach soldiers basic Arabic quickly, University of Southern California computer
scientists are developing a system that merges artificial intelligence with computer
game techniques.
The Rapid Tactical Language Training System created by the USC Viterbi School
of Engineering's Center for Research in Technology for Education (CARTE) and partners
test solder students with videogame missions into animated virtual environments
where soldier students have to successfully phrase questions and understand answers
in Arabic to pass.
Special Forces troopers will be using the 80-hour system now being finished at
CARTE's headquarters at the Information Sciences Institute.
"Most adults find it extremely difficult to acquire even a rudimentary knowledge
of a language, particularly in a short time,” said CARTE director W. Lewis Johnson.
"We're trying to build an improved model of instruction, one that can be closely
tailored to both the needs and the abilities of each individual student."
Cadets at the U.S. Military Academy at West Point studying Arabic triede an early
version of the system in October 2003, and offered suggestions. December trials
at Ft. Bragg by enlisted personnel were encouraging and led to further guidance
on making the material accessible.
Johnson leads a six-person CARTE team that is spearheading the effort. The Defense
Advanced Projects Agency (DARPA) and the Office of Naval Research are funding
the work. The Rapid Tactical Language Training System is one of several instructional
programs in a DARPA initiative aimed at developing a heterogenous set of active
learning tools.
Part of the system, the "Mission Skill Builder," resembles an intensive version
of the language laboratory programs that have been in use for generations, in
which students imitate and practice words and phrases pronounced by native speakers.
"While it's similar to drill-and-practice language programs that have been in
use for some time, the Skill Builder incorporates some important innovations,"
Johnson elaborated.
These include:
- Speech recognition technology tailored for language learner speech, that is able
to evaluate learner speech and detect common errors;
- Pedagogical agent technology that provides the learner with tailored feedback
on their performance; and
- A learner model that dynamically keeps track of what aspects of the language
the learner has mastered and what areas the learner is deficient in.
Along with linguistic skills, this section of the program also instructs students
in non-linguistic cultural matters of importance in communication. "People don't
just communicate with words," said Johnson.
"In face-to-face conversation, nonverbal behavior such as gesture, posture, gaze,
head movements and facial expression play an important role in coordinating a
successful exchange," explained ISI research scientist Hannes Hogni Vilhjalmsson,
a specialist in modeling human non-verbal communication. "Wrong interpretation
of nonverbal cues or the wrong nonverbal responses can lead to serious misunderstanding
and escalate hostility. Since these cues can depend on culture, it is important
to include all of these behaviors when teaching conversation skills in a foreign
language. "
Vilhjalmsson says that "by exposing learners to realistic face-to-face situations
and by training them to be culturally sensitive, we prepare them to become effective
social players as well as speakers in the new language."
Points covered in culture training include:
- social skills necessary to build rapport with local people;
- appropriate degrees of politeness to use in different social situations;
- how to disagree with someone without offending;
- and how to respond to offers to hospitality.
Gesture training includes common Arabic gestures that a Westerner might misinterpret
(for example, Arabs may roll their eyes to mean "no") and American gestures (such
as thumbs-up) that an Arab might misinterpret.
The examination or application part of the training system, the "Mission Practice
Environment," is still more innovative. It is designed to give students an unscripted,
unpredictable, and therefore challenging test of their mastery of these elements.
In this segment, students wearing earphones and microphones control a uniformed
figure moving through a Lebanese village, complete with outdoor coffee bar. They
meet animated Arabic speakers, who (thanks to artificial intelligence driven voice
recognition programs) can carry on free-form conversations.
"These AI figures can understand what the students say, if it's said correctly
- or won't, if it isn't. And they will respond appropriately," said Johnson.
In the exercise, after exchanging greetings the student learns the names of locals,
the name of the place, the identity of the local headman and the location of his
house, and must follow these directions through the game interface to get there.
"In typical videogame fashion, the idea is to get to the next level," said Johnson.
"In this game, in order to get to the next level, the learner has to master the
linguistic skills."
The program already has features to adapt it to each individual user, noting
consistent errors or difficulties, which can be targeted for extensive or remedial
practice.
So far, researchers have completed approximately seven hours of the program.
The full program will have about 80 hours of instruction, and introduce perhaps
500 carefully chosen words of the "Levantine" Arabic spoken in Lebanon to learners.
If all goes as planned, the system may be deployed next year.
"We here in the Department of Foreign Languages are very excited about the Tactical
Language Training System and the new capabilities that it can provide to military
language learners, including our cadets," said Colonel Stephen LaRocca of the
Department of Foreign Languages at West Point.
"This system allows learners to rehearse real-world tasks in the most realistic
environment technology can provide," LaRocca continued. "It will be available
wherever and whenever the learner is available, and activities can be repeated
as often as the learner desires. ... It has the potential to greatly expand speaking
opportunities in a meaningful, motivating context."
Working with CARTE on the project are the USC Integrated Media Systems Center;
UCLA's Center for Research on Evaluation, Standards, and Student Testing; and
the Micro Analysis & Design Inc. firm of Boulder, CO.
CARTE is headquartered at the Information Sciences Institute (ISI), which is
part of the USC Viterbi School of Engineering. Besides Johnson and Vilhjalmsson,
the ISI researchers involved include Stacey Marsella, Catherine M. LaBore, Dimitra
Papachristou, Carole Beal, Nicolaus Mote, Shumin Wu, Hartmut Neven, Ulf Hermjakob,
Mei Si, Nadim Daher, and Gladys Saroyan.
- Eric Mankin
