One is a small screen connected by wires to a handmade circuit board and a cell phone network to help people from getting lost when traveling around the city. A second is a computer program that downloads maps of local stores and creates routes in these stores for people to find things on a shopping list when they go shopping.
Both are devices designed to increase independence of people with developmental disabilities that were programmed, and built over the course of two semesters by two Viterbi School Computer Science student teams for this year’s Senior Design Capstone Projects.
Point A to Point B: help for special needs persons. This early prototype turned into an Android phone app
Parents and education and support personnel who have attended presentations by these students at USC and tried these devices with the targeted population now can’t wait for something like them to be generally available.
"I really think it worked out very well; I wish it could be further developed," said Isis McDonald, Director of Business and Employment Services at AbilityFirst,which trains and employs people with developmental disabilities. "If the engineering students can actually work more and perfect it, it will be fantastic."
This is the first year Professor David Wilczynski’s computer science capstone class (CSCI 477) has partnered with Dr. Barbara Wheeler, who is Associate Director of the USC University Center for Excellence in Developmental Disabilities based at Childrens Hospital Los Angeles, and Associate Professor of Clinical Pediatrics at the Keck School of Medicine. Wheeler worked with the classes in both the Fall and Spring semesters.
She began the year by providing the students with a brief introduction to how an intellectual disability may affect the individual’s ability to process information without assistance. Potential barriers experienced by this population including problems with comprehension, memory, sequencing steps in a complex task, problem-solving, evaluation of people and situations (safe/dangerous), and communicating.
Wheeler went into more detail: “A number of individuals with intellectual disabilities can execute tasks independently when they are routine and repetitive, and if they have had numerous opportunities to practice these rote tasks. If the task has multiple steps, then the individual may have difficulty remembering all of the steps or doing them in the correct order. If the individual deviates from a routine pattern, they may have difficulty back-tracking or reconstructing what they should have done. Intervention (human or technology) can be utilized for either need--there are obvious benefits to technology over human solutions where a high level of practice and repetition is needed to learn a task over an extended period of time, and to give individuals confidence in functioning more autonomously.”
So they could better understand possible needs of this population, Wheeler connected USC students to the AbilityFirst employment training center for people with developmental disabilities, which is located near the USC University Park Campus. "They put things in envelopes,” said Wilczynski, “they put pieces into a package; have machinery where they bend things, they get paid. The students talked to the staff and the clients. They got a pretty good look.”
Shopping center guidance: John Baldo, Jeffrey Chan, Jordan Brown (in back), Phillip Ho, Clem Bradley (in back), Patrick Foley, David Wilczynski, (not shown: Kris Tai)
Wheeler has been working with AbilityFirst for a number of years. The students came, and "we brainstormed," recalled McDonald. "I mentioned we had a client who got lost, not knowing which was the right bus. Also, we talked about many communities needing devices to live more independently, to do routine tasks. The device had to be easy to use, but others, not just our clients, might use them: Alzheimers patients, or brain trauma victims."
With the needs they had observed in mind, the students went off to the drawing board. The choice of one group involving Ryan Romanowski, Alex Lee, Julian Vergel de Dios, Brandon Angelo, Chris Gordon, Tim Zimmer, Joanna Zeta and Alec Patino, grew out of subsequent meetings with AbilityFirst administrators and staff discussing possible solutions for clients who were often getting lost on buses coming to work. The result of weeks of intensive work by this group was a mobile device, also configurable as an application for the Android cell phone, designed to help clients with disabilities to use public transportation to get to and from their jobsite safely and on time. It filled four separate functions: navigation, speech assistance, scheduling, and a ‘panic button.'
Navigation. Brandon Angelo, Tim Zimmer, Julian Vergel de Dios, Ryan Romanowski, Chris Gordon, Alec Patino, Alex Lee, David Wilczynski, (not shown: Joanna Zeta)
The device provides prompts at every juncture where the route changes - literally, at every turn. If the user finds him/herself at an unrecognizable point, the device can provide her or him with corrective actions. Or the user can hit a panic button.
This command brings up a screen button array of faces of caregivers, parents and other recognizable people - faces, not names, since many people with disabilities have limited reading ability. Pushing the button put the user in instant phone contact with the face.
Another the navigation feature is a smart speech synthesizer that people with verbal communication difficulties can use to enlist the help of bystanders or fellow passengers, relying on the location sensing information built in to the device, and website-based knowledge of the location to which the individual was traveling. Pressing this button would politely ask a fellow traveler “Excuse me, could you help me get to Orchard and Adams?”
As a built-in safety and training feature, the geographic tracking feature keeps family members and/or support personnel constantly aware of the position of the traveler. It also keeps a log of times and details of problems encountered, successes and errors experienced by the client, which staff could examine and then use to develop appropriate interventions focused on further training of the individual and/or modifying the steps the individual is required to follow so there is improved success.
Barbara Wheeler
A second Wilczynski group (Phillip Ho, Patrick Foley, John Baldo, Kris Tai, Jeffrey Chan, Clem Bradley and Jordan Brown) chose to focus their design on individuals living in the community. Wheeler connected this group to a 21-year-old woman (Sarah) with Down Syndrome who lives with her family and is still in high school. The group interviewed Sarah, her mother, and her “transition specialist” in high school (who helped her with independent living skills training), and observed her working with the specialist.
They decided to create a navigation/tasking system to facilitate more independent shopping activity by individuals with intellectual disabilities (like Sarah). It featured an automatic, annotated shopping list, guiding a shopper (in this case, Sarah) through a Trader Joe store seeking specific items. The user with assistance from family or support personnel enters the shopping list and the identified store where the shopping will occur.
The app downloads the maps of the identified store (iTrader Joe was a test case), identifies the location of the items, sorts the list by zones, and presents it in a logical navigational pattern for Sarah to follow.
For someone whose reading ability may be limited, the device provides both visual (pictures) and verbal prompts. The user must also confirm success when an item is found by checking it off. If an item cannot be found, the device will move it to the bottom of the list and guide the user to the next item. It will then determine the navigation route so the user can try to find the item at the end of her shopping activity.
The app also keeps a record of successes and “errors” the user experiences so that intermediate human assistance can be planned in the form of training or modifying the task requirements of the device to better match the skill level of the individual.
CS Chair Shanghua Teng, David Wilczynski, Dean Raghu Raghavendra, ARC's Kevin MacDonald, Ability First Director Isis MacDonald
Wilczynski invited engineering leadership and disability experts to attend all student presentations in order to provide feedback in the development process -- in addition to McDonald, Kevin MacDonald, Executive Director of ARC of Southeast Los Angeles, another program providing supported living and employment training for people with developmental disabilities was present.
The result? “My own take was it was spectacular,” said Wilczynski. “Everyone in the disability community who saw the work was excited, quite obvious that our USC engineering students had produced a design that was in concert with needs in the field of disability.”
Wheeler was also impressed. “Both sets of disability community partners commented on the clear interest, intelligence, professionalism, and thoughtfulness of the engineers involved in these Capstone projects. They were impressed by the functionality of the prototypes developed and how the user interface features matched the needs of certain segments of the disability community."
"The senior design projects of VSOE are a valuable opportunity for exposing engineering students to applications of their science and engineering training to benefit individuals with intellectual disabilities," she continued. "Once refined and affordable, these devices have the potential to increase the independence of individuals with intellectual disabilities living and working in the community. USC is becoming known as a university that does excellent research in disability and is
preparing future scientists and leaders to help our community."
Hoping to manufacture: (from left) Julian Vergel de Dios, Pat Foley, Ryan Romanowski
Ryan Romanowski (who graduated in May with a degree in computer science and is now working as an embedded software engineer at Lockheed Martin) still hopes with fellow Capstone students Julian Vergel de Dios and Pat Foley to find a way to produce and market a variation the system he demonstrated to Dean Yannis Yortsos in May.
And this prospect exhilarates AbiltyFirst's McDonald. "It will be a waste if we went this far, and doesn't go further ."
