TESSA, the TExt and Sign Support Assistant, has been developed to assist people who are deaf or hard of hearing in transactions at a Post Office.
The TESSA system combines speech recognition technology and state of the art virtual human animation to enable Post Office assistants to communicate with deaf customers. The assistant speaks into a microphone which is then recognised by a computer speech recognition system. The speech is converted to British Sign Language and signed by the virtual human for the customer. English text can also be displayed for those who do not use sign language.
TESSA is being developed as a collaboration between The School of Information Systems at the University of East Anglia (UEA), Televirtual , Consignia (the UK Post Office) and the Royal National Institute for Deaf People. UEA carried out system design, speech, linguistic and interface development, while Televirtual developed the Avatar and animation. Consignia and the RNID are involved in assessment of the system.
The movements of the virtual human are copies of those of a native sign language user. Software specially developed for the project captures the signers hand, mouth and body movements using a variety of electronic sensors. These movements are then stored and used to animate the avatar when required. The Royal National Institute for Deaf People, the UKs largest charity representing deaf people, are closely involved in the project. Their advice concerning the quality of the avatar and animation as well as information concerning the needs of deaf people have helped to shape the systems development. During the summer of 2001, the system was on display at the Science Museum in London and was simultaneously on trial at the Post Office there. You can see some pictures from this event.
The Tessa Virtual Human or 'Avatar'
Signs Being Motion Captured
The project uses commercially available speech-to-text software, which has been optimised to work in a noisy Post Office environment with several different speakers. TESSA enables over 370 of the most frequent Post Office transactions including those involving variable quantities such as money and days of the week to be completed more effectively. The system has been developed around the principle that British Sign Language is simply another language. Thus the technology can also be used to translate into other written languages: French, German, Welsh and Somali are currently implemented. This means the system is particularly useful in Post Offices with large numbers of tourist visitors.
| While the system is
currently targeted at transactions in the UK Post Office, the ease
with which new signing lexicons can be produced means that the system may
be easily adapted to other areas. A recent development of TESSA has been to
move from simple speech recognition to speech "understanding" so that the PO
clerk is not so constrained in what he or she can say to the system.
Investigations are also being undertaken into the possibility of the system
recognising a limited number of signs made by a deaf customer. TESSA is just
part of the EU funded ViSiCAST
project aimed at benefitting deaf citizens by allowing them access to
information and services in sign language. The ViSiCAST partners are also
developing the technology to facilitate the use of sign language on television
and the Internet.
TESSA has recently won both a gold medal and the overall IT award at the prestigious British Computer Society's Information Technology Awards.
TESSA development is funded by Consignia. Further research into TESSA is also being funded by ViSiCAST , a European Union Information Societies Technology (IST) project.