ITF supports body-driven technology in psychotherapy

Grace Ho

06 Apr 2004

Can art and technology meet and put to good use in psychotherapy? The answer is a resounding yes. One City University computer scientist, for example, has been successful in applying his technological research to the world of artists, musicians and therapists for helping the emotionally or physically handicapped.

Building on his interactive human-computer technology, the “Body Brush” (translating body motion into real-time 3D painting) and the “Body Baton” (translating body motion to digital music), Professor Horace Ip, Chair Professor of the Department of Computer Science, recently received over HK$1 million from Hong Kong Government’s Innovation and Technology Fund to help turn his research into a new form of therapeutic treatment for the disabled.

“Body Brush” and “Body Baton”, which Professor Ip developed jointly with research artist Mr Hay Young and research student Mr Alex Tang, use advanced visual and audio rendering techniques to express body motion into 3D paintings and music. The system comprises a 3D motion tracking system aided by computer vision frontal-infrared illumination technology. The motion capture system analysis technique is capable of providing information such as 3D body position, motion speed, motion acceleration, body size change, and multiple person capturing. “Body Brush” maps the body gesture and motion path to a set of visual attributes, turning the whole body into a dynamic painting brush. “Body Baton” resorting to the same principles, goes to the audio dimension, turning the human body as a digital musical baton. Raise a hand and the musical pitch increases, for example, while spreading your arms controls the volume.

Professor Ip is working on an innovative approach called “Smart Ambient Therapy” (SAT), in which he will extend his motion tracking and body-driven interface technologies for the special needs of the handicapped. He is developing a state-of-the-art Virtual Reality Therapy (VRT) system – a semi-immersive virtual environment with body-driven audio-visual feedback suitable for use in psychotherapy and rehabilitation. He introduces the term “Smart Ambient Therapy”, or SAT, to describe this innovative concept.

“The SAT system is a new approach that can be integrated into the therapy process. It is unique in that it allows the participants to use their body as the main tool for expression in a safe and authentic environment,” said Professor Ip. In mid-March, the research team invited a group of disabled children from a local social service organization to try out the Body Brush and Body Baton. “The feedback was extremely encouraging: the disabled children became more proactive in expressing and communicating themselves through their body movements. The movements of these children within the Smart Ambient environment surprised even their own teachers.” Professor Ip said. “We will explore further collaboration with professional therapists and rehabilitation centres to evaluate the effectiveness of our SAT prototype. Eventually we hope to set up a Smart Ambient Therapy centre, using the technology in the project.”

“Body Brush” and “Body Baton” are world recognized projects at leading international premier conferences and exhibitions. They have attracted media attention world-wide, including the BBC and the CNN. Details: http://www.cs.cityu.edu.hk/~bodybrush/bb_news.htm

In the latest round of ITF funding announced in late March 2004, three CityU researchers received support:

Principal Investigator (Department)

Project Title

Dr Cheng Suk-han
(Department of Biology and Chemistry)

Screening Agents for Angiogenic Modulating Activities Using Teleosts Embryos