It is the first 3D virtual reality learning environment developed with an intuitive motion and gesture-driven interface for severe intellectually disabled students in Hong Kong.

 

The unique pyscho-educational programme, which is called “Interactive Sensory Programme for Affective Learning” (InSPAL), aims at strengthening the SID students’ pre-learning abilities, promoting their self-awareness, decreasing behavioural interferences with learning as well as social interaction, enhancing their communication, and thus promoting their quality of life.

 

It is a combination and integration of interactive media, 3D motion capture, and virtual reality technologies with the principles of art therapy and relevant pedagogical techniques.

 

In collaboration with The Mental Health Association of Hong Kong – Cornwall School (Cornwall School) and funded by the government’s Quality Education Fund, the team developed the InSPAL training protocol based on their continued exposure to the varying levels of cognitive and physical functioning of the students.

 

Eight learning scenarios have been developed within the framework of four learning domains: safety awareness (Safety at Home and Safety at School); cause and effect (Touch to Change and Coloured Balloon Sculpture); balance and coordination (Magic Carpet and Walk across the Water); and sensational experience (Four Seasons and Swimming with Fishes).

 

The psycho-educational training sessions involved eight classes of SID students scheduled in an 18-month period. Before the training, each class went through a preparation programme which allowed the SID students to get accustomed to wearing 3D viewing glasses. Then they went through 32 training sessions to experience four scenarios of two learning domains.

 
The results showed that after going through the InSPAL programme, the SID students were aroused and showed better attention, participation and engagement in different school settings, thus improving the effectiveness of their daily learning activities.

 

“In this project, we successfully overcame major technical and pedagogical challenges such as developing computer algorithms to automatically recognise the body movement and gesture of students who are sitting in wheelchairs or who have restrictions in their natural movements, and also designing pedagogically relevant virtual reality scenarios to assist SID children to learn,” Professor Ip said.

 

He added that the Quality Education Fund had enabled the deployment of the latest virtual reality technologies to enhance the learning experience for severe intellectual disabled children in Hong Kong and to assist them in acquiring certain pre-learning skills, for which Professor Ip’s team was extremely appreciative.

 

“We hope to extend this technology to further benefit students with different special education needs,” he said.

Ms Choi Lui-yin, Principal of Cornwall School, praised Professor Ip and his team for their contributions to enhancing the pre-learning skills of the SID children. “This project not only creates virtual reality learning scenarios for our students, but also reflects the spirit of community harmony. We sincerely thank CityU, Professor Ip and his team for creating such an innovative and unique learning programme for our students.”

 

InSPAL is built upon previous innovations created by the AIMtech Centre such as Smart Ambience Therapy (SAT), a virtual environment for psychotherapy for abused children, and Smart Ambience for Affective Learning (SAMAL), a smart classroom for affective learning. SAT which won a Gold Medal from the Salon International Des Inventions, Geneva, has been a semi-permanent exhibit at the Life Sciences Hall of The Hong Kong Science Museum.