A creative media expert at City University of Hong Kong (CityU) specialising in human-computer interaction has received an international award from Google Research in recognition of her promising work on the development of voice interaction for text composition.

Dr Can Liu, Assistant Professor in the School of Creative Media (SCM), is the only Hong Kong scholar to receive a Google Faculty Research Award for 2019/20. Her proposal was titled “Voice-based Text Composition with Minimal Visual Aids” and the award comes under research into human-computer interaction.

This year’s Google Research round was highly competitive. Only about 15% of proposals were funded after an extensive review involving 1,100 expert reviewers across Google.

Dr Liu’s research focuses on designing future interfaces based on an empirical understanding of human perception and behaviour. In her award-winning proposal, she points out that voice typing is commonly available on smartphones nowadays with decent speech recognition accuracy. However, the entire user experience of using voice for text composition hits its bottleneck when there is a need to review and correct the transcribed text. When users have to roll back to keyboard, the voice-based experience breaks down and the benefit of using voice gets overkilled. In fact, traditional text editor interfaces are not designed for using voice as the main input modality.

In view of these limitations, Dr Liu explores to develop new voice-based graphical interfaces to support a smooth transition between visual and auditory modalities. It should demand less visual attention than traditional editor interfaces to support mobile scenarios where there is more incentive to use voice. By observing users’ natural speaking behaviours and gaze patterns when composing and editing text, she looks for design implications for new interaction solutions using machine learning and natural language-processing technologies.

Her goal is to redesign a smooth text composition experience that takes advantages of using natural speech, where people are faster at creating text yet not having to stop other actions and stare at a screen.

“The research can shed new light on the further development of the voice interaction paradigm. It will also benefit the visually impaired community by demanding minimal visual attention,” said Dr Liu.

Established in 2005, the Google Faculty Research Awards Program aims to recognise and support world-class research in computer science, engineering and related fields performed at academic institutions around the world. The awards encourage world-class faculty collaboration in pursuing impactful research to the community.