Most research processes involve retrieving, organising and interpreting a massive amount of data. These procedures are important but usually painstaking to conduct manually. Making use of computational methods, such as deep learning, might be a way out. A branch of artificial intelligence, deep learning has trended in the past decade and has been applied in aspects such as academic research.

In January 2020, Professor Jonathan ZHU, Dr Tetsuro KOBAYASHI, and Dr LIU Xiaofan of the Department of Media and Communication of CityU’s College of Liberal Arts and Social Sciences participated in the First International Workshop on Computational Humanities and Social Science at Tohoku University, Sendai, Japan. Let us take the works they presented as examples to see how computational methods help in humanities and social sciences research and lead to insightful findings.

Zhu and his team have been working to develop methods to measure mobile use “sessions”. The team opined that while the time users spend on mobile devices is widely studied, it does not capture the frequency, timing and other temporal patterns of mobile use. Imagine that users A, B and C each use their devices for 120 minutes on a given day, but A uses it mainly in the morning, B in the afternoon and C throughout the day. The typical duration measurement can capture the data as “120 minutes”, but is unable to tell A and B have different timing and C has a high frequency of use. The researchers, therefore, introduced sessions as a complementary measurement of mobile phone use.

A session is constructed based on the activation and deactivation of the device. It excludes all machine-activated tasks and considers only the users’ activities. Through analysing a set of open source mobile phone logs collected from thousands of users by the Device Analyzer Project at University of Cambridge, the team showed how to quantify mobile use sessions to uncover temporal patterns within and across users.

The team believes session-based measures can be applied to research in, and beyond, mobile phone use. It also invites new analytical frameworks or tools, such as deep learning methods, to dig out information embedded in sessions.

Kobayashi and his team used deep learning algorithms to detect politicians’ faces in television news. According to the team, news is the major source of political information for most people. Television news can lower barriers for people to learn about politics as it comes with visual illustrations. It exerts a strong influence on how people understand politics. Therefore, tracking the appearance of politicians on television news is important from the perspectives of political neutrality, comparing public and commercial broadcasting, and so-called presidentialisation (which means political leaders gain more power and public attention than political parties and other institutions).

However, using human effort alone for such tracking is tedious and can undermine precision, especially if the news archive covers more than a decade. Hence, the team proposed the Deep Neural Network-based method (DNN method) to complete the task computationally rather than manually.

To test the DNN method, the team identified 22 key Japanese politicians who represent the main political actors and the opposition during the observation period. Then, the method was run to detect the appearance of these politicians against sample clips from NHK News 7, a Japanese evening news programme broadcast domestically and internationally. As a comparison, an existing tracking method (the VJ method) and a text-based method (which detects the politicians based on the caption given along with the video) were also evaluated with the same sample set.

The performance of their new method was promising. It outperformed the other two methods with a tripled true positive rate and a lower false positive rate, and it succeeded in capturing the change of the ruling party in 2009. The team believes the analysis on public and commercial networks would bring deep insight that helps people understand the effect of visual stimuli on public opinion, and endeavours to improve the new method further.

Next up was research by Liu and his fellow researchers, which investigated the long precepted “fragmentation” problem of the communication research field. They conducted a scientometric study, which analyses the importance of scientists and their works, the construct and trend of research realms and the relationships between different research realms, on articles published from 1970 until the first half of 2019. Similar studies exist, but the team was trying to expand the breadth and depth of data investigated by using deep learning and cluster analysis techniques.

The number of articles analysed was well over 50,000. Though the study worked just on authors’ names and affiliations, article title, abstract, publishing venue, cited references and publication year, it resulted in a sizable amount of information. The result showed that these social sciences realms are probably integrated rather than fragmented in terms of their research topics and methods but fragmented from a scientific collaboration perspective.

The team suggested that a knowledge graph is essential to delineate the ecology of the communication field’s development. Appreciating the diversity of communication studies and embracing more fresh ideas from other disciplines, instead of thinking about how to unify the field, is also a future focus.

The three faculty members showed that the use of computational methods in humanities and social sciences have been making progress, but it has a long way to go. They also met with Dr Hiroki TAKIKAWA of Tohoku University to forge further collaboration, making the trip more rewarding.