How Tencent is using AI and VR tech to revive traditional art of Chinese opera, with Hua Mulan just a beginning

Hua Mulan, a legendary heroine in ancient China who took her father’s place in conscription for the army by disguising herself as a man, is now known all over the world thanks to The Walt Disney Co’s 1998 animated motion picture Mulan and its 2020 live-action film with the same title.

Chinese people were reacquainted with the character in the 1950s, when the story was performed by well-known traditional opera artist Chang Xiangyu across the country to raise funds for a military plane for the Korean war.

A revamp of Chang’s 1956 show by Chinese video gaming and social media giant Tencent Holdings was recently live-streamed, attracting more than 7 million views. It used artificial intelligence (AI) to improve the quality of the video, by fixing scratches, noise and image discolouration in the early recordings.

Tencent takes control of Chinese gaming studio behind Swords of Legends franchise

A live broadcast of the show on February 2, which also included clips from several other decades-old recordings, is just the beginning of a bigger project Tencent and the Ministry of Culture and Tourism are working on to “use digital technology to renew traditional Chinese opera”.

Besides the Mulan show, the broadcast also showcased Tencent’s 6DoF (6 degrees of freedom) technology, widely used in virtual reality applications to display how an object moves through three-dimensional space.

It decoded the dance movements of Imperial Consort Mei, a fictional concubine of Emperor Xuanzong of the Tang dynasty in the Eighth Century, who was said to dance as beautifully as a wild goose. Her story became a famous show of the Peking opera, the most dominant form of Chinese traditional opera.

“If old opera art wants to catch up with the modern era and have a new audience, it must make use of new media and technology”, said Wang Kui, head of the traditional opera research institute of the Chinese National Academy of Arts. He said that the revamp would not only help promote the art of opera, but also train young performers.

Tencent, which has an edge in visual technology due to its video gaming and video-sharing social media expertise, had to overcome various challenges to complete the revamp. For example, Xu Xiaozhong, tech director of the Tencent Media Lab, pointed out that opera-related data needed to train AI models is “limited and hard to find”.

Xu said that one of the solutions was to “fully explore the data” in different contexts and in different combinations so that it can be analysed more deeply.

Tencent also plans to establish a digital library for creators and enthusiasts to find and share restored opera content, said Shu Zhan, director of the Digital Culture Laboratory under the company’s Sustainable Social Value unit.

More Chinese companies have been making efforts to promote local opera. ByteDance’s Douyin, the Chinese sister app of TikTok that has more than 600 million daily active users, said in 2022 that live-streaming of Chinese operas surpassed 800,000 sessions in the past year, amassing an average 3,200 views, at a time when the pandemic restricted theatre business.

Other cutting-edge technology has been put to use to highlight traditional culture. For example, Alibaba Group Holding’s fintech affiliate Ant Group in 2021 launched two images in the form of non-fungible tokens (NFTs).

These featured two wall paintings from the Mogao Caves in Dunhuang city, northwestern Gansu Province, which has a complex of grottos adorned with Buddhist statues and frescoes. Alibaba owns the South China Morning Post.

NFTs are known as digital collectibles in China, where authorities have declared all cryptocurrency-related transactions illegal.