Researchers teaching machines to talk like humans

In the book The Mathematical Theory of Communication, Bell Labs researchers Claude Shannon and Warren Weaver outlined a basic framework for constructing communication networks in 1949. Many of their core ideas laid the groundwork for the astounding advancements in communications networks and computers that have occurred in the subsequent decades.

However, seminal ideas about the semantics and effectiveness of communications were glossed over at the time, in part because computers and networks were too primitive to investigate seriously. A global team of researchers led by Prof. Merouane Debbah’s AI and Telecommunication Systems research group at the Technology Innovation Institute in the UAE has been investigating how these old ideas can lead to better networks.

“Semantic Communications will be at the heart of the next 6G revolution which will focus on connected intelligence. We are inventing here at TII the next machine-learning based communication protocols,” said Debbah.

This could allow wireless networks to scale in capacity and performance while consuming less wireless spectrum. Mehdi Bennis, Professor at the University of Oulu, Head of the Intelligent Connectivity and Networks/Systems Group (ICON) who is working on the project, said, “We cannot continue to scale things up with more bandwidth and more base stations. Rather, we need to leverage data semantics and knowledge to communicate more resiliently and efficiently.” 

Building on an old foundation

Shannon and Weaver’s theory included three levels of communication. The technical level concentrated on encoding information more efficiently or with adequate redundancy in the event of noise or loss. The semantic level considered the precision and efficiency with which meaning was transmitted. The effectiveness level investigated the ability of a communication to influence the receiver’s behavior and conduct.

Existing radio networks have only seriously considered the technical aspects. But Mehdi Bennis believes that achieving the next level of efficiency will also require including semantics and effectiveness as part of engineering design. He said, “With 6G, we need to be thinking about the ability to communicate meaning or intent from source to destination, and then the receiver’s ability to act on it.”

Semantics has traditionally been more of a focus of linguistics and philosophy researchers rather than engineers. Their research pointed to some of the underlying challenges in making sense of the world and communicating it with others but never formalized it in a mathematical framework accessible to communications engineers. 

“Our contribution was to formalize things in a more principled manner,” Bennis said. Specifically, they developed a mathematical framework for semantic communication among two agents (a speaker and listener), in which a speaker extracts concepts from observations in the world and symbolizes them so as to help the listener infer the intent of the speaker. 

Bennis said communication researchers could benefit from incorporating Daniel Kahneman’s model of System 1 and System 2 thinking spelled out in the book, “Thinking fasts and slow.” 

System 1 thinking is about learning the statistical correlation of data, which relates directly to the Shannon-Weaver level of technical effectiveness. It is faster and better suited for making decisions on what has been previously seen. System 2 thinking applies reasoning and abstraction to understand the world at a deeper level and abstracting away irrelevant information through semantic understanding. It is slower and better suited for planning, handling exceptions and being resilient to unforeseen changes in the network. 

Putting semantics into practice

In real life, semantics underpins communications among people. A small child often knows to look for the expression on mom’s face to see if his behaviour is OK. 

Existing communication protocols are hard-coded in terms of rules and vocabulary. An AI-based approach lies in developing emergent and dynamic protocols that arise from the data. This requires developing new vocabulary and rules for communicating and sharing spectrum. 

In the case of communication networks, these techniques might help develop new models for communicating meaning between things like wireless network base stations. For example, two base stations might be able to communicate about the causes of network interference, the importance of different messages, or changes in traffic patterns in a way that helps improve communications for users. 

This vision of the future could replace larger chunks of radio spectrum with shared understanding. If the receiver can infer the semantics of the message, then the transmitter needs to send far less data to get the meaning across. This will lead to better energy efficiency and greater resilience.

The researchers hope to inspire collaboration across various disciplines, including information theory, machine learning, cognitive science, logic, and control theory. Bennis said, “This will require various kinds of multidisciplinary research.”