Voice assistants: social consequences and legal design

A speaker that whispers comforting words, encourages you, takes your moods into account, can be asked anything and is never at a loss for an answer, reminds you of all appointments and agreements, and makes recommendations in difficult life situations - this is no longer a distant vision. Voice assistants that constantly learn and improve from their communication are on the rise. They promise the appropriate instruction, conversation, support and entertainment in each case. But they are also associated with risks: "How is artificial intelligence changing us? How do voice assistants affect users' mental models, human communication culture, and the forging of relationships? How do we need to design them so that they are conducive to the free development of personality, support us without discrimination, and keep the information entrusted to them confidential? These are the questions," says Prof. Dr. Alexander Roßnagel, a lawyer at the University of Kassel, "that we have been investigating since April 1, 2019, in the four-year interdisciplinary research project IMPACT."

Together with social psychologists from the University of Duisburg-Essen (consortium leader), computer scientists from the University of Bielefeld and ethicists from the Protestant University of Applied Sciences in Nuremberg, the Project Group for Constitutionally Compatible Technology Design (provet) at the Scientific Center for Information Technology Design (ITeG) at the University of Kassel is investigating how voice assistants can be further developed, their effects empirically tested and ethically and legally evaluated. Together, design proposals are being developed to strengthen the benefits and minimize their risks. The project is one of a total of six projects in the Volkswagen Foundation's funding initiative "Artificial Intelligence ─ Its Impact on Tomorrow's Society" and aims to find out in a practical way how society and technology can be changed and shaped in the future with the help of artificial intelligence. The Volkswagen Foundation is supporting the project with 1.5 million euros, around 250,000 euros of which will go to the University of Kassel.

The project takes a lifespan perspective that includes groups such as children and seniors. In a first scenario, it examines children's interaction with talking devices and thus the most sensitive group. Here, the main focus is on relationship-building issues. In the second scenario, it analyzes adults' interaction with a health app that makes health-related suggestions via a conversational interface. The systems' explanatory capacity and the mental models that users make of them are analyzed in particular. The third scenario focuses on communication with a virtual agent that helps seniors plan their daily lives.

"Specific legal issues of the project are the effects of such systems on the realization conditions of fundamental rights and the possibilities to design them in a way that promotes fundamental rights," says Dr. Christian Geminn, managing director of the Project Group for Constitutionally Compatible Technology Design (provet), who plans to address these issues in his habilitation. Examples of legal goals that may be affected by self-learning voice assistants include the free development of personality, humane communication, sensitive handling of human emotions, freedom of choice, especially for children and senior citizens, informational self-determination with its requirements for sufficient transparency for the persons concerned and purpose limitation of the processed information, communicative self-determination and freedom from discrimination.

The research project is relevant to practice not only because it takes a design approach, but also because it is complemented by a citizen science approach. Due to the high social importance of the topic, the public will not only be involved through participation in the empirical studies, but active lay scientists will ask relevant questions and provide answers in the context of two workshops and an empirical study.

Contact:

Prof. Dr. Alexander Roßnagel
Scientific Center for Information Technology Design (ITeG) at the University of Kassel
E-mail: a.rossnagel[at]uni-kassel[dot]de