AI Perspectives: When do we blindly trust AI?

We are on the threshold of the next IT evolution: the use of artificial intelligence

AI not only affects the work at DESY and other research institutions of the Helmholtz Association, but will also have a lasting impact on our society. Together with experts from the natural sciences, technology, philosophy, music and art, we therefore wanted to explore the question: When will we be able to trust AI? After all, how does the AI actually come to a decision? And on what mechanisms is this decision based? What happens when the machine is trained with unnoticed biases? Two event formats were created: one for AI experts and one for the public.

Circle of chairs in a bumper car — first AI Perspectives exchange of experts took place in an unusual place

Two conferences - a joint day. That was the start of the combined "Unconference" of the Helmholtz AI Conference and the Helmholtz Imaging Conference on 14 July 2023 on the grounds of the Kuppel Hamburg. Just a stone's throw away from DESY's side entrance, a spacious event area stretches out and reminded us of something between a circus and a fairground. In the midst of old carousels, our interdisciplinary expert's workshop took place on the track of a bumper car.
For those who might not know: an "unconference" is a conference format where the participants, not the organisers, set the topics they want to talk about. For our project of an interdisciplinary workshop on the topic of Explainable AI (XAI), this was the perfect setting and the special location inspired our discussion.

"Here I hear the subject of fear for the first time. That doesn't exist in my science bubble."

For the initiators of AI Perspectives, which are Katja Kroschewski, Miriam Huckschlag and Klaus Ehret from DESY, Knut Sander and Katharina Kriegel from Helmholtz Imaging and the theatre director Jari Niesner, it was important to bring together different ways of thinking and working in order to approach the topic of XAI in as many different ways as possible, to inspire  and to learn from each other.

What challenges does the creative sector face, when AI now paints pictures and composes songs? How can XAI help create transparency? Two of the workshop participants, the musician Gregory Beller and the artist Anissa Tavara, use AI in their artistic work. And yet there is also fear of the unknown consequences of AI. How does AI change the perception of art and artists, of the creative process and the concept of ownership?

Another attendee, software engineer Lukas Klein from the German Cancer Research Centre in Heidelberg, commented: " Here I hear the subject of fear for the first time. That doesn't exist in my science bubble. Sure, the big debates like the Statement of AI Risk are familiar, but researchers are currently buoyed by the enthusiasm that so many helpful applications are possible, e.g. radiology, weather and climate predictions."

Does an AI need values? Interdisciplinary cooperation as the key to a safer future with AI

It was clear to the workshop participants that the greatest challenge was to make AI programmes explainable, transparent and safe from misuse. This requires the cooperation of various disciplines, such as philosophy, mathematics, neuroscience and psychology. A real danger arises, for example, when AI makes decisions on its own without questioning the consequences. "There are entire research areas that deal with the introduction of values in AI," knows philosopher Timo Freiesleben from the University of Tübingen. There is still a long way to go and many questions remain unanswered.

"How much responsibility am I giving away?"

After a discussion-filled afternoon in summery temperatures, the evening continued with an interdisciplinary panel discussion for the public in the DESY lecture hall. In his welcoming speech, Edgar Weckert, DESY Director in charge of Photon Science, raised the question "How much responsibility do I relinquish? In her opening remarks, DESYan Judith Katzy of the Helmholtz AI Steering Committee pointed to the fantastic machine learning methods "that were simply unthinkable in the past. We can address problems much faster with AI in research." Theatre director Jari Niesner moderated the evening. Joining him on the panel were musician Gregory Beller, philosopher Timo Freiesleben, software expert Lukas Klein, DESY physicist Isabell Melzer-Pellmann and IT specialist Peter Steinbach.

"AI is good at remembering - humans are good at generalising"

Lukas Klein from the German Cancer Research Centre in Heidelberg summarised one of his findings as follows: "AI is strong in applications that require a lot of knowledge and memory. AI doesn't forget anything and can recall everything quickly. So AI is good at remembering - humans are good at generalising. Because if I see an unknown animal with sabre teeth, I know: danger! But we don't yet know exactly how the brain does that." The democratisation of AI was considered a decisive factor by all participants in the discussion, so that as many people as possible can work on explainable, comprehensible AI software and participate in it. We shouldn't have to rely on a company saying, "Yes, it has all been properly trained. Fortunately, open source beats the corporations anyway. The corporations had to find that out for themselves - more people are simply faster," Peter Steinbach summed up and found successful closing words to the applause of the audience.

You can also find the recording of the evening on the DESY YouTube canal.