Category: Human-computer Interface

Posted on

WhereIsIt Project Comes to an End

On August 3, 2026, the final presentation of the WhereIsIt project took place at the FHNW School of Business. The project introduced a voice-based assistant that helps blind and visually impaired people remember and retrieve the locations of everyday objects through simple spoken interaction. Designed with a clear focus on accessibility and ease of use, the prototype works entirely offline, stores all data locally, and avoids cloud services and cameras, ensuring that sensitive personal information remains under the user’s control. The streamlined push-to-talk concept makes the application intuitive and focused on its core task: remembering where things are, while its local architecture demonstrates how AI can support users without compromising privacy. The project was initiated by Prof. Dr. Oliver Bendel of the FHNW School of Business, and the prototype was developed by Damian Huckele as part of his bachelor’s thesis. The project builds on several years of research into Inclusive AI at the FHNW School of Business. In 2024, Oliver Bendel presented his paper “How Can Generative AI Enhance the Well-being of the Blind?” at the AAAI Spring Symposium “Impact of GenAI on Social and Individual Well-being”. Based on extensive tests of the Be My Eyes app and its MLLM-powered Be My AI feature, the paper demonstrated how multimodal AI can significantly increase independence and environmental awareness for blind and visually impaired people, while also discussing the ethical implications of AI-mediated perception. This research laid an important conceptual foundation for subsequent projects. In 2025, the VISUAL project, initiated by Prof. Dr. Oliver Bendel and implemented by Doris Jovic, explored how multimodal AI could make wildlife livestreams accessible through detailed spoken scene descriptions generated from public webcam images. WhereIsIt now extends this line of work by shifting the focus from perception to memory support. Together, these activities illustrate how Inclusive AI can address different everyday challenges through practical, privacy-conscious, and user-centered applications that promote greater independence for blind and visually impaired people.

Posted on

GPTs as Virtual Tutors

In the spring semester of 2024, Prof. Dr. Oliver Bendel will integrate virtual tutors into his courses at the FHNW. These are “custom versions of ChatGPT”, so-called GPTs. Social Robotics Girl is available for the elective modules on social robotics, and Digital Ethics Girl for the compulsory modules “Ethik und Recht” and “Ethics and Law” within the Wirtschaftsinformatik and Business Information Technology degree programmes (FHNW School of Business) and “Recht und Ethik” within Geomatics (FHNW School of Architecture, Civil Engineering and Geomatics). The virtual tutors have the “world knowledge” of GPT-4, but also the specific expertise of the technology philosopher and business information systems scientist from Zurich. A quarter of a century ago, he worked on his doctoral thesis at the University of St. Gallen on pedagogical agents, i.e., chatbots, voice assistants, and early forms of social robots in the learning sector. Together with Stefanie Hauske from the ZHAW, he recently wrote the paper “How Can GenAI Foster Well-being in Self-regulated Learning?”, which was accepted at the AAAI 2024 Spring Symposium “Impact of GenAI on Social and Individual Well-being” at Stanford University and will be presented on site at the end of March. It is not about teaching at universities, but about the self-regulated learning of employees in companies.

Posted on

Mind-controlled Four-legged Robot

“Researchers from the University of Technology Sydney (UTS) have developed biosensor technology that will allow you to operate devices, such as robots and machines, solely through thought-control.” (UTS, 20 March 2023) This was reported by UTS on its website on March 20, 2023. The brain-machine interface was developed by Chin-Teng Lin and Francesca Iacopi (UTS Faculty of Engineering and IT) in collaboration with the Australian Army and Defence Innovation Hub. “The user wears a head-mounted augmented reality lens which displays white flickering squares. By concentrating on a particular square, the brainwaves of the operator are picked up by the biosensor, and a decoder translates the signal into commands.” (UTS, 20 March 2023) According to the website, the technology was demonstrated by the Australian Army, where selected soldiers operated a quadruped robot using the brain-machine interface. “The device allowed hands-free command of the robotic dog with up to 94% accuracy.” (UTS, 20 March 2023) The paper “Noninvasive Sensors for Brain–Machine Interfaces Based on Micropatterned Epitaxial Graphene” can be accessed at pubs.acs.org/doi/10.1021/acsanm.2c05546.