Drinking is a multi-sensory experience in which taste, sight, sound and context all contribute to the final perception of the product. Building on this premise, Zheng and colleagues have developed XTea, a mixed reality system designed to investigate how an adaptive virtual environment can enhance the experience of drinking bubble tea. The system combines a real drink with an immersive environment called Flavor Drift. Thanks to an AI-based linguistic model, the consumer’s verbal comments are interpreted in real time to adjust the colours, shapes, lighting and sounds of the virtual environment. If the user desires a sweeter or more tart sensation, the system adapts the visual and auditory experience accordingly. The design is based on cross-modal correspondences: sweetness is associated with shades of pink, rounded shapes and fluid movements, whilst sourness evokes yellow colours, angular geometries and faster dynamics. These elements are modulated whilst the drink is being consumed to influence the sensory perception of the beverage. The experiment involved twelve bubble tea enthusiasts.
The results show that the most noticeable visual changes increase engagement and can accentuate the perception of specific taste characteristics. The reverse is also true. Participants can ‘shape’ the experience through their own voices, transforming themselves from mere consumers into co-creators of a dynamic sensory journey. The system’s voice messages also make the moment of consumption more mindful and engaging.
This research opens up new possibilities for the beverage industry: using immersive technologies and artificial intelligence to create personalised experiences that enhance the perception of the product without altering its formulation.
Bibliography: Yuchen Zheng, Zhuo Wang, Hongyue Wang, Don Samitha Elvitigala, and Florian ‘Floyd’ Mueller. 2026. Towards Understanding the Design of Mixed Reality Systems to Enrich the Beverage Experience. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26). Association for Computing Machinery, New York, NY, USA, Article 850, 1–23. https://doi.org/10.1145/3772318.3790615
