VRKitchen: AI-Guided Procedural Training and Risk Management in a Virtual Cooking Environment

Abstract

In this paper, we present VR Kitchen, an intelligent virtual reality-based cooking assistant that provides real-time procedural guidance and safety management within a fully immersive simulation. The system integrates a Unity based VR environment with a backend powered by AWS and Amazon Bedrock, leveraging large language models (LLMs) to interpret and manage user actions. Rather than relying on traditional ingame logic to track recipe steps, VR Kitchen delegates all progression control to the AI by sending structured natural language queries directly from within the game.The AI validates the player’s in-game actions against a stored instruction flow, ensuring consistent guidance and strict step by step adherence. In scenarios where a risk management task is triggered, the system temporarily halts recipe progression and prioritizes resolving the risk before continuing. This approach enables flexible, model driven task execution and reduces hardcoded logic on the game side. The system has been fully implemented and tested across multiple scenarios, demonstrating accurate instruction tracking, rapid response times, and robustness in handling unexpected actions.VR Kitchen risk management system is assessed through usability tests with 14 participants. After experiencing the system, most participants reported positive experiences with usability and training effectiveness, although some noted slight VR-related discomfort, such as dizziness or nausea. Overall, VR Kitchen offers a scalable framework for intelligent procedural training in interactive, task oriented virtual environments.

Ähnliche Arbeiten