SBIR-STTR Award

Dexai uses hygienic robot arms to automate food preparation activities, starting with the assembly of prepared ingredients commonly found in buffet or salad lines. Dexai's solution significantly reduces health risks for military personnel and food consumers, without compromising on taste or quality. Dexai’s plug-and-play solution consists of a commodity robot arm adapted to kitchen cleanliness standards, machine vision algorithms to recognize ingredients, and machine-learning based manipulation policies to pick and scoop those ingredients. For each item, the robot, nicknamed Alfred, attaches the required utensil (thus satisfying ServSafe allergy requirements), locates the required ingredient based on its live video feed, picks the ingredient with the allocated utensil, and places an accurate portion into the appropriate food container before continuing to the next task. Under this program, Dexai will adapt its existing single-robot salad assembly station, initially conceived for restaurants, into a touchless fully automated solution. This would enable the conversion of existing self-serve food stations, such as salad bars and hot food lines, into hygienic automated stations operated without human touch. Moreover, Dexai will develop a technical plan to extend its automation solution to other activities in the kitchen, such as operating grills, fryers, and other kitchen equipment, or general prepping of raw ingredients. To do this, Dexai will also develop an intent-based voice and gesture input system, enabling military personnel to place their orders and interact with Alfred without using a touchscreen. Additionally, Dexai will improve the speed and single-pick precision of its manipulation strategies by developing a new machine learning model to supplement existing visual feedback with the robot’s joint torques and velocities. Furthermore, Dexai will develop a technical plan enabling Alfred to perform other major tasks required for food preparation in industrial kitchens operated by the Department of Defense (DoD). Alfred is built on Dexai's proprietary artificial intelligence technology, enabling it to leverage visual feedback and perform complex tasks required for many different kitchen activities. These tasks include: frying, sautéing, stirring, stewing, baking, grilling, chopping, slicing, dicing, blending, mixing, and more. During this Phase 1 effort, Dexai will develop a hands-free interface to command Alfred, test the solution in a real environment at Scott Air Force Base, and develop a technical plan to extend Alfred’s capabilities to other kitchen activities. At the end of this effort, Dexai will deliver a report detailing recommended adaptations of Alfred for Phase 2 work, along with the results of the pilot, no-touch salad bar or hot food line installation of Alfred.

Dexai uses hygienic robot arms to automate food preparation activities, starting with the assembly of prepared ingredients commonly found in buffet or salad lines. Dexai's solution significantly reduces health risks for military personnel and food consumers, without compromising on taste or quality. Dexai’s plug-and-play solution consists of a commodity robot arm adapted to kitchen cleanliness standards, machine vision algorithms to recognize ingredients, and machine-learning based manipulation policies to pick and scoop those ingredients. For each item, the robot, nicknamed Alfred, automates the assembly process end-to-end: Alfred autonomously picks a container, attaches the required utensil (thus satisfying ServSafe allergy requirements), locates the required ingredient based on its live video feed, picks the ingredient with the allocated utensil, and places an accurate portion into the appropriate food container. Alfred then repeats this process for all other ingredients in the order, before finally placing a lid and label on the finished meal. Under this program, Dexai will extend the robot’s supported ingredients to include coverage for all meals of the day and a variety of ingredient types. Specifically, Dexai will develop support for breakfast, lunch, dinner, and snacks (e.g., yogurt-based meals), including food types of different texture (e.g., hummus, chicken, soups), temperature, and delicacy (e.g., herbs, a single chicken breast or salmon fillet). In addition, Dexai will develop new features to maximize the utilization of the robot by enabling a wider range of food preparation activities. Relevant examples include batch portioning, mass-production mode (e.g., for grab-and-go meals), on-the-fly menu roll outs, and plating in container of different shapes and sizes. To do this, Dexai will further develop its artificial intelligence and machine learning models, building support for new types of food ingredients and different holding conditions and containers. Moreover, Dexai will leverage state-of-the-art controls algorithms and computer vision techniques to enable Alfred to perform the required complex kitchen tasks. Furthermore, Dexai will develop a user interface allowing kitchen staff to access all features and easily switch between configurations, menus, and tasks. During this Phase 2 effort, Dexai will retrofit and install 2 robots at the Scott Air Force Base and develop a deployment plan to automate all other military dining facilities. At the end of this effort, Dexai will have deployed 10 robots at several different dining facilities, deliver a report detailing the new supported ingredients and features, and provide a plan for further deployments at other military dining facilities.