Demonstrating Large Language Models on Robots
Google DeepMind · 2023
Large Language ModelPolicy Code block_names = detect_objects("blocks") bowl_names = detect_objects("bowls") for bowl_name in bowl_names: if is_empty(bowl_name): empty_bowl = bowl_name break objs_to_stack = [empty_bowl] + block_names stack_objects(objs_to_stack) def is_empty(name): ... Perception APIs Control APIs User Stack the blocks on the empty bowl.def stack_objects(obj_names): n_objs = len(obj_names) for i in range(n_objs -1): obj0 = obj_names[i + 1] obj1 = obj_names[i] pick_place(obj0, obj1) Human Robot Planning & Interaction Grounded Closed-Loop Feedback Can you bring me the drink from the table?Robot Action: "pick up the coke" Robot Action: "bring it to you" Robot Action: "pick up the coke" Success Detector Action was not successful.Success Detector Action was successful.