Sungboo Yoon

In the context of the unstructured and fast-changing construction environment, the adaptability of robots to human improvisation is crucial. However, existing construction robots are limited in understanding spatial goals communicated spontaneously in the workplace and require substantial human-user training. Incorporating deictic gestures into human–robot interfaces holds promise for enhancing the intuitive operation of construction robots and for on-site collaboration. However, applying deictic gestures in precision-demanding construction tasks presents challenges because in a large-scale work environment, humans have limited accuracy in pointing at objects at a distance. This study introduces LaserDex, a novel interface for distant spatial tasking based on a global-to-local identification approach, in which the human first guides the robot toward the general area of the task using a deictic gesture and then indicates the precise area by dynamically using a laser pointer. Our user study with 11 participants demonstrated that LaserDex can achieve an intersection over union (IoU) of 0.830 when outlining a rectangular drywall opening (compared with an IoU of 0.514 for the baseline, a handheld controller), and an 11.4-mm distance error of the center of the estimated rectangle compared with the center of the targeted rectangle. The findings of this study underscore the potential of LaserDex to seamlessly combine intuitive human–robot interaction with precise robot operations.

Construction robots are expected to frequently communicate in situ improvisations with human workers to adapt and change their workflow and methods. One way to achieve this is through deictic gestures that are one of the most effective forms of human–robot interaction (HRI) in delivering spatial information. Nevertheless, the limited coverage of deictic gestures in large-scale environments poses some challenges for both humans and robots in leveraging such techniques for HRI in construction. To identify the feasibility of deictic gestures in the construction domain and find applicable solutions for improving performance, this study aims to extend current knowledge on the performance in communicating positional information using deictic gestures by investigating the effects of spatial characteristics on spatial referencing, focusing on the target configuration, target distance, and relative position of human and robot. We observed that the recognition and estimation of deictic gestures were affected by the target plane, target position, and the target layout and that the robot performance was significantly reduced as the distance between the human and robot increased. The findings of this study demonstrate the challenges in spatial referencing within a large-scale environment and highlight the need for bidirectional communication in HRI.