publications

2024

1. LaserDex: Improvising Spatial Tasks Using Deictic Gestures and Laser Pointing for Human–Robot Collaboration in Construction

   Sungboo Yoon, Moonseo Park, and Changbum R Ahn

   Journal of Computing in Civil Engineering. Editor’s Choice , 2024

   Abs HTML PDF Demo

   

   In the context of the unstructured and fast-changing construction environment, the adaptability of robots to human improvisation is crucial. However, existing construction robots are limited in understanding spatial goals communicated spontaneously in the workplace and require substantial human-user training. Incorporating deictic gestures into human–robot interfaces holds promise for enhancing the intuitive operation of construction robots and for on-site collaboration. However, applying deictic gestures in precision-demanding construction tasks presents challenges because in a large-scale work environment, humans have limited accuracy in pointing at objects at a distance. This study introduces LaserDex, a novel interface for distant spatial tasking based on a global-to-local identification approach, in which the human first guides the robot toward the general area of the task using a deictic gesture and then indicates the precise area by dynamically using a laser pointer. Our user study with 11 participants demonstrated that LaserDex can achieve an intersection over union (IoU) of 0.830 when outlining a rectangular drywall opening (compared with an IoU of 0.514 for the baseline, a handheld controller), and an 11.4-mm distance error of the center of the estimated rectangle compared with the center of the targeted rectangle. The findings of this study underscore the potential of LaserDex to seamlessly combine intuitive human–robot interaction with precise robot operations.

2. A Deictic Gesture-Based Human-Robot Interface for In Situ Task Specification in Construction

   Sungboo Yoon, Jinsik Park, Moonseo Park, and Changbum R Ahn

   2024

   Abs HTML PDF

   

   Despite the potential of robotic systems for automating the construction industry, the role of human operators remains essential for the success of these systems in complex and dynamic environments. However, current human-robot interfaces are often limited to low-level interactions that require constant micromanaging of robot movements. To address this limitation, this study proposes a deictic gesture-based interface that enables high-level task specification for construction robots. To evaluate the user experience and task performance of the proposed interface, we conducted a laboratory experiment with six human subjects who interacted with the robot to make openings in drywall panels. The results show that the proposed interface significantly reduced mental demand and effort levels among the participants compared to the conventional joystick interface. Moreover, task performance using the proposed interface was comparable in accuracy and efficiency to that achieved with the joystick interface. These findings highlight the potential of the proposed deictic gesture-based interface to facilitate intuitive human-robot interaction and precise operation of construction robots, particularly in situations where as-planned building models are not readily available.

3. Evaluating Viewpoint Control Techniques in Virtual Reality Interface for Teleoperating Construction Welding Robots

   Sungboo Yoon, Seungmin Shin, Sanghyun Lee, Moonseo Park, and Changbum R Ahn

   In Proceedings of the 31st International Workshop on Intelligent Computing in Engineering, 2024

   Abs PDF Poster

   

   Effective visual feedback is critical for successful teleoperation of construction robots in remote locations. Recently, virtual reality (VR) has been leveraged to enhance teleoperation interfaces, providing enriched visualization and interaction capabilities. However, despite the critical role of viewpoints in 3D exploration and manipulation, the choice between coupled and decoupled viewpoints—where the user’s camera is tethered or independent from the end-effector’s position—in VR teleoperation interfaces for construction robots remains unclear. In this study, we explore the effect of viewpoint control technique in VR (coupled vs. decoupled viewpoint) on task performance, perceived workload, and perceived usability in the context of teleoperating welding robots in construction. Our comparative study with 10 participants demonstrated that under the coupled condition, participants completed the welding tasks significantly faster and with greater consistency across different locations when compared to the decoupled condition. These findings offer design implications for the integration of viewpoint control techniques in the development of visual interfaces for robotic teleoperation in construction.

4. Interpreting Spatial Instructions for Effective Human-Robot Communication in Construction Environments

   Chaeeun Lee, Sungboo Yoon, Moonseo Park, and Changbum R. Ahn

   Korean Society of Automation and Robotics in Construction in Journal of Construction Automation and Robotics, 2024

   Abs HTML PDF

   When utilizing robots to perform tasks in construction sites, it is important to ensure efficient communication between humans and robots based on spatial instructions and the intuitive decision-making of the operator, in order to adapt flexibly to changes in the environment or alterations in plans. However, it is challenging to accurately conveying the intent of the operator to the robot in complex construction environments. This becomes particularly difficult when dealing with tasks involving objects with curvature or geometric patterns, which increase the complexity of robot trajectory generation. Therefore, this study proposes a method to estimate the robot trajectory from spatial instruction that include deviation, specifically for targets with various forms. It focuses on interpreting spatial instructions in circular, rectangular, and linear shapes commonly used in construction, and the proposed approach involves two steps: classifying the trajectory shapes from spatial instruction and fitting the shapes through regression analysis. The accuracy of this method was evaluated in a scenario where 8 participants collaborated with a robot to cut ceiling-mounted components using spatial instructions provided through a laser pointer. As a result, the proposed method achieved an average Root Mean Squared Error of 2.398mm, compared to 7.274mm for the conventional B-Spline method. These results suggest that the regression analysis-based approach to interpreting spatial instructions has the potential to improve safety and productivity in construction tasks that rely on design plans and layouts.

5. A Taxonomy of Extended Reality for Human-Robot Interaction in Construction Based on a Systematic Literature Review

   Sungboo Yoon, Chaeeun Lee, SangHyun Lee, Moonseo Park, and Changbum R. Ahn

   In Proceedings of the Creative Construction Conference 2024, 2024

   Abs PDF

   

   Extended Reality (XR) applications for Human-Robot Interaction (HRI) in construction, while still in a developmental stage, are rapidly gaining much attention for their ability to enhance communication with robots. However, this area of research often consists of individual explorations, leading to a lack of uniformity in terminology and interaction design techniques. Our study addresses this issue by proposing a comprehensive taxonomy for XR-HRI in construction. We conducted an exhaustive literature review of 51 papers in the construction domain to synthesize the state of the field. Our findings led to the construction of a novel taxonomy comprising three primary design spaces: (1) interface, (2) interaction, and (3) context. Our work contributes significantly to the field by providing a foundational framework that supports researchers and practitioners in the systematic development, standardization, and evaluation of XR-HRI designs in construction.

2023

1. Effects of Spatial Characteristics on the Human–Robot Communication Using Deictic Gesture in Construction

   Sungboo Yoon, YeSeul Kim, Moonseo Park, and Changbum R Ahn

   Journal of Construction Engineering and Management, 2023

   Abs HTML PDF

   

   Construction robots are expected to frequently communicate in situ improvisations with human workers to adapt and change their workflow and methods. One way to achieve this is through deictic gestures that are one of the most effective forms of human–robot interaction (HRI) in delivering spatial information. Nevertheless, the limited coverage of deictic gestures in large-scale environments poses some challenges for both humans and robots in leveraging such techniques for HRI in construction. To identify the feasibility of deictic gestures in the construction domain and find applicable solutions for improving performance, this study aims to extend current knowledge on the performance in communicating positional information using deictic gestures by investigating the effects of spatial characteristics on spatial referencing, focusing on the target configuration, target distance, and relative position of human and robot. We observed that the recognition and estimation of deictic gestures were affected by the target plane, target position, and the target layout and that the robot performance was significantly reduced as the distance between the human and robot increased. The findings of this study demonstrate the challenges in spatial referencing within a large-scale environment and highlight the need for bidirectional communication in HRI.

2022

1. Measuring the Impact of Supply Network Topology on the Material Delivery Robustness in Construction Projects

   Chan Heo, Changbum Ahn, Sungboo Yoon, Minhyeok Jung, and Moonseo Park

   2022

2021

1. Challenges in deictic gesture-based spatial referencing for human-robot interaction in construction

   Sungboo Yoon, Yeseul Kim, Changbum R Ahn, and Moonseo Park

   In ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction, 2021

   Abs HTML PDF Talk

   

   As robots are envisioned to be deployed in construction job sites to work with humans, there is an increasing need for developing intuitive and natural communication between robots and humans. In particular, spatial information exchange is critical to navigating or delegating tasks to collaborative robots. However, such deictic gestures are inherently imprecise and ambiguous. Thus, it is challenging for robots to reason about the exact region of interest, especially in a cluttered large-scale construction environment. To address this limitation, this study evaluates the performance of spatial information exchange through the experiments based on pointing targets on the wall and ceiling, which are the most common workspaces in construction. We observed that the current deictic gesture-based method can estimate the pointed position on the wall and ceiling with a mean distance error of 0.767m, while the error tends to increase by 0.715m in the ceiling and 0.115m in the side panels. Our experimental results indicate that the deictic gesture-based method has some challenges in ceiling and side panel conditions, while the overall panel recognition shows acceptable performance. The findings of this study will help novice construction workers naturally and effectively communicate with robots by delivering spatial information on specific objects or regions in the shared workspace.

2. Multi-objective Optimization Model for Tower Crane Layout Planning in Modular Construction

   Sungboo Yoon, Moonseo Park, Minhyuk Jung, Hosang Hyun, and Suho Ahn

   Korean Journal of Construction Engineering and Management, 2021

   Abs HTML PDF

   

   With an increasing trend toward high-rise modular construction, the simultaneous use of tower cranes at a modular construction site has recently been observed. Tower crane layout planning (TCLP) has a significant effect on cost, duration, safety and productivity of a project. In a modular construction project, particularly, poor decision about the layout of tower cranes is likely to have negative effects like additional employment of cranes and redesign, which will lead to additional costs and possible delays. It is, therefore, crucial to conduct thorough inspection of field conditions, lifting materials, tower crane capacity to make decisions on the layout of tower cranes. However, several challenges exist in planning for a multi-crane construction site in terms of safety and collaboration, which makes planning with experience and intuition complicated. This paper suggests a multi-objective optimization model for selection of the number of tower cranes, their models and locations, which minimizes cost and conflict. The proposed model contributes to the body of knowledge by showing the feasibility of using multi-objective optimization for TCLP decision-making process with consideration of trade-offs between cost and conflict.