Detail of the student project

Topic:Scheduling in Human-Robot-Collaboration
Department:Katedra kybernetiky
Supervisor:Jan Kristof Behrens
Announce as:Diplomová práce, Bakalářská práce, Semestrální projekt
Description:The project topic is concerned with a Human-Robot-collaboration in industrial
manufacturing processes. Traditionally, such processes are either fully automated
(worker is just loading and unloading parts) or manually (robot as fixture, parts provider)
which limits the flexibility and performance of the production. Recent works show
that the use of integrated planning and Scheduling methods can improve the performance
of multi-robot systems significantly [3, 1]. We have shown that how humans and
robots can safely work together [2]. The logical next step is to integrate robots and human workers in collaborative tasks. The main challenge is the inherent uncertainty in the human actions and decisions. For example, the human worker might exhibit stochastic behavior in the duration and placement of a pick and place action. Furthermore, the worker might decide among a set of possible manufacturing steps. The system has to adapt and count with these options.

Project goals
- Get acquainted with the robotic setup (simulation and real robots)
- Programming robot motions and skills
- Query the perception system
- Adapt the STAAMS execution engine and planner [1-3] for the setup of the robotic work cell
- Build workspace model (robot mount locations and static structures)
- Incorporate tracked human, e.g., based on HTC Vive or open pose
- Record human action executions and build scheduling representation
- Plan human-robot schedules e.g., using the STAAMS solver
- Design example use cases:
- Building structures from CROW objects (3D printed set of cubes, pegs, screws, and fitting tools)
- Define Human-Robot-Interaction
- what would be the distribution of work between the human and robot?
- Which decisions each one can make? What is controllable and what is observed?
Bibliography:[1] J. K. Behrens, K. Stepanova, and R. Babuska, “Simultaneous task allocation and motion scheduling for complex tasks executed by multiple robots,” presented at the 2020 International Conference on Robotics and Automation (ICRA), Paris, May 2020.

[2] P. Svarny, M. Tesar, J. K. Behrens, and M. Hoffmann, “Safe physical HRI: Toward a unified treatment of speed and separation monitoring together with power and force limiting,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Nov. 2019, pp. 7580–7587. doi: 10.1109/IROS40897.2019.8968463.

[3] J. K. Behrens, R. Lange, and M. Mansouri, “A Constraint Programming Approach to Simultaneous Task Allocation and Motion Scheduling for Industrial Dual-Arm Manipulation Tasks,” in 2019 International Conference on Robotics and Automation (ICRA), May 2019, pp. 8705–8711. doi: 10.1109/ICRA.2019.8794022.

[4] J. K. Behrens, K. Stepanova, R. Lange, and R. Skoviera, “Specifying Dual-Arm Robot Planning Problems Through Natural Language and Demonstration,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2622–2629, Jul. 2019, doi: 10.1109/LRA.2019.2898714.

[5] R. Skoviera, J. K. Behrens, and K. Stepanova, “SurfMan: Generating Smooth End-Effector Trajectories on 3D Object Surfaces for Human-Demonstrated Pattern Sequence,” IEEE Robot. Autom. Lett., pp. 1–8, 2022, doi: 10.1109/LRA.2022.3189178.
Responsible person: Petr Pošík