Bin-picking of randomly piled shiny industrial objects using light transport matrix estimation

The use of robots in automated factories requires accurate bin-picking to ensure that objects are correctly identified and selected. In the case of objects with multiple reflections from their surfaces, this is a challenging task. We attempted to address this problem by developing a 3D measurement method based on a Light Transport Matrix (LTM), which can be applied to shiny objects or semi-transparent objects. The study presented herein evaluates the accuracy of the proposed method as well as the method for 3D pose estimation we previously reported, by examining a bin-picking task, which is a well-known robot application in factory automation. There is considerable demand for automated bin-picking systems for general objects. However, in the use of cost-effective measurement systems, some objects such as shiny metallic objects continue to prove problematic in terms of bin-picking because of the difficulty to measure their shapes accurately. Our 3D measurement method uses only a projector-camera system; thus, it is cost-effective, and it does not require any special optical system. It is based on fast LTM sparse estimation. We previously demonstrated that this approach can measure the 3D shape of metallic objects and showed that our pose estimation method is applicable to bin-picking. However, we did not verify its accuracy with the application of 3D robot vision. In this study, we integrate these two methods, and demonstrate that our 3D measurement method, in combination with our pose estimation work, can successfully accomplish bin-picking tasks involving shiny metallic industrial objects. Ultimately, we achieved 100 [%] picking success for 5 scenes including 15 pieces. We concluded that our proposed methods are sufficiently accurate to carry out bin-picking tasks in automated factory environments.