PUB - Publikationen an der Universität Bielefeld

The objective of vision-based robotic grasping is to determine the parameters of a gripper, including its position, orientation, and opening width, in order to maximize the probability of successfully grasping an object. However, we identified three issues with the current approaches: they neglect the gripper-object dynamics, lack comprehensibility, and fail to account for reasons behind failed grasps. To address these problems, this thesis proposes a three-step grasping process, composed of the steps: lowering, closing, and lifting. The lowering step identifies possible grasp points where the gripper can be lowered without collision, using an optimization algorithm (Lowering Model). The closing step employs a convolutional neural network (CNN) to predict the gripper-object interaction in the image domain (Closing Model). The lifting step uses a CNN to classify images as liftable or not liftable (Lifting Model). A grasp is considered as successful if the gripper can be lowered without collision, encloses the object and successfully lifts the object. Failed grasps can be explained by analysing the outcome of each of the three steps. Unlike the direct image-to-success mapping, our approach allows users to evaluate predictions in each step based on their own experience, enhancing trust in the model. The three-step model is initially developed on orthographic images of prismatic objects with complete object information. The insights gained from the implementation are used to extend the three-step model to perspective depth images of 3D objects with incomplete object information. This extension includes probabilistic CNNs for depth regression and uncertainty prediction. To overcome the lack of object-gripper dynamics in existing datasets, a custom grasp dataset is created using generated prismatic objects and categories from the ShapeNet dataset. The stability of predicted grasps is validated by comparing them with analytically determined grasp points using a force-closure algorithm on prismatic objects. The predicted grasps from the three-step model on 3D objects are qualitatively compared with two widely used grasp models: GG-CNN and DexNet 2.0. Overall, the three-step model shows a competitive performance, while also successfully predicting the object-gripper dynamics and providing users with insights into failed grasps. Thus a prototype solution was successfully created for addressing the three identified limitations of current approaches.