PUB - Publikationen an der Universität Bielefeld

Recently, scientists investigating advanced human-computer communication discovered interactive toys and game playing scenarios as a suitable setting to test their algorithms. The SFB 360 at Bielefeld University is among those projects that pioneered this idea. It aims to develop a robot that - instructed by spontaneous speech and gesture - assembles wooden toy objects. As these parts can be arbitrarily arranged in the construction cell and as its instructor should be free to decide what to assemble, the machine must be equipped with flexible algorithms to appropriately perceive and react to its surroundings. Emerging from the SFB 360, this thesis addresses the problem of assembly recognition from computer vision. It presents a structural approach that combines syntactic and graph-based methods for pattern recognition: simple context free grammars are well suited to model entire classes of mechanical assemblies. Implemented as semantic networks and applying parsing strategies known from discourse theory, they enable the flexible detection of arbitrary assembly structures in image data. Graph-based models are derived from syntactic structures and subsequent graph matching enables the reliable recognition of individual assemblies. Regarding the problem of input of varying complexity, this thesis proposes a task dependent evaluation of results. Given a set of influencing features, the difficulty of a detection task is quantified using a fuzzy classification scheme. The results obtained therefrom underline that using context free grammars for modeling, semantic networks for implementation, discourse parsing strategies for detection, and graph matching for recognition provide a reliable solution for flexible assembly recognition in an unconstrained environment.