PUB - Publikationen an der Universität Bielefeld

This paper introduces a benchmark frameworkto evaluate the performance of reaching motion genera-tion approaches that learn from demonstrated examples.The system implements ten different performance mea-sures for typical generalization tasks in robotics usingopen source MATLAB software. Systematic comparisonsare based on a default training data set of human mo-tions, which specify the respective ground truth. In techni-cal terms, an evaluated motion generation method needsto compute velocities, given a state provided by the simu-lation system. This however is agnostic to how this is doneby the method or how the methods learns from the pro-vided demonstrations. The framework focuses on robust-ness, which is tested statistically by sampling from a set ofperturbation scenarios. These perturbations interfere withmotion generation and challenge its generalization ability.The benchmark thus helps to identify the strengths andweaknesses of competing approaches, while allowing theuser the opportunity to configure the weightings betweendifferent measures.