PUB - Publikationen an der Universität Bielefeld

In this thesis, I investigate the possibilities in supporting cognitive robots through commonsense knowledge, large language models or both. For this, I begin with a systematic literature review focused on the current approaches that employ commonsense knowledge for cognitive robots. Due to the found limitations, I create a benchmark that evaluates how well LLMs can handle robot-specific commonsense knowledge by grouping different commonsense aspects into tasks. Additionally, I performed an experiment in which I used LLMs to generate manipulation plans within a complex cognitive architecture. This experiment showed how current LLMs struggle with generating plans for these architectures, despite their available background knowledge. As an extension, I want to create a pipeline, inspired by the RAG architecture, to allow LLMs during the generation progress to access a task-specific world representa- tion to increase its performance. This world representation is either created from scratch or based on already existing models. Lastly, I investigate an alternative to plan generation in creating a probabilistic graphical model capable of parameterizing generalized manipulation plans based on the given instruction and the current environment.