Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos

Arent I, Schmidt FP, Botsch M, Dürr V (2021)
Frontiers in behavioral neuroscience 15: 637806.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
OA 4.05 MB
Abstract / Bemerkung
Motion capture of unrestrained moving animals is a major analytic tool in neuroethology and behavioral physiology. At present, several motion capture methodologies have been developed, all of which have particular limitations regarding experimental application. Whereas marker-based motion capture systems are very robust and easily adjusted to suit different setups, tracked species, or body parts, they cannot be applied in experimental situations where markers obstruct the natural behavior (e.g., when tracking delicate, elastic, and/or sensitive body structures). On the other hand, marker-less motion capture systems typically require setup- and animal-specific adjustments, for example by means of tailored image processing, decision heuristics, and/or machine learning of specific sample data. Among the latter, deep-learning approaches have become very popular because of their applicability to virtually any sample of video data. Nevertheless, concise evaluation of their training requirements has rarely been done, particularly with regard to the transfer of trained networks from one application to another. To address this issue, the present study uses insect locomotion as a showcase example for systematic evaluation of variation and augmentation of the training data. For that, we use artificially generated video sequences with known combinations of observed, real animal postures and randomized body position, orientation, and size. Moreover, we evaluate the generalization ability of networks that have been pre-trained on synthetic videos to video recordings of real walking insects, and estimate the benefit in terms of reduced requirement for manual annotation. We show that tracking performance is affected only little by scaling factors ranging from 0.5 to 1.5. As expected from convolutional networks, the translation of the animal has no effect. On the other hand, we show that sufficient variation of rotation in the training data is essential for performance, and make concise suggestions about how much variation is required. Our results on transfer from synthetic to real videos show that pre-training reduces the amount of necessary manual annotation by about 50%. Copyright © 2021 Arent, Schmidt, Botsch and Durr.
Erscheinungsjahr
2021
Zeitschriftentitel
Frontiers in behavioral neuroscience
Band
15
Art.-Nr.
637806
eISSN
1662-5153
Page URI
https://pub.uni-bielefeld.de/record/2954840

Zitieren

Arent I, Schmidt FP, Botsch M, Dürr V. Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos. Frontiers in behavioral neuroscience. 2021;15: 637806.
Arent, I., Schmidt, F. P., Botsch, M., & Dürr, V. (2021). Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos. Frontiers in behavioral neuroscience, 15, 637806. https://doi.org/10.3389/fnbeh.2021.637806
Arent, I., Schmidt, F. P., Botsch, M., and Dürr, V. (2021). Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos. Frontiers in behavioral neuroscience 15:637806.
Arent, I., et al., 2021. Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos. Frontiers in behavioral neuroscience, 15: 637806.
I. Arent, et al., “Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos”, Frontiers in behavioral neuroscience, vol. 15, 2021, : 637806.
Arent, I., Schmidt, F.P., Botsch, M., Dürr, V.: Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos. Frontiers in behavioral neuroscience. 15, : 637806 (2021).
Arent, Ilja, Schmidt, Florian Paul, Botsch, Mario, and Dürr, Volker. “Marker-Less Motion Capture of Insect Locomotion With Deep Neural Networks Pre-trained on Synthetic Videos”. Frontiers in behavioral neuroscience 15 (2021): 637806.
Alle Dateien verfügbar unter der/den folgenden Lizenz(en):
Creative Commons Namensnennung 4.0 International Public License (CC-BY 4.0):
Volltext(e)
Access Level
OA Open Access
Zuletzt Hochgeladen
2021-05-21T08:56:17Z
MD5 Prüfsumme
1b72f9b084d76099b4e0602cc04dde20

Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®

Quellen

PMID: 33967713
PubMed | Europe PMC

Suchen in

Google Scholar