Benchmarking: the next challenge in Humanoid Robotics Research

Organizers: D. Torricelli, K. Mombaur, N. Tsagarakis

When: November 15th, 2017. Full day, starting at 8:30 AM.
Where: room to be defined


Achieving stable, versatile and efficient walking is one of the current challenges of humanoid research. Biologically inspiration is gaining relevance in the design of bipedal machines, due to the outstanding abilities of humans in coping with out-of-the-lab conditions, in presence of variable and unexpected perturbations. Nevertheless, the process of transferring human abilities to machines is a complex process, which involves questions from scientific and engineering perspectives, such as: “Which are the human properties that should be transferred to the humanoid? How do limitations of the current technology (e.g. actuators, number of DoF, etc..) influence this process? Can the cause-effect relationship between a given solution and the resulting performance be quantified?” Benchmarking may constitute a powerful tool for researchers and developers to answer these questions.

In the framework of five European projects (H2R, Koroibot, Walkman, Biomot and Balance) we recently developed a benchmarking framework for the evaluation of bipedal functions [1]. In past workshops, we shared this preliminary scheme with the Humanoids and related communities [3-8]. In this workshop, we will gather worldwide experts in the field of humanoid locomotion to discuss emerging methods, protocols and devices for the assessment and comparison of walking and postural robotic performance.

This WS is linked with the Frontiers Research Topic: “Assessing bipedal locomotion: towards replicable benchmarks for robotic and robot-assisted locomotion

Call for Posters NOW OPEN!

If you wish to present your work as poster during the workshop, please send a 1-page abstract to diego.torricelli@csic.es. Topics of interest are:

  • Methods and protocols to test humanoid locomotion and/or posture in unstructured environment (e.g., pushes, rough terrains, unstable surfaces, slopes, soft ground)
  • Assessment of physical and cognitive interaction between robot and humans
  • Computational methods for assessing human and human-like locomotion
  • Novel equipment for performing benchmark tests
  • Simulated benchmarking scenarios
  • Databases of human and robotic motion
  • Competition approaches

Deadline for submission: October 23th, 2017
Notification of acceptance: October 31st, 2017

Best contributions will be invited to submit extended papers to the Frontiers Research Topic: “Assessing bipedal locomotion: towards replicable benchmarks for robotic and robot-assisted locomotion (deadline December 15th, 2017)

Invited speakers

  • Katja Mombaur, University of Heidelberg
  • Nikos Tsagarakis, Italian institute of Technology (IIT)
  • Tamim Asfour, Karlsruhe Institute of Technology (KIT)
  • Matthew Studley, University of the West of England
  • Thomas Mergner, Freiburg University
  • Rehinard Gerndt, Ostfalia University
  • Amit Kumar Pandey, SoftBank Robotics
  • Francesco Nori, Italian institute of Technology (IIT) & DeepMind
  • Olivier Stasse, LAARS
  • Diego Torricelli, Spanish National Research Council (CSIC)


[1] D. Torricelli, J. Gonzalez-Vargas, K. Mombaur, N. Tsagarakis, J. C. Moreno, and J. L. Pons, “Benchmarking Bipedal Locomotion. A Unified Scheme for Humanoids, Wearable Robots, and Humans,” IEEE Robot. Autom. Mag., vol. 22, no. 3, pp. 103–115, 2015.

[2] Webpage on benchmarking bipedal locomotion www.benchmarkinglocomotion.org

[3] 2013 IEEE-RAS International Conference on Humanoid Robots, Workshop on “Benchmarking of human-like robotic locomotion”

[4] 2014 International workshop on wearable robots, Session on “Benchmarking, Regulatory and funding aspects of WRs”

[5] 2014 IEEE-RAS International Conference on Humanoid Robots, Workshop on “Benchmarking of bipedal locomotion”

[6] European Robotics Forum 2015, Session on “Replicable robotics research and benchmarking”

[7] International conference of Rehabilitation Robotics (ICORR 2015), Workshop “Benchmarking lower limb wearable robots: towards practical and evidence-based solutions”

[8] IEEE-RAS Humanoids 2015, Workshop “Benchmarking bipedal functions of humanoids robots: towards a unified framework”


Time Talk
8:30 – 10:00 Talks
10:00 – 10:30 Coffee Break
10:30 – 12:30 Talks
12:30 – 13:30 Lunch Break
13:30 – 15:00 Talks
15:00 – 15:30 Coffee Break
15:30 – 17:30 Discussion and round table
17:30 End



Diego Torricelli
Spanish National Research Council (CSIC), Spain

Katja Mombaur
University of Heidelberg, Germany

Nikos Tsagarakis
Italian Institute of Technology, Italy



Diego Torricelli
Neural Rehabilitation Group
Cajal Institute, Spanish National Research Council (CSIC)
Avda. Doctor Arce, 37, 28002 Madrid, Spain