In the field of humanoid robots, the main obstacle in identifying common benchmarks is that different methods and metrics are typically employed and reported for specific robotic systems and functional scenarios.
Benchmarking of humanoid locomotion is typically approached on a competition-based level, and it is mostly focused on global functional goals. Complementary to these competition-based approach, we envision a scenario in which system are not compared to each to see which one is better, but to assess the several aspects of multi-facetted performance, allowing a truthful comparison of each feature independently. This approach requires collaboration between different research groups, towards the consolidation of standardized benchmarks and experimental procedures.
About the author: