ieee icorr2015

Benchmarking lower limb wearable robots: towards practical and evidence-based solutions

Venue and date: Singapore, August 14, 2015 (13:30 – 17:00)
Type: Half day workshop
Keywords: Benchmarking, wearable robots, walking, performance, human-machine interaction

This workshop is supported by the EU projects H2R (, BALANCE ( and Biomot ( and by the recently created network on benchmarking bipedal locomotion (



Wearable robots (WR) are entering an exciting era. An increasing number of solutions are moving out of the lab, approaching the everyday rehabilitation practice and home-based assistive applications. In this context, the quantitative assessment of the technology is crucial for its correct introduction in the market. Nevertheless, a systematic framework to evaluate the WR technology under all its facets is still lacking. In the framework of the three European projects H2R, Biomot and BALANCE, we are developing a scheme for the definition of specific benchmarks designed to lower limb WR, focused on both the global functional perspective and user-machine interaction.

This workshop wants to draw the attention of the WR community on the importance of having a common benchmarking perspective. We believe that reaching an international consensus will be extremely beneficial to boost the process of finding reliable methods to test and compare different rehabilitation systems and identifying robust metrics to measure the Technology Readiness Level (TRL) of the solutions.

This workshop comes after a series of actions directed to involve the international community in the definition of real needs and critical issues related to benchmarking of lower limb functions. These actions included: 1) a web-based survey [1], which received more than 160 responses from robotics and clinical researchers, 2) several sessions and workshops in the WR and humanoids communities [2-5], 3) an international network ([6], [7]) specifically addressed to this topic, to support the clustering of people and ideas.

With this workshop we would like to make a step forward, and bring together different researchers already active in the development of evidence-based evaluation methods and protocols, with the goal of identifying the critical issues in translating these approaches into consented benchmarks. The workshop is organized in 9 talks and a final round table covering different perspectives, including research, industrial, institutional and regulatory aspects.


LIST of Speakers

(* = confirmed)



Diego Torricelli*, Neural Rehabilitation Group, Spanish Research Council, Madrid

Title of the presentation: A unified scheme for the benchmarking of walking functions

Biographical sketch of the Speaker: Dr. Diego Torricelli received the Mechanical Engineering MS and the Biomedical Engineering PhD degrees from the University of Roma TRE in 2004 and 2009 respectively. Since 2010 he is with the Neural Rehabilitation Group of CSIC, Spain. His research line spans the understanding of human motor control principles related to coordination, the development of quantitative measures of neuromotor impairments, and the development of bio-inspired machines for neural rehabilitation. He is now the scientific coordinator of the FP7 project H2R, focused on the development of a bio-inspired humanoid biped.



Jan Veneman*, Rehabilitation Application Area, Tecnalia Research and Innovation

Title of the presentation: Benchmarking of balance performance in perturbed standing and walking

Abstract: This presentation focuses on benchmarking of balance performance, specifically around two subtopics. The first subtopic is related to the BALANCE project, which aims to control lower extremity exoskeletons in such a way that they support human balance-control in standing and walking. The first subtopic can be characterized as: “how to benchmark an exoskeleton, on its ability to support a human in balance performance”. Around this subtopic potential protocols for such benchmarking will be discussed. The second subtopic is related to an initiative called STARS, part of the “COST TD1006 European Network on Robotics for NeuroRehabilitation”, which aims to define metrics that allow robotic assessment of several motor skills in patients. The specific discussions around assessment of human balance performance in standing, using robotic devices, will be summarized. This second topic can be characterized as “how to assess human balance performance, using robotic devices, in the context of neurorehabilitation”.
Finally, the interconnections between both topics and the potential and difficulties for general benchmarking protocols to address wearable robotic performance as well as human performance on balance skills, will be discussed.

Biographical sketch of the Speaker: Dr. Jan Veneman is mechanical engineer and doctor in biomechatronics with University of Twente, Netherlands. Since 2008 working in Tecnalia (former Fatronik) on human-robot interaction, rehabilitation of gait and posture, and robotic orthoses for gait; lines in which he was active since 2002. Since 2013 he is coordinating the EC-FP7 consortium research Project BALANCE, on the development of a robotic exoskeleton with self-balancing control and since 2012 international expert in several ISO and IEC standardization working groups.



Jose Gonzalez*, Neural Rehabilitation Group, Spanish Research Council, Madrid

Title of the presentation: Benchmarking human-like walking during variations in walking speed

Biographical sketch of the Speaker: Dr. Jose Gonzalez is an Electronic Engineer with a PhD in Medical System Engineering from Chiba University, Japan. In 2006 he obtained the Japanese MEXT scholarship and in 2010 obtained the JSPS young research fellowship. Since 2013 he is with the Neural Rehabilitation group of the Cajal institute, CSIC. He was a co-organizer of the Summer School on Neuro-rehabiltiation (SSNR2014) and the International Workshop on Wearable Robotics (WeRob). He is currently working in the FP7 projects BIOMOT and H2R towards improving the human machine interfaces and motor coordination control.



Robert Riener* ETH Zurich

Title of the presentation: Cybathlon: Benchmarking Assistive Devices

Abstract: The Cybathlon is an international championship for racing pilots with disabilities (i.e., parathletes) who are using advanced robotic technologies that provide assistance for daily life activities. The competitions are comprised by six different disciplines that apply the most modern actuated devices such as powered prostheses, wearable exoskeletons, powered wheelchairs, functional electrical stimulation as well as novel brain-computer interfaces. The main goal of the Cybathlon is to provide a platform for developing and benchmarking novel assistive technologies useful for the daily life of persons with motor disabilities. The well-defined admission rules, race tracks, normed obstacles and competition rules will make it possible to compare the performance of assistive devices in a life-relevant setting. The first Cybathlon will take place in a large indoor stadium in Zurich, on 8 October 2016 and will be broadcasted all over the world. Thereafter, the Cybathlon will be held periodically every 2-4 years.

Biographical sketch of the Speaker: Robert Riener is full professor for Sensory-Motor Systems at the Department of Health Sciences and Technology, ETH Zurich, and professor of medicine at the University Hospital Balgrist, University of Zurich. He obtained a MSc in mechanical engineering in 1993 and a PhD in biomedical engineering 1997, both from TU München, Germany. In 2003 he became professor at ETH Zurich and University of Zurich. Riener has publ ished more than 400 peer-reviewed journal and conference articles, 20 books and book chapters and filed 20 patents.


Sin título2

Rajiv Dubey*, Ph.D. Professor and Chair, Department of Mechanical Engineering. Director, Center for Assistive, Rehabilitation & Robotics Technologies (CARRT). University of South Florida

Title of the presentation: Multi-modal assessment tools for lower limb prostheses

Abstract:  The CAREN (Computer Assisted Rehabilitation Environment) is a versatile, multisensory system that includes a real-time motion capture system, a 6-DOF motion platform with a spilt belt instrumented treadmill, a projection screen, a surround sound system and an integrated software package. The CAREN system can be used to measure how individuals walk with a prosthesis and adapt to changes in the environment. By requiring lower limb amputees to face more real life scenarios while collecting objective measures using the CAREN system, issues such as prosthetic fit, alignment, and function related to patient attributes (i.e. age, etiology, amputation level) for various prosthetic configurations and components can be determined. Baseline assessments and re-creation of functional clinical tests will help examine and develop patient-centered outcomes for lower limb prosthesis users. The development of a multi-modal assessment tool, allows for a better understanding of the following: specific time points in patient treatment where prosthetic intervention is most effective; different treatment algorithms to determine the most appropriate prosthetic devices; effects of different prosthetic devices on achievement of maximal functional ability; prevention of secondary adverse consequences of prosthetic device use; and application of specific rehabilitation interventions.

Biographical sketch of the Speaker: Dr. Rajiv Dubey received his Bachelor’s degree from IIT Bombay, and Master’s and Doctoral degrees from Clemson University, all in Mechanical Engineering. Associate Editor of the IEEE Journal on Robotics and Automation for eight years and has been on numerous organizing committees for major international conferences in robotics including ICRA and IROS. Dr. Dubey has received $25 million in research funding as a Principal Investigator from various agencies including NSF, NASA, Department of Defense, Department of Energy, Department of Education, and the private sector. He is a regular reviewer on NSF panels (CISE, RAPD) and served as a Director on the USF Research Foundation Board and on several advisory boards.



Gurvinder Virk*, University of Gävle, and KTH Royal Institute of Technology, Sweden

Title of the presentation: Benchmarking safety for wearable robots

Biographical sketch of the Speaker: Professor of Robotics, University of Gävle, and KTH Royal Institute of Technology, Sweden. Convener of ISO WG on Personal care robots and IEC/ISO JWG Medical electrical equipment and systems using robotic technology. Involved in research and development of wearable robots for lower-body and for full-body assistance.



Nicola Vitiello*, Head of the Wearable Robotics Laboratory, The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy

Title of the presentation: To be defined

Biographical sketch of the Speaker: Prof. Nicola Vitiello received the M.Sc. degree in biomedical engineering (cum laude) from the University of Pisa, Italy, in 2006, and from Scuola Superiore Sant’Anna, Pisa, Italy, in 2007. He also received the Ph.D. degree in biorobotics from the Scuola Superiore Sant’Anna, Pisa, Italy, in 2010. He is currently an Assistant Professor with The BioRobotics Institute, Scuola Superiore Sant’Anna. He is the author or co-author of 38 ISI/Scopus papers and 30 peer-review conference proceedings papers. His main research interests include the development of wearable robotic devices for human motion assistance and rehabilitation and of robotic platforms for neuroscientific investigations.



Jose L. Pons*, Neural Rehabilitation Group, Spanish Research Council, Madrid

Title of the presentation: Coordinating efforts towards benchmarking of wearable robots.

Biographical sketch of the Speaker: Prof. Pons obtained his PhD in Physics, Universidad Complutense Madrid, in 1997. In 1998 he was appointed as Postdoctoral Fellow at the Institute for Industrial Automation of the Spanish Council for Scientific Research, CSIC. In 1999 he was awarded a position as Tenured Scientist, in 2007 a position as Research Scientist and eventually in 2008 a position as Full Professor, all of them at the same institution. He is coordinator of several National, European and International RTD projects in the area of rehabilitation robotics, new actuators and control technologies.



Diego Torricelli -IEEE Member
Neural Rehabilitation Group, Spanish Research Council, Madrid (see speakers section)

Jan Veneman -IEEE Member
Rehabilitation Application Area, Tecnalia Research and Innovation (see speakers section)

 Jose Gonzalez -IEEE Member
Neural Rehabilitation Group, Spanish Research Council, Madrid (see speakers section)


[1] Benchmarking bipedal locomotion – A survey. Still accessible at:
[2] 2014 International workshop on wearable robots (, Session on “Benchmarking, Regulatory and funding aspects of WRs”
[3] European Robotics Forum 2015 (, Session on “Replicable robotics research and benchmarking”
[4] 2013 IEEE-RAS International Conference on Humanoid Robots, Workshop on “Benchmarking of human-like robotic locomotion” ( )
[5] 2014 IEEE-RAS International Conference on Humanoid Robots, Workshop on “Benchmarking of bipedal locomotion” ( )
[6] Mailing list on “benchmarking bipedal locomotion” ( )
[7] Webpage on benchmarking bipedal locomotion