During the 14th International conference on Rehabilitation Robotics 2015 in Singapore, the H2R project together with the Balance project organized a workshop to open the discussion about the challenges on benchmarking lower limb wearable robotics
The first part of the session focused on presentations from researchers that are applying different benchmarking in their projects. Dr. Jan Veneman, Dr. Diego Torricelli, Dr. Jose Gonzalez-Vargas, Prof. Robert Riener (Cybathlon), Prof. Nicola Vitiello, Prof. Rajiv Dubey, Prof. Kyle Reed, and Prof. Gurvinder Virk.
The second part focused more on discussing of the key aspects that are needed to develop a comprehensive benchmark scheme for interaction between the exoskeleton and the human that can be used by the community. Figure 1 shows a diagram that describes a general framework in which the discussion was based.
In this diagram, the first layer is mainly focused on experimental protocols and devices that causes Interaction between the human and the robot, as well as Functional outcomes from the robot and the human working together. In this sense, The Interaction and the Functional outcomes are closely related, since how the human and the robot interact will affect the functional outcomes and vice versa. The Benchmarking Bipedal Locomotion Community has discussed benchmarks focused on the Functional outcomes (for a publication see here). However, benchmarks that focus more on the Interaction hasn’t been discussed in depth yet.
During this workshop, the discussion focused on five specific aspects of the Interaction:
- Physical Impact: It is important to measure the physical contact between the human and the robot to be able to assess if the wearable robot is: Effective in compensating, Non-intrusive, Comfortable, and Safe. During the discussion several variables to measure were discussed: Contact Forces (Normal forces, shear forces), Kinematic compatibility, subjective perception.
- Cognitive Impact: During the interaction between the human and the robot there is also a cognitive interaction that results from the possible difference between the intention of movement and the actual movement. In this case the benchmarks should focus on assessing how fast, reliable and robust is the intention detection, as well as if the system is Non-intrusive, Intuitive and Safe. Several variables can be used for this assessment: Time (e.g. delay of the response), reliability (e.g. accuracy of performance), robustness (e.g. reaction to perturbations) and usability (e.g. training requirements).
- Physiological Impact: It is important to measure what is the physiological impact on the user. This will help assessing if the impact to the user is negative or positive, if the fatigue is reduced or increased. The variables that can be measured for this purpose are: Oxygen uptake, the heart rate and blood pressure, and clinical scales.
- Subjective Impact: It is also important to take into account the perception of the wearable robot from the user. By this it is possible to assess the feelings, the sense of improvement and the intuitiveness of the device. For this purpose the Mental effort and the feelings can be measured using subjective questionnaires such as NASA TLX of the Self Assessment Manikin.
- Safety: This is one of the most important factors that have to be considered for assessing the interaction. This benchmark should reflect if a system is completely safe, if it is in accordance with international standards and if contingencies are well defined.
In this case it is important to measure possible risks and hazards, as well as the probability of failure of a device.
At the end of the workshop, participants agreed that the presented classification to benchmark Interaction was good as a starting point, but that more discussion is still needed to clearly define the benchmark and propose a set of protocols that the community can follow when doing experiments with wearable robots. Also, everyone agreed that this has to be a collaborative effort between all the groups that are interested in developing wearable robotics for rehabilitation.
About the author: