User view on performance assessments and lab/clinic user communication [comment after ICORR Webinar #3 – Functional assessment]

Questions that came out of this webinar are, how to test, what to test, and how to go about getting clinic/lab and user on the same page.

Webinar#3 of ICORR Consortium 03.06.2024

Transcript via Youtube in footnote 1, and structured summary of Youtube transcript by ChatGPT2

The approach from my view is as follows.

Amputees are different.

We are all different. We use technology differently. We think, act, are, different people. The body part affected for a prosthetic arm is an amputated arm. Now, service providers have a hard time understanding the implications of that. My body image is subject to my own tyrannic and willful decisions. Even if I do not want that, that is what happens. Our bodies are like that. They enforce us. I am the only one to rightfully determine everything there, that attributes meaning to pain, that sustains and bears, that holds out. While others probably take some part in that, have to, want to, will, usually that is not a given. With the amputation, on top of that being a tyrannic command chain, I will be disturbed, troubled, confused, bewildered and traumatized. There are a lot of things that are not as they were. Society acts, reacts, significantly differently. I react, am, different. There are no real deals to be made, no reliable foundation is there, not for many arm amputees. Only after a while, the dust may settle. It remains my tyrannic decision if I accept a totally nonfunctional piece of junk on my stump, and it remains for my body to work out the total pain from skin bruising and blisters in context of junk parts that cost as much as three cars without technical promise of improvement, no potential really – or whether I go on my own tangent and go/become all mechanic, top class functional, where things can still be optimized, as there is so much potential. But: we are different among each other and from non-disabled people, also in ways that exceed imaginations. That said, a surprising thing to do in that situation is have a competition that has arm amputees all try to open a jar that has been closed before by some other folks [etc]. But real dedicated rehabilitation and design discussions usually work differently.

What we can approximate from a testing side is this.

  1. Test 1. How does the arm look. If there is anything that can undergo quality control, it is the look of the prosthesis. The bottom line is, if no one else sees that I wear a prosthetic arm, then test 1 has been passed. No prosthetic arm ever passes 1. You think I, anyone else in need of a prosthetic arm, ever forgets that? Take that, be empathic, let it sit. The degree to which this is earth shattering is not measurable.
  2. Test 2. Is the prosthesis superbly comfortable, robust and functional. Individual focus and trained proper care does that, not 20 folks tested all trying to grab a clothespin. Test 2 is actually not passed by any service provider anywhere on the planet. The first prosthesis may be even made from scrap parts the prosthetist has lying around and that they couldn’t get anyone else to wear, like what I assumed was done in my case. The first prosthesis will cause outrageously uncomfortable skin itches and burns, rashes that take any fun out of anything there. Parts will fall apart left and right, they did on my arm for two years or so. After one realizes this, I suggest every arm amputee to follow this advice here. I built this website on that path, browse it. But it is an arduous path to, as amputee, achieve a pass for test 2, but rest assured, other people won’t do it for you. In an ideal case, other people support you, not necessarily service providers, but colleagues, friends, other professions. Can I do everything, over a day, week, month, year, for work and other life, so my body does not ache and so that I do not feel overly stressed out in the evening? Then the prosthesis solves medical problems of asymmetry caused orthopedic neck and shoulder problems. To achieve that, a medical approach needs to consider work/activity related hazards (link) and work with these. The highly frequent/repetitive, and, the seriously heavy tasks, these require design and build focus. Can I wear the prosthesis for weeks, 14 hours a day, without skin problems? A well set up prosthesis avoids also any skin problems. Read down, you’ll see. After you navigated to pass test 2, the world is basically your oyster.  Then you also managed to turn an insurance employee that didn’t want to replace a moldy gory sweaty liner; I once made it abundantly clear to one of these that they will smell my old liner, if they don’t approve a new one at once.
  3. Test 4. Train lifting, pushing, grasping, handling all things that are dear to you, to oblivion, to the moon and back. Train until cows come home. Do complex and mixed training, do serial drills if necessary. But train, for the rest of your life. Read down, more ideas there.
  4. Test 5. Ah, chill. No more “tests”. Now they can throw anything at you. It is literally a different world. Doing a Cybathlon would be unfair to others. But I once visited one of these Cybathlon demo setups at our Uni, with a friend, over lunch, it was right there in front of us. I showed him how to put a USB stick into that USB port. It was one sweeping motion with the prosthesis. Should have seen the face of the woman trying to use the prosthesis simulator to do same. The organizers there also never passed tests 2 or 3. You cannot hold it against them but they are probably trained differently, so, clueless [link]. With regard to testing, my prosthetist does not need me “tested”. I do things at work some of my colleagues with two hands cannot do.

Or do the 8-Question test. There are eight simple questions to ask if you need to know whether a particular prosthetic arm will work for everyday use in real work situations  [link]. All that is not exactly rocket science or neurosurgery though [link].

Who is who.

User do this. Clinics do that. Manufacturers do something else. Prosthetists be prosthetists. Dermatologists, orthopedic doctor, neurologists, they have questions and problems. Everyone is headed somewhere else. And all that is normal. No one really discusses things with each other. What happens, people talk to each other in various single directions. There are several monologues without much counter traffic. There is no real ball game so to speak.

First of all, the personal and institutional goals of the stakeholders mostly seem at odds, asymmetric, if not conflicting. The user usually is overwhelmed and intimidated, and in no ice-cold-blooded expert modus mind to play a few years of test piloting super fragile toys. There user is not in a mind to spend gagazillions of hard and after the amputation impossibly hard earned money on insanely overpriced betaversions of technology that never really worked when it came out in the sixties such as the “Russian Arm” (link). The clinic or lab representative may have a critical and missionary attitude as they think they have the pinnacle of technology to administer to a possibly ungrateful or impatient patient, and they tend to focus on very selected aspects of prosthetic use, usually far or at least a bit away from the user’s real life. Does he flip the newspaper pages with the “bionic” hand as he said he would. They may want the user to pinch a coin when the user just has maybe like to move 1 household, keep a job but now with just 1 hand, and at the same time juggle all sorts of other responsibilities and tasks, go mow lawn for uncle and auntie, mount shelf for nephew and niece, then off to the roller coaster where s/he isn’t admitted because everyone failed to make the prosthesis look lifelike as they had promised they would – and the world collapses, did you count how many (tearful?) support groups posts there are about getting denied rides on fiveflags or their likes. The manufacturer wants to sell materials for money, and certainly not deal with angry or difficult users, and definitely not spend money on new stuff such as R&D, testing, or patents, or write extensive compliance or medical device registration documentation (here, here). But the user just once lifts something or even moves 20 or 30 boxes and realizes s/he needs an urgently better device, which when the sad realities of the manufacturer is revealed to the user. The manufacturer’s preferred mode may thus even be to produce cheaply elsewhere, anywhere on the planet, not overly do quality testing and let the prosthetist deal with all the fallout. The insurance seems to prefer a silent client, if necessary happy, but if not necessary, then unhappy and silent is probably fine for them, too. The academic researcher usually tests ill understood ideas that other academics also did not understand that well and that they therefore treat as pioneer idea. Such as whether a prosthetic arm needs to feel. The sensory feedback has a funny story though, but it was not meant to tell academics to put it into the devices. They still think that only the most sophisticated electronic devices can feel, and so it was a surprise to them that the Cybathlon 2020 with an extensive sensory feedback test wasn’t won by a myoelectric device user but by a man wearing a cheap 3D-printed body powered arm. There would be much to say was it a dialogue, but that in essence characterizes these people, always looking for new ideas none of which help me carry, sweat and control, work. Being academic to a large part may be understood to keep a visible distance from anything that remotely smells like dirty work, as it embodies a leisure class of its own [link]. The prosthetist typically just may want a modular kit to assemble, for whatever is being built, and not much difficult R&D or design thinking to do. As income / revenue is a real big issue for the prosthetist, devices that are expensive (as they may earn like 30% on hardware or such) and then not used too often (and that therefore do not break that often) are relative if not absolute gold. Myoelectric prostheses tend to fall in that niche. After all, no normal user in their right mind will even think of trying to attach a plough to a tractor with that, or think it is a good idea to rip the weeds out all afternoon with one of these, or shred 4 hours of (manual) salad harvesting, or go hammer some nails in (link). Sure, they may occasionally “use” or maybe “flash” the myoelectric, sure, but not for real work. A body powered arm that gets the user into a high wear and tear and repair use mode, in other words with a device where repairs don’t pay well for the prosthetist, are, for the prosthetist, of course not so interesting. With one of these, one may easily go about some 20-40 kg things over an afternoon and then find that the stock part body powered arm doesn’t hold up after all. Because once one starts with the real action there’s always a next weakest link. Like, I sawed a few wood pieces to fit under my curved cellar stair case for a shoe shelf, drilled a few holes with drills in the drill hammer, weakest link for me were the drill bits, because the concrete is really hard, so I got carbide drills. It is a different game after all but once you open the door for higher loads (weight, sweat, vibration, dirt) there is no end. So that is what prosthetists do not like, affordable mechanical prostheses that are truly robust at least to a degree and that make the user go use and wreck them, like, literally all the time. And that then want more. After all, life is life lived, not spent in prosthetist waiting room waiting for yet another repair. Which also risks to get forgotten, over all that toing and froing with endless frustration and resignation. But prosthetists want things their way.

And that brings us to the users. We? We just want our life back, have a prosthesis that is comfortable and good looking and that always works. Yeah, we know, … – that? That doesn’t “just” exist out of the box. And as clearly no one else cares about the parts that I really need on my prosthesis, then I’m a fixing to look for my own parts (summary here), because these other monologues aren’t all that interesting.

The prosthetist. If they want, they make great sockets. Otherwise, they use commercial components and these are mostly only so so, if at all. Then also the way they deal with users. My prosthetist lied to me. Several times. In my face. I tried three different prosthetists with that question: when I leave you and your services because I am unhappy, and switch to a different prosthetic service provider, will you talk bad about me behind my back? All affirmed, two even laughed. So they have clearly different goals with regard to some things. So we, the arm amputees, usually tell the prosthetist the non-offensive story, the things they want to hear. They may tend to not deal with any other conflicting information. So, all that sensory feedback drama we get these days? Yes, that is a great prosthetist amputee client relationship story, we can learn from that. Prosthetist (with stern look), you do wear the myoelectric prosthesis, do you. Amputee (shivering already), um yes. Prosthetist (keen eye for detail), but the prosthesis really lacks wear and tear. Can’t believe that. Admit it, you don’t really wear it? Amputee (trying to find an excuse that doesn’t blame the prosthetist) well it isn’t comfortable really, is it? Prosthetist (getting upset) That cannot be, we already had three appointments, so what are you saying, that I am a bad prosthetist? Amputee (a bit in trouble now) Um the arm isn’t just that functional, it has a mind on its own and it fails often. Prosthetist (more upset), but we bought the lastest parts, and they cost a fortune! How can you be so ungrateful, saying these things! Amputee (at wit’s end), well most importantly, I cannot feel that well with the prosthesis. When I just have my stump out, I feel everything (adding truthful smile of happiness). Prosthetist, oh well, we cannot fix that though (takes note to pass on to researcher) (researcher later makes questionnaire) (detaches verbal content from actual meaning). — As you have to realize, adding more contraptions to a contraption won’t make it less of a contraption. But sensory feedback wasn’t what we ever really wanted to say there.

As manufacturers usually only sell what they sell, their component offering is of limited use. I want perfect body powered control. It is possible to achieve that. So we built a perfect shoulder anchor from carbon fiber and other plastic (read our paper, or, here). My reach space is near perfect, with rather little compensatory motion, I can work also over the head (videos here), I have only 4 cm of arm stretch from fully closed to open so hardly any compensatory motion — totally unlike the attempts that prosthetists usually give to users under the umbrella term of body powered arms (link). Look if you can’t properly build stuff that is not my problem. I patented my own cable mount, it lasts over 6-9 months and also works about twice the grip power of commercial cable mounts with around the same cable force (here). Commercial cable mounts were not that swell, I these tore up after 4-10 days over the first two years (here). And I invited to cooperate with Brad Veatch for the V2P (here). There are things one does, but only once one knows one is on one’s own. I now have a body powered prosthesis that levels my asymmetry almost perfectly, no neck/back pain even after weeks of real work. My arm needs 1 repair every 8-10 months that I perform at home, with a few USD cost, as modular industry parts design can also be done if one fixes oneself to do that.

Academic researchers, in their own world, need to write scientific papers that have high impact (like, here or here or here). If they can research isolated problems, then they are in heaven. No need to deal with amputees trying to get their life back. Smelly stumps with friction rashes, eww. Hard testing? Nope (here and here and here). I once had a researcher in his lab tell me his accelerometer based control was perfect. I should try it at home. So I grabbed and hovered the paper cup with water in it, wearing his prosthetic device, over his laptop and asked him, sure about that? He went pale and said no, but, he said that in a split second he learned more about real prosthetic arm use than in years of academia before. So also, they now use “healthy” study subjects, because amputees are increasingly hard to come by (here).

I could go on. But there is no multidisciplinary team. This all. This is pure dystopia. No need to watch Bladerunner reruns (link). People wearing toy technology for nuclear power plant prices denigrate and vilify hooks (link). Blessed anyone of these stakeholders who does not rely on any of these other stakeholders to paint their room of dreams and goals. As discussed elsewhere (link): this very clearly is a world of survival of the fittest. We cannot even properly blame anyone, as everyone has their own little motives and constraints. But that is about where it’s at.

So the question that was really asked was how to bring amputee everyday life and reality together with the reality in the clinic or laboratory.

You may do that as Facebook or Google does it. You sell to the user the prospect of a bright future while asking for their privacy. You monitor the hell out of the user. What you may achieve is that privacy is further split, into a prosthetist’s privacy part and the real private aspects of real life. But accept that the real reason for excessive monitoring is probably a lack of understanding and trust.

What you really may want to do instead is the hardest: build good prostheses that are so good the true smile on the face doesn’t require tracking. In order to do that, you need technical delimiters (grip angles, max weights, max torque, etc.) of tasks, and your device just has to be able to deliver/withstand a bit more than these. In doubt overengineer a little.

Use aspects.

My prosthesis is a balancing tool first and a gripping tool second. First of all it has to alleviate or reduce my shoulder-neck-back pains from asymmetry. For that it has to provide weigh wise balance and be very comfortable first, and reduce “compensatory” motion second, but not for all tasks and with perfection – noo. Only for these things where I do dozens or hundreds a day, or for the heavy ones. So I need perfectly symmetric equipment box lifting. I need symmetric arm load for the hedge cutter, 2 hours in over 30 deg C summer sunshine (whaaat). I need symmetry for industrial amounts of keyboard typing (here). Whereas I do not care a bit whether I have compensatory motion taking the trash out (once every 5-6 days), make coffee (3-4 times a day), or, open a curtain. Heavy or repetitive: there, in that subject area, all analysis for builds and compensatory motion needs to go. Not a few clothespins a week. At the end of the day, I feel the cumulative strain without any extensive monitoring or device tracking (link EODF and link pain) — and it is lowest or even absent when I wear my body powered arm, that one with the shoulder anchor, tuned cable mount, own wrist, and so on, and with a good gripper. Strain will be far more when I do not wear the prosthesis, or when I wear a nonfunctional device such as a myoelectric arm that does not even approximate my activity/use levels with robust and reliable support.

In order to then grip with confidence and help with all overuse, training has to be absolutely dominating. For that, zero downtime is key. Zero downtime concept starts by having the prosthesis not cause any skin issues, like, at all. For that, I wear a body powered arm, Ohio Willowwood Alpha gel liner and cotton tubular gauze as liner liner. It has to work under any amount of sweat, and that setup does it. Control needs the feel of an extremely tight road bike, not like the steering of a 50’s US station wagon, so that was what we built, shoulder anch0r and my cable mount, absolutely unlike what is sold elsewhere. Robustness must be there, not the slightest wiggle is tolerated, so my wrist connector does that. We built that, too. So once I have such a device, it goes everywhere all the time. I wear it so much that using it has become second nature. I relied on it even more after shoulder surgery [due to accident, not overuse related] (link). No one else has that? No one else actually understand body powered technology? Well, you want to pay me, I can explain all that to you … again.

The elements.

So, grip. I want a grip angles of the always pre-configured hook, hand, gripper, whatever, that is compliant with everyday objects and their statistically most frequent angle occurrences. Drive car, make coffee, type, all that should be effortless, access angle wise. Hosmer hook, V2P, these work. iLimb, not so much, got to contort a bit much. Read here [link]. Grip surfaces got to be soft and compliant [link]. We are all constrained, in our world access angles, through a range of anatomical and not so anatomical aspects, and a prosthetic arm is a big factor here.

Then, control reliability. I need the arm to work so well that I can trust it with critical items. Only my body powered arm that is – as stated above – most severely tweaked, tuned and modified, does that. If you are happy with 85% or 95% control reliability, not my problem. Sweat is an issue. To give a number, 1% grip error costs 3000 USD for using that per year in one estimate that I made, if one consistently uses the arm. Yeah, I can neoliberal this, too. Read this here, [link, link, link]. Yeah, I know you didn’t really want to read that. Crappy control arms cost the users a lot of money, and if one does not want to spend that, one will not use the crappy control arm. Wear it, maybe, use it, no. Don’t track that though. Build better controls.

Skin needs to stay clear. My skin usually starts to flash itchy motion/sweat/friction rash after maybe 1-2 hours of prosthetic use. Myoelectric was a bit of a real nightmare [link]. So I use body powered and there, we fixed that for good [link]. I occasionally get fan letters from other users for that blog post.

Mechanical reliability. Wrist, screws [link, link]. Don’t get one if you won’t use it anyway.

Wasn’t that hard, now, was it.

The drop outs.

Yeah, I tried the myoelectric, I have an iLimb even. Jeez. Here. And of course, Britt H. Young. Here.

What are user evaluations.

There are a few archetypical or standard users.

We are, however, mentally impenetrable with regard to the predicament. You won’t get to the bottom anywhere. Never mind.

  1. OMG I have a new myoelectric 3D printed arm! 4 photos on Instagram. After that, no more use evidence. Life is fun!
  2. I feel not authentic with a prosthetic arm. But I have the most serious overuse and asymmetry problems. I have cervical disc hernia, rotator cuff damage, and wrist arthritis along with carpal tunnel syndrome. Life is hard.
  3. I feel authentic only with my myoelectric arm. Despite that, I have the most serious overuse and asymmetry problems. I have cervical disc hernia, rotator cuff damage, and wrist arthritis along with carpal tunnel syndrome. Life is hard. [The device does not really work. No myoelectric REALLY helps there, I tried, I do overuse type stuff].
  4. I have a prosthesis but it does not help me. I wear it because then people talk about it. In a restaurant I get the waiter to cut the food for me. Just because I can. Life is fun.
  5. I have a body powered arm but it breaks all the time. My prosthetist does not seem to help. Life is hard.
  6. I have a myoelectric arm, but it breaks all the time. My prosthetist does not seem to help. Life is hard.
  7. I have a myoelectric arm but d0 not use it. It failed me once when I was distressed and probably sweated. I do n0t feel safe using it any more. Life is better without it on.
  8. I look at the device and it looks like shit. They all look like shit.
  9. Britt H. Young. Here.

Failure? Hard testing!

Usually there are non-negotiable fail situations. Once these occur, the user may not go back to really using the device, or have (possibly unconscious) “penalty use rules” in place.

If your user has such tasks, go for these if you are a provider.

  • Garden and house work. Pick the harder ones and start with exhaustive drill exercises. I taught myself laundry work and kitchen stuff by declaring these as “hook zones” from day 1. Got to have a bit of discipline. I managed to pick up stuff from the ground when standing in a single sweep motion, using my prosthesis, after maybe 2 weeks of continuous use, and if I can do such, anyone can.
  • Freestyle hard and difficult manual work. Repairing my bicycles forced me to go with the flow, to use my brain on autopilot. Changing watch band or battery, puh. Do that to anyone, expose them, let them figure it out. Only dead fish swim with the stream.
  • Move a household or furniture. Go for sports events that are absolutely exhausting. Use a real power tool for a real amount of time. When we rode the bicycles up to Stelvio pass from Prado. But, reach for some stars. Go past comfort zone. Then look back and analyze. After that, you learned something.
  • Once you master hard and difficult things, the other stuff follows. Like, once I bolted up a shelf, making coffee felt like, nothing really.
  • It is the big things that matter. Not (just) the small things. These are OK. But the big things, they are the ones that put a grin on your face for years or decades even. With a potentially devastating and depressing disability, you want to go for the big grins that last for years or decades even. Unless, of course, you are not very empathetic and heartless and do not care. Well, read above again, with that in mind now. Isn’t it a different read now? So much for circular hermeneutics.

Ice cold.

The number of absolutely ice cold blooded test pilots that go invent their own prototypes are an absolute minority. That are far from impulsive.

For the rest of the users, you need 1000% empathy for a life situation that no one else can mentally penetrate. Try to get them to cool TF down. Also, try to build a prosthesis for them that makes them smile in a very sustained way when they wear it.

But then, seeing as if the actual goals are so different anyway (see above), who cares.

Footnotes

  1. I guess we’re then in so to one and live great third dor webinar welcome to the whole world dear colleagues and Friends ladies and gentlemen good afternoon good evening or good morning depending on where you are today we have um we have guests from the the further Western section of North America I guess we’re then in so to one and live I guess I need to silence my own live stream good um going through Friday to new BRS week and to Europe so that’s great right my name is cloudo I’ll be moderating actually doing as little as possible because um our guest need space today this is a webinar about functional assessment and I am particularly proud of having collected four of the best experts in the world in this sector and um let me start with a personal note um I wanted this badly because it’s well known that functional assessment is let me say it it’s not among the top priorities of Engineers I mean and and I am one so I know what I’m talking about my students I always need to uh you know push them to get out of the lab get into the clinics and and try stuff on you know with therapists and patients and there’s always some kind of resistance to that So today we’re going to discover how we know how to make our devices and systems acceptable to people and how to actually check that they are worth anything because if they don’t translate then the value is really little so I’m going to show a couple slides just to give you an overview of what today’s what today’s webinar is going to be about so our four speakers are Professor K fosl Professor lelot hamson uh Professor Wendy Hill and Dr jacn Hebert um in turn from the Netherlands Sweden and Canada Canada wise um today’s agenda is roughly organized as follows so I’ll be um um shutting up in two minutes and and then we have four talks minutes each in turn by Cory liselot Wendy and jacqulyn and then we’ll have well to minutes of free panel discussion with questions by well the speakers to one another myself and we encourage everyone who’s watching us through the YouTube live to F us questions write them in the chat of YouTube and I will rely those questions to our speakers now about our speak ERS um Cory funersl uh from croning and Netherlands is a professor and consultant for the rehabilitation medicine at the University Medical Center of kingan her professional activities comprise patient care teaching and research me show you our faces once again why not um so her research coris research focuses mainly on upper Li amputations and Prosthetics Innovative Technologies such as serious games and VR are integrated in this research I can personally say that C’s cor and Raul bonger um work on translation of skills from games to the Practical life are particularly interesting to me and we started teaming up about years ago so is the author of over internationally peer reviewed Publications and she has been giving over International presentation our second speaker Professor lelot hamon is a full professor in occupational therapy Affiliated to the orro university located in Sweden her professional background is mainly within Children’s upper L Prosthetics in which area she teaches and does research lisot hamson has developed occupational therapy training protocols and instruments to measure and follow up the progression during therapy and evaluate the outcome one of these measures which is one of the best known in the world and I teach about that is called acnc the assessment of capacity of my electric control that protocol has received interest from both clinicians and prosthesis developers so thank you Ki thank you lot for being with us Professor Wendy Hill is the research occupational therapist working with the Atlantic clinic for upper Li Prosthetics in frederickton New Brunswick that’s located in the Southeastern section of Canada specializing in aine my electric fittings and providing service to the clinic I mean the providing service to clients of all ages from all over Atlantic Canada the LM Clinic is housed within the Institute of biomedical engineering at the University of New Brunswick where Wendy is involved in many research projects in the areas of upply motion analysis outcome measures and prosthetic hand control Wendy moreover is a an honorary research associate with the faculty of kinesiology at unb University of New Brunswick again she is the North American instructor for the ACMC assessment and I proud myself of having been your pupil is at Mech I think I attended your work workshop and I I still got my ACMC raer certificate uh in front of me on the on the wall um um yeah and and she’s been co-chair of the my electric control Symposium since I urge all people who are listening to consider attending the conference which will take place this year in August in frederickton new brunes so last but so Wendy thank you very much for being with us last but absolutely not least professor jacn Hebert um is a professor at the University of Alberta Canada again and a physiatrist at the adult amputation rehabilitation program at the Glenrose Rehabilitation Hospital in Edmonton in Alberta Canada as a clinical investigator her research goal is to facilitate the implementation of Rehabilitation technology to maximize function and quality of life after amputation and to create a sound evidence base to assist with clinical decision making Dr Hebert was instrumental in bringing the first targeted reation surgeries for Upper Limb amputation in Canada and established a research program for advanced motor control and sensory feedback systems for prosthetic devices at the blink the bionic limbs for improved natural control lab I am a big fan of the gamma setup and and jacn is among the founders of the of this very well- instrumented setup for checking how well our patients are actually performing so I just stopped sharing my screen and I believe without much further Ado we might start with the first talk by Cori Cory would you please unmute yourself and share your screen thank you very much the floor is yours thank you claudo uh just to check can you see my screen no I don’t think so we see your screen the wrong one if you now go to presentation mode yes that’s it perfect that’s it yeah thank you uh thanks Claudia for the introduction so to answer the questions raised by claudo just a few minutes ago we need to look at outcome measures and I’m going to try to answer um going to try to pose questions and provide their answers on outcome measures and upper liim differences first question is why do we need outcome measures well we need outcome measures because you want to measure your patient progress over time or you want to uh measure treatment Effectiveness or you have developed a new device and you want to evaluate it or you want to compare your new device with an older one and so there are more reasons why we need outcome measures uh for example also we need data to convince policy makers and health insurance companies that we do the right things and that they are going to reimburse our treat treatments second question is what kind of instruments are available for Upper Limb differences there are a lot of instruments available as you can see here this is only a selection these are more generic instruments and also instruments more disorder specific for appal liim differences the third question is how can we sort these outcome measures there are a few way to do a few ways to do this we can sort them in questionnaires or in observational measurements we can sort them in pediatric uh measurements or adult measurements and some of the instruments are applicable for uh children and adults at the same time but mostly we look at the international classification of functioning disability and health the ICF or in case of children or youth we look at the ICF Cy at the top you see the aalim differences and we can measure at three level levels body functions uh for example pain or uh joint range of motion or muscle strength activities so what does a person do for example with their aalim prosthesis and participation and then we look at participation in uh Society uh for example uh work or uh leisure activities or uh social relation relationships and these three levels function activities and participation can be influenced by environmental factors for example the country you live in and all the rules this country has or uh personal factors like age or uh gender or um amputation level or education what we also have to look at if we’re talking about activities is capacity and performance capacity is what a person can do and that is mostly what we measure in the lab because we ask them uh we ask the participants uh to do things that they would normally not do in the daily life and we can also measure performance um that is what a person does do in daily life for example in their homes or at uh work so there are a lot of things to consider if we sort um the measurement instruments into function activities and participation you you can see that we have quite some instruments in each uh level that we can measure however we can also see that no single outcome measure reflects all ICF domains the fourth question is about how many disorder specific outcome measures are available for Upper Limb differences so these are the measurement instruments that are spefic specifically made for the Target population upper Lim differences and then we see that we have only a few in the body function domain also only a few in the participation domain and a bit more in the activities uh domain what determines the quality of an outcome measure the fifth question I would like to address then we look at the picture of the cosmin research group and they talk about validity and N Nar atal said validity is the ability of an instrument to reflect the conceptual basis that is intended to be measured we look at reliability which is the ability of an instrument to produce consistent results we look at responsiveness the ability to detect clinical changes and interpretability if you would like to have a bit more uh definitions of these ter terms and we look at reliability here inter rator reliability intrator reliability measuring between rers and within the same rater test retest reliability so um doing the the same test uh two or more times with a few weeks in between the internal consistency the the degree to which all of the items are related to each other if we look at validity then mostly Criterion validity is uh is measured as um it can be related to a gold standard or content validity the degree to which the items or questions completely assess the domain or construct validity that is the degree to which the outcome measure adequately measures the underlying construct responsiveness as I said is the ability to measure a real change and then we look at a minimal detectable change which is a statistical figure and it um says something about whether we measure a um a measurement error or a real statistical change but more important for us as clinicians is the minimal clinically important difference which measures a real important change for a patient and finally we can say something about interpretability and that is the capacity of an instrument to assess Improvement sealing effects or deterioration floor effects in a condition like upper Lim differences the six question I would like to address is how is the quality of disorder specific outcome measures for upper Lim differences to answer this question I composed a table as you can see on the horizontal um I put some of the most used uh measurement instruments and on the vertical axis um I po uh put some psychometric properties if you look at this table then you probably will see that the refined Clos pin relocation tests which is a frequently used test has hardly been investigated on its psychometric properties and if you look on the horizontal lines then you can see that inter rate reliability test retest reliability and construct validity have been investigated quite intensely but and that you can see the the red arrows that are four or more white boxes so not investigated for all the other psychometric properties so we can concl conclude that a lot of work still has to be done the seventh question is about what determines the quality or the applicability of an outcome measure further an outcome measure should be relevant not only to the professional who is measuring but also to the Target population in our case Apper Lim prestigious wearers or Apper liim different patients with Apper Lim differences it should not take too much time an outcome measurement instrument should not be too costly because otherwise a lot of people will not buy it it should preferably not have a trained raater because um that also prohibits a lot of people to use the outcome measurement the rating Sy system should be simple and and transparent and it should also be a bit of fun to um to measure because if questionaires are too long people will not fill it in anymore or um they skip certain questions and if observational measures are too tedious people will also not be willing to do it a second or a third time what are further quality considerations well let’s have a look at the Southampton hand assessment procedure which is also qu used quite frequently it’s a self-timed test and it consists of abstract and ADL tasks as you can see here on the picture we did a um a bit of research in the shap we measured on four consecutive days and participants did the shap two times per day and you can see here the figure what we learned from it is that there’s a strong learning effect in the shap so if you do the shap on the first day you are far slower than if you do it on the fourth day for the eighth time what we also saw there that there were warm up increments so the second time the participants performed a ship on a certain day they were faster than when they did it the first day and what we also discovered is that the Sheep does not have a transparent scoring system you have you to feed in your data in a in a website and you get a score but you don’t have a clue what the scoring system is so that’s why we proposed uh a more transparent scoring system so all these kind of things you have to take into account if you want to use outcome measures the ninth question how can usage of outcome measures be improved I think we need a core set of outcome measures and that we need a core set for research which might differ from a core set of for clinical applicability we need to standardize uh our outcome measures and we need to uh do more research uh in the psychometric properties of the score set outcome measures these measures then can be used worldwide they should easy uh be easy accessible and of course it should be cheap simply in the rating uh not a trained user should be quick and it should be Rel as I argued before so my final considerations and my final question what are considerations for the future I would suggest that we develop a core set for all ICF levels in collaboration with IPO uh sigal and that is the international Society for Prosthetics and Orthotics the special interest group for upper limbs if we uh determine the score set then further psychometric testing of these outcome measures should be done for questionnaires we might consider to shift towards a cat computer adap adaptive testing because that would prevent participants from getting bored and we should consider that testing should have a a fun aspect so maybe we should integrate measurement outcomes in serious games virtual reality or web- based instrument ments and finally if we develop this score set we should should do this in co-creation with service users pris users because only then the outcome measures will be relevant so I addressed questions for you and I also try to answer these I can imagine that you have any further questions but uh I would suggest to put your questions in the chat and that we try to answer these uh at the end of all the presentations that’s perfect thank you very much goofi yes your up next I guess I will unshare my screen yeah no uh stop sh yeah mhone thank you claudo for your introduction and for inviting me to give this presentation thank you codyy for the for the um background so I am about to talk about the use of functional outcomes and um see here if I can change my slide So I myself I’m from Sweden and Sweden is a long country I am situated in Oru in the southern part more populated part of Sweden and so this is my my um context for my work see here so um my main professional uh life has been within treatment for children with reduction deficiency present at birth and we follow a procedure where we fit a prosthesis at six months of age where the child is almost newly born and we fit them with a passive hand and this is to adjust for the limb length difference but also to um prepare the child for for a future proces you usage and the aim at this stage is to learn the child to wear and use the assistive device excuse me and when the child is about three years of age the child is fitted with a mycc hand with a dual site electrod system and now the aim is to learn to operate and use the assistive device this is often done in together with the family and uh you can see here the child is trying to operate the hand to move the hand and has not yet found the key how to do it so this is the first time this child is fitted with a mycc hand and we fit the state-ofthe-art Pediatric mctic hands that is they contain two types of grip the power grip the Precision grip and the grip has three movements either closing the hand holding that is nonactive movement or Nona non-activity in the hand and opening the prosthetic hand to let go of the objects so to learn to use the prosthesis the occupational therapy teaches how to use prosthesis by teaching uh a habit of wearing proceses controlling the device and performance of daily tasks because in order to use the procedures and daily tasks the person has to develop these skills to control the hand and also to wear the hand and this measurements of this can help us to assess the person’s need for f further training so we have a need to evaluate the outcome of treat M but how do we measure prosthetic control so to me prosthetic control based on my clinical experience these six areas are part of the prosthetic control and they they um uh influence the control so the adjustment to the weight of the hand the ability to adjust the grip Force control the arm when with the arm in different positions to do iterative hand movements like within hand movements coordination between hands and to have no need for visual feedback these are the uh major six aspects of of um being able to control the hand this is an example of um a child who can operate the hand control it without support for the arm uh he can he can operate the hand in different positions on the floor or up up standing upright he can control the grip with the timing so he can let go of the object in just the right moment to hit the ball he can do this by coordinating both hands and he doesn’t have to look at the object he doesn’t have to look at the hand when he lets go of it so uh these are examples from my clinic and when I wanted to measure the outcome of the treatment I had different options as Cy already mentioned there are a number of outcome measures in this field there was the box and blocks tests where you measure the number of blocks you can move in one minute we have the timed standardized So-Cal functional tests uh where you can see a patient grasp release objects of different shapes different texture and these are based on normal hand functions for example ranul Lo Amigos test jebson Taylor test of hand function and more previously the Shar Southampton hand assessment procedure but we already know the technical limitations of the prosthetic hand I know that the hand can only do power grip and precision grip so most objects or tasks can be handled with a standard myoh hand in these tests but they they ask for different grip types uh so the I didn’t find these tests um appropriate for my my uh need and the table top assessments of this uh capacity for grasper release they were not sufficient to guide the treatment so there was no measure of quality of control how the quality to find to measure the development of the control from for example a small child learning to open and close the hand and to hold the object in the hand from from this stage over to uh the adult being being able to to operate the hand without visual feedback with the arms at the back of the body so there was a need for a new assessment a new measure this is why I developed this ACMC test which is observational based unid dimensional meaning that validity tests has shown that we measure one single construct the capacity for control we perform this test in daily tasks and the statistics or the development of the test is Rush based so we have this clinical assessment self-chosen by manual tasks we have a number of items we have a rating scale and the result is one measure of capacity for control and this is an example of the output from a rash analysis where you can see oh sorry you can see the um persons with their ability from the least able persons to the most able persons and they are distributed on a scale where the on the same scale as the items that is the most difficult items on the top and the least or the easiest items on the bottom and this uh so this is an hierarchy of items and persons and when we tested it was in accordance with our clinical experience such as the item difficulty matches the person’s ability more difficult items more able persons and more easy items less able persons and in this way we could have uh a measure on for each person on this scale and as Cory mentioned we have the ICF International classification and in in um ACMC can be in in relation to the ICF we can see that this is the integration of body functions here and the environmental Factor the pris into actions such as grip hold and release and these can enable the person to participate in society being more able so we have done pyic testing of this instrument also Cy showed in her graph or her table the the validity and reliability testing that has been performed and we have improved the tests refined the . rating scale and reduce the number of items to make it easier for the later to to score the test and the o rating scale is from the beginning as you you have two extremes the not capable to the extremely capable but then we can grade capacity on this rating scale by looking at the time to perform and the success so we can have one two and three in the rating scale and we have done this uh tested the scale with Rush analysis and here you can see the cat the probability for uh person with a low ability that is minus and how the um so the probability is high for having a zero whereas when the capacity increases in the person the probability to get a higher score increases in time so um we have now used this ACMC and you can see in an in in an individual so we can see both the items where the client has a problem and direct training towards that area but we can also measure the change so we can see if there is a change more than . ACMC units on a zero scale this is a true change so it exceeds the minimal detectable difference or the minimal detectable change we can see that the persons are not capable on this scale with the score from zero to somewhat capable generally capable and extremely capable on the top up to we have used this in a group level to confirm the validity also and we can see that persons who have um experience from using prothesis they stay stable on a high level of capacity whereas new users increase as expected and then we have those people with with less experience and with additional problems that did not um improve in their capacity so we have also developed different tasks to use when we do the assessment the observation and we can see that two row scores difference between the difficulty of the tasks so the difficulty should not influence the scoring so you can use any of these tasks and now we are in the phase of developing new items to um um to uh use or to be able to use the ACMC with the new uh technical development of the multifunction hands because ACMC is not only for children it is also for adults so for example the IIM or the bionic hand tasa hand or whatever we have to have uh items to measure also those the capacity for controlling or operating those more advanced hands so these are the new items that have been um developed and we are now in the phase of data collection uh in a collaboration between different centers to perform a new Rush analysis to see how well they fit in the original ACMC um construct and we should slowly get to the end of your talk okay I’m here I’m almost done so um yes so um I should say what aspect you the the the functional outcomes are useful for certain aspects of function and what I said with the previous instruments it that they were not useful for my my needs but depending on what aspect you’re interested in you can have different outcome measures okay thank you well wonderful thank you very much thank you calling to the stage Wendy our third speaker that’s perfect thank you all right great you can see everything thank you so much claudo for asking me to be a part of this I’m thrilled to be among this group of of researchers um I will what I want to talk about today is really what’s in my toolbox and as claudo mentioned in the introduction I’m an occupational therapist and I work in a in a clinical setting within a research institute so my job is mainly clinical I work with children and adults with Upper Limb loss but I also work with a group of Engineers a group and engineering students biomedical engineering students who are doing research in this so I will tell you a little bit about my experiences with outcome measures and what’s in my toolbox from Clinical perspective but uh I think some of that connection um between the engineer and and the uh clinicians will come about I think in the next talk with Dr Heber so we’ll uh we’ll start here and so this slide we’ve seen earlier in Cory’s presentation and I do think that this is a perfect way of thinking about outcome measures before we decide about which outcome measures we want to use we have to be very clear about what it is we want to measure in my clinical role I’m usually wanting to know for example what is important to my patient in deciding about an appropriate device or was my training effective has the patient learned how to incorporate the use of the hand into his life at home if you’re designing a research project you need to be very clear about what your question is before you decide on a particular comination I’ve used this slide as well thank you Corey you’ve done such a great job in your presentation um I circled the tests in in this um in this tree to identify the the tests that I’m most familiar with and the ones that I use somewhat regularly just because a test is discussed frequently in the literature does not mean that it is the best test to use in any given situation and just because I have test circled here does not mean that I necessarily recommend that’s my disclaimer there is no single outcome measure that reflects all of the domains that’s been said but I think it was worth repeating so we’ve also looked at at at various types of outcome measures according to this sorting strategy and I I think this is a good method for talking about the specific measures that I use clinically I’ve added Advanced measures in here although I won’t speak about them specifically but we do sometimes combine Advanced measures like da data or motion capture with standardized measures in the research setting to get more information to answer a specific question there are advantages and disadvantages to every type of test and I’ve outlined a few in the following slides time tests these are very popular in research settings because they can be confined to a particular setup space they’re easy to combine with kinematic data the instructions for use are very clear and the results are objective in box and blocks you’re you’re counting the number of blocks moved in a minute or the time to complete a particular movement or task in my experience Engineers love these tests from a therapy perspective these time tests really don’t tell us much we find that patients tend to use poor body mechanics to go faster when they’re doing time tests uh so the compensatory movements are ignored for the most part but we do know that tests like especially box and blocks is actually correlated well with some of the other functional tests like ACMC and amula and so in a research setting it really is not a bad test to do to look at grip and control before it gets to a stage of being used in functional activities you can see in these pictures that uh there there doesn’t seem to be a standard way of doing box and blocks test and that is something that’s that’s very important it’s important that you do especially when you’re doing this for research that you’re performing the test in the same way with every person in tes the table height should be adjusted for every person you’re either doing get seated or standing there’s no movement allowed from one part of the box to the other it’s difficult to compare results if the procedure using is not standardized shop kit um we’ve heard a bit about shap and it it’s this test in itself um patients don’t love doing it I have to say that um but it does include by manual functional tasks this timed uh environment and it does encourage use of multiple grip patterns and it uses the prosthesis as a dominant hand which is really what therapists and patients have the hardest time with they don’t enjoy using the prosthesis in an unnatural way if they have an intact hand they want to use that hand uh for manipulation and not the prosthesis but in doing it this way it does allow you to compare devices and control strategies um but we do know that sha has a learning effect so it may not be the best of of the tests to use observational assessments um there are several that have been developed for use in Upper Limb Prosthetics and uh they are very useful clinically they require a patient to complete a functional task using the prosthesis and their performance is scored using graded scales they give us information about the person’s ability and they may help identify problems with control or needs for further training the limitations in these types of tests is that um they can be costly to become a certified Raider uh and the tasks being requested of the patient may not be relevant or may not be familiar to every patient so that’s something that needs to be factored in and considered when when um doing these tests the unb test and the ACMC are similar in that they both look at how the hand is used in familiar tasks and they assess the skill and spontaneity of use of the prosthetic hand the unb test is a good one to use with children but I do use the ACMC test most frequently because it’s easy to administer once it’s set up in a space it involves doing a familiar a familiar task for example packing a suitcase which we see the objects in this picture um and uh it gives a lot of information about how someone is using their prosthetic hand and where their issues are so I I do enjoy obviously clinically these are great tests to use self-report questionnaires I’ve I’ve noted a few here puffy Opus tapes Pi ads that’s a new one I’ll I’ll talk about briefly in a second here and in the check children’s hand use experience question so self-report questionnair are popular in many clinics as they can be completed at home or online as a part of intake forms you often see the dash or Opus used in this way it’s an easy way to collect data for your patient population but I think the information that we get from these types of questionnaires is sometimes misleading Dash and Opus both ask about functional tasks but many of the items are easily accomplished one-handed so it may look like your patient doesn’t have difficulty doing anything thing um from these tests but in reality we haven’t asked them where they do have def so I I take that information um with a grain of salt there are self-report measures that cover satisfaction with Device um pain quality of life and some cover more than one domain the puffy and the check are both used with children both of these assessments are very useful for assessing what activities are challenging as home and how the child can accomplish a particular task these are great for helping to determine therapy goals Wendy you have two minutes left what already okay I’m almost done tapes is a useful questionnaire if you’re looking for information about psychosocial impact of limb loss impact of pain or satisfaction with the processis I’ve also recently discovered this pads assessment so this is the psychosocial impact of assistive devices scale um I’m interested to see this can be used either in looking at expectations for how an assistive device can be useful or to assess the assistive device that you’re using it’s used a lot in other areas of rehab for things like wheelchairs hearing aids and communication AIDS I think it’s actually it it looks to be a useful tool individualized measures this is something that’s really important goal setting measures where you can really get at what’s important to the patient these measures are useful to help identify challenges and make spe specific measurable goals that can be rated over time the copm is one of the measures that I use most frequently to help I identify and prioritize occupational challenges that are important to the what’s nice about this test is that you’re rating not only the person’s ability to perform a particular activity but also their satisfaction with their performance on a scale of to so when you’re looking at re assessing that goal after intervention you may find that performance has improved a little bit but their satisfaction may have improved even more and that users’s perspective is very important last slide what’s missing from my toolbox I think that um we really do need more functional tests that are appropriate or sensitive to looking at the features of all multi-grip or multifunction hands um and I think a method of measuring expected outcomes with a particular device or control system to help help us justify the cost of these expensive more advanced technology components um and again we’ve said this already but I think the input from users about what we should be measuring is missing missing from this panel missing from all of our discussions about outcome measures we need to include them that’s the end of my talk you’re totally right yes we could have a second one with experience from users i’ be very glad about that yes so just on time uh jackly your next on my list will you please share your screen how’s that we see your M yes that’s perfect thank you great well um yeah thank you very much for inviting me to be part of this uh panel with my esteemed colleagues very nice to see everyone and I’m going to talk about the challenges for my perspective from the lab to the clinic and in particular umies here and in particular um this topic of Bridging the Gap from the lab to the clinic and I think we all struggle and understand that it’s hard to move developments from the lab to the clinic so my perspective being a researcher in the blink lab you know and and doing all these these really interesting things with motor control and sensory feedback and then it feels sometimes like an uphill battle to move these Technologies to clinical care and I think that we all struggle with that um in my lab this is a big overview if you just look across the top of we kind of run the gamut from surgical interventions to control feedback and training but I really spend most of my time on the far right hand side thinking about outcome measures because they really are the most crucial to all of the work that we’ve done and when um we like classify or choose outcome measures it is difficult we just have to admit that it’s difficult as our prior speakers have shown there’s quite a few things that you can choose from and it doesn’t always give you exactly what you want so I always refer back to the ICF as the best way of classifying measures on the basis of what we’re actually measuring and in research I think we kind of like to focus on this blue box right we like to restore body function and structure we’re doing new technologies new control control strategies and we want to measure how well are they grasping or moving and that’s fine because we need to know that at a basic level are we restoring body function and structure but the vast majority of measures really function on activities and that is how does a person perform with the device and that is really crucial because we’re measuring actual capacity so will this person be able to use the device in reality What patients probably care about the most is the far right hand side participation regardless of what we do in the lab or in the clinic are they actually going to use the device and is it going to help them engage with the world and what as we move from the left to the right the the challenge is it becomes more and more difficult to measure because there are these different environmental and personal factors that we cannot control in the home environment but we can tightly control them in the lab and steeped in this is the concept that was mentioned earlier and that’s of capacity versus performance so measuring what a person can do does not always correlate with what they actually do in the real world so the key question is how do we measure capacity whether it’s in the lab or the clinic that actually leads to meaningful performance with the prosthesis in daily life so my approach to try and solve this issue is to focus on the human prosthesis as a system and try and measure what is that interaction is the device actually being integrated into this human Loop of control and I really feel that this is one of the key constructs that may help us determine whether um they’ll actually use this device in real world and so my goal was to develop a lab-based assessment that measured capacity but also performance of the human um prothesis system um particularly during goal oriented tasks and the Cornerstone of this assessment is really that visual motor actions are really how we interact with the world we have these multiple underlying autonomous neurophysiologic circuits that end in hand movements and I eye actions that guide our movements and we don’t really control them they happen automatically and so if we can measure this we might be able to reveal a person’s tendency to actually use the prosthesis in the real world and then work to toward sort of optimizing this uh human prosthesis Loop and so the test we developed in our lab is called gaze and movement assessment or gamma and it was really actually developed for the testing of multifunction hand sensor feedback and advanced control strategies and measures two main things where where are people looking and how are they moving and it’s a quantitative assessment that allows us to implement this um um actual values and quantitative measures to these behaviors so that we can Define some body function measures some performance activity measures but also these underlying neurological indicators such as vision and high I hand coordination now I want to pause here for a sec to refer back again to the to Corey’s diagram of the concept of validation and I think what’s important when you’re in the research lab if you’re developing or choosing new metrics to use in the lab you still need to go through all of these rules of validation that everyone else does and so as we developed this metric we were careful to follow the rules of validation we have about published validation papers which I won’t go over now but I’ll just going to give you some examples of how we’ve progressed this work through the validation stages um so the first question was why visual motor assessment well we know from extensive literature that if you have an intact visual motor system you have consistent stereotypical gaze transitions when you’re performing tasks so we started with a normative validation study and that helped us Define our content and construct vity and our test retest and interrater testing reliability and we confirmed that there are these features of eye and hand movements that are very consistent across um normative participants so we have smooth coordinated movements we optimize our our hand trajectory and we always use the VIS use Vision essentially to plan our next action next when we studied prosthesis users we showed that we could quantify how they compensate for their lack of prosthesis dexterity and sensation by fixating more on the prosthetic hand not being able to disengage vision and then spending more time with the object manipulation and difficulty with grasping releasing with you know less efficient movements essentially and this gave us some content validity and understanding that this is what we expect to see with prosthesis users further validation work showed that gamma was sensitive to detecting differences that we thought should exist such as transhumeral prosthesis users have more visual fixation to the hand and more time and grasp than trans radial users you know a trans radial user with a body powered prosthesis can have a very short grasp phase and not have to look at their hook compared to someone who’s using a myoelectric prosthesis and then people who are more skilled have less compensatory movements and less visual fixation to the hand so this really gave us again some some validation that this is what we would expect to see then we went on to look at responsiveness and responsiveness in a single subject design is when you try an intervention and you know you have a reliable test and it actually shows difference so we were able to show that gamma was responsive to providing sensory feedback within a prosthesis system so we had participants with targeted reinnervation surgery and when we added touch and kinesthetic feedback we showed that their visual motor Behavior incl improved closer to normal they didn’t have to look at the hand as much and they had um better movements with grasping and concurrently they also showed things like better error correction and measures of stronger prosthetic ownership so this gave us some concurrent validity that we were moving in the right direction we ALS our most recent study showed that with gamma we could also detect important my electric control issues such as the limb position effect so in certain grasp phases at a high position participants would have you know more challenges with with control and we could see that this unreliability actually Drew more visual attention to the prosthetic hand and when you gave them a more reliable control strategy not only did they improve their ability to manipulate the object but it reduced their visual attention and improved their gaze transitions so I’ve given you an example of a lab-based test focused on visual motor behavior during functional performance that we hope provides some insight on this human preis system but there’s still this Gap and just like the other measures what we really need to be looking at is how do we correlate these features to long-term success with daily prosthesis use and I I put this slide back in um some of the colleagues um on this call were responsible for this initial you know concept of the prosthesis life cycle when you develop something in research and you want to send it out to home use you have to progress through these proving these stages of function activity and participation and I was greatly influenced by this concept and I’ve kind of expanded this framework in my mind to include how do you consider burden of evidence and the role that outcome metrics play when you’re trying to transition things to the to long-term home use pardon me two minutes left thank you and so when you um look at research and development and trying to go to First inum testing you really just want to prove this proof of concept but when you want to go from Clinical application to clinical uptake this is where there is this High burden of evidence and you really need this direct proof of e efficacy to make this a viable therapeutic option and this is where we tried to really get that information um with the gamma assessment but you also o need to start looking at things like personalized goals as been has been mentioned and then making that leap to long-term use and standard of care this is where you really need to be looking at quality of life and and more of these um real world measures and what we need to start doing is tying these constructs together from the start to the end so what can we do and this is my last slide we need to design studies when we’re doing them in the lab that include functional prosthesis system testing there are a wealth of desktop virtual testing platform simulated prostheses wearable protheses There is almost no excuse to still be doing work in the lab that doesn’t involve functional prosthesis testing we need to then compare how participants do with that testing compared to um a non-arable testing system to help develop Identify some of those gaps and then we need to study long-term prothesis users and this is where my next validation work with gamma is going what character istics in the lab actually correlate with successful long-term home use and the last point that I haven’t address but hopefully we have time for in the panel discussion is we need to include prothesis users in design testing and evaluation whether this is through qualitative work participatory design mixed methods but their voices need to be heard right from the beginning um through to the end so thank you very much and I’m looking forward to our panel discussion that’s great thank you very much um jacking thank you very much all of you this was a very engaging um set of talks and I was a little bit too optimistic as far as the time is concerned but I believe that we can enjoy a few minutes uh discussion here so so first thing first the we have a couple questions through the chat so um there’s a there so there’s a note by a user called Swiss and I think this is my old friend wul from T and Vol is is a is a high critic he’s a big critic of my electric prosthesis so he’s an avid wearer and user of uh um body power prosis Vol just that you know this is still on my this is still printed and it appears on my desk on on the wall there so this is a sort of ironic but but serious uh you know uh how can I call it said depiction of how um my electric prosthesis still have a big challenge in body power prosthesis so I’ll feel you a question which probably W would ask directly how would a body power prosthesis fair in these uh um um functional assessment protocols I believe quite well but if you have a transradial amputation I believe a BPP is still quite a challenging if not better solution than than a than a my electric prothesis what is your take on that who wants to go first Jacqueline please sure yeah I’m happy to address that and we certainly um designed our measures so that we could test all types of technology and all types of users and that’s one of the keys because when you look at you’re right patients with body powered prosthesis their grasp is very fast they have prop reception from the cable they don’t need to look at the prosthesis but what you do see is they have altered um body compensations so they’re using their shoulder and their trunk quite differently so they’ll have more abduction and more movements in shoulders compared to someone who uses a my electric prosthesis so I think there’s a tradeoff and then um when it comes to some of the more dextr grasp capabilities that’s going to also be a challenge but we also let them do the tasks the way that they do them best so if they can’t do a full hand grasp and they use their hook to pinch the edge of the cup to move it say example that’s fine but what we’re looking at is how well they’re integrating into their visual motor system and I will say we’ve seen some very natural visual motor behavior from trans radial body powered users but they have a lot of body compensations and those are the patients that I see years down the road coming in with so much shoulder and neck pain that they can’t use their arms anymore so it’s a it’s a it’s a two-prong challenge so I I I see that as why why would a myoelectric hand user have a better have a so a lower amount of compensation movements provided that these people have no rotation wrist because then if you give me the the risk now that that the the amount of compensation goes dramatically down provided that you can control it in the right way which we’re far from being able to do does anyone else want to take that I’ll just say I am often referring to testing multiple degrees of freedom and depending on the task you’re absolutely right someone who has a you know a a static wrist and a standard autobo hand without a multifunction hand also can show compensations and they also show different compensations so I’m not saying that they don’t have any compensations saying that each person has a different set of visual motor behaviors and so it needs to be individually customized and assessed right that’s that’s great um there was another question in the chat by Eran capanu he says so this is for ly I guess why so can you explain no visual feedback needed how can they so the patient change orientation for grasping without seeing an object and then we got an answer by V himself who says no visual feedback needed as a user I can hit the target without looking or I know the state of gripper without visual check and that’s because bpps still give you an idea of how much force you’re applying right yeah exactly so and also they can have U support from in the example where the lady who was tying her apron behind her back that she also has information from the other hand ah right ah yeah and she’s a monolateral translator so she does yeah yeah yeah right but this is um I had to say this is more a problem for us Engineers than I mean we’re we’re trying to make control so the control is still the bottleneck in my opinion as far as the as far as the the general acceptance of self- power prosthesis is concerned and I believe we still have a lot of but this is a talk this is a webinar about functional assessment so I’m really happy that we have see in such a Consortium there’s a lot more regard for the patient than usually I hear in conferences specifically of Rehabilitation Robotics and this is great I so I’ll I’ll ask a question to all of you in a in around Robin I feel that there’s a so in the functional assessment world I feel that there’s a tradeoff between the simple ones which are clearly measurable say the shap or the box and blocks but they don’t you know they hardly translate to ADLs I guess and then the ones which are more realistic the ACMC being an example but in that case there’s a lot of subjective assessment provided by the Raiders and I believe that Jaclyn’s attempt at measuring the Gaze and placing someone in this in this in this cage with motion tracking is an attempt at Bridging the Gap over there so getting a a highly realistic uh functional assessment protocol that we can clearly assess using objective data is that the case I think you’re right but uh in one sense that’s only for research settings so in clinic we cannot have this so we have to rely on the u s to say objective measures or observational based and those we are are trying hard to um standardize but still there is a subject subjective um uh influence part in it anyone else want to add something uh I would say that in in clinical spaces it’s um it’s probably that we need to have more than one measure to to do this and so we’re either combining something like um an ACMC and some goal setting measures to get more of the patients perspective of what’s Happening um but I also think that Bridging the Gap um is is more easily done if you have a a therapist involved in that research design um and can help you know answer those questions in the lab setting um before it ever gets to a clinic setting I had a question actually in that regard um maybe for you claudo uh just knowing from that research end of working in an engineering lab um what are the barriers to having a therapist or to having therapy um consultants in research designs there’s none so actually there’s actually none it’s it’s easier not to do it because there’s fewer people involved and okay you need a lot of interdisciplinary understanding if you do that and I believe so we have so cor and I have a project proposal under under review and and and one big part of that a third of the whole budget slightly less goes to those institutions which are involved in codesign so like you know we plan to design stuff for ues why not going to ues and asking them what is it really that you want K would you like to say something about this possibly yeah I think that the the codesign and co-creation aspects get more attention nowadays and I think it’s it’s really good that we also address this in future uh um research proposals um so that’s what we did with this European Grant with that we ask for um and we also mentioned it in in our presentations as you um uh as you noticed I also uh coming back to the former discussion um I’m also wondering what um activity monitoring will bring us in the future it’s not good enough I think at the moment to see what kind of activities people do and for example also how they do it qualitatively but um I have some hopes that in the future we will make progress and that we can see what people do with the prestiges at their homes how many times they will do it and and what kind of uh compensatory movements they make um maybe Claudia you as an engineer can reflect on that uh I think jacn is even more yeah well well yeah actually picking up on all of the comments from lah Wendy and Corey I I think that where the Partnerships can really help us is you know as clinicians and and patients Define what’s important all of these measurements that we do are becoming easier and easier so yes right now gamma we took all the equipment to make the most fine grain analysis that we could but now we’ve integrated it into VR so as they’re doing training we’re getting their eye tracking and those measures automatically without any extra effort ey trackers are quite like low profile now and easy to anyone can put one on and do a system the ACMC and quantifying their visual motor behavior and so when we look at what’s what’s important is still to identify like what are the keys is it grass duration is it eye latency is it like we still need to know and once we narrow down like in all of these measures the commonality is this like it’s how well they’re how well or their timing of their gate of their of their grip and we can know that from the observational measures and from things like technically like gamma then we need the researchers to help us decide how do we or develop ways to implement this in the clinic so it is easily done and I think that that’s still you know a few years work in progress but having that information it there’s you know you can put cameras up and do an algorithm and get all the motion capture with zero markers right now so you know there’s there’s all of this potential to iterate down the technology I’m not worried about the technology burden I think what we need to do is focus on on identifying the key measures that correlate with their atome usage as you mentioned Corey and then developing the technologies that answer those questions for us I’m also I can something yeah so the technology gets every year literally more and more wearable cheap lightweight I mean we track the the body configuration using inertial measurement units we don’t use motion capture anymore well that doesn’t tell you where you are around but for instance if if you are in the gamma setup then you don’t move too much so that can be used immediately to gather how the amount of compensatory movement compensatory movements so yes so technology gets cheaper and cheaper and more and more effective and we should grab advantage of that I got a got a last question by um Mr Moes Amani and well this is a this is a wonderful question is there a lot of difference between the results you get in your lab and the results in the real world at home Okay who wants to start here Mr zani your question has a lot of Ingenuity and it’s it’s the it’s the the million question say yeah it’s a really good question I think and I think um I don’t know I’m not aware of any any uh study that has investigated this particular topic uh so doing the same uh things in the lab and and uh using the same uh measurements instruments at home but it definitely will be a huge uh huge difference and that’s that’s my feeling what we ask from participants in the lab is my opinion totally different from what they do at home so uh probably there will be a huge difference between what people do at home and that’s why we I think need uh activity activity monitoring or um motion caption uh systems that can be integrated in the home situation um because it’s so different from what they do uh in the lab uh versus at home right I had one one question or topic that I think is also interesting and that’s what we mean by use proes is use and this is related to what you talk about if it’s in the lab or if it’s in real life so if you ask the patients they may say that they use the prosthesis full day but you cannot monitor any AC actions in the hand so it how how would you value this usage I think this is something that already Blair and Levi harra highlighted must have heard them talk in recent conference so they’re get they they’re collecting a lot of data because they have a few think thousands COA systems around but then the point is to my personal question would be how do you know what people were doing when producing those data because you cannot have a motion tracking system at home this is clear so there’s still some very interesting statistics are coming out so like how how often you switch the grips how often you use this grip or that grip the other challenge in looking at that in looking at the usage of home and is really uh who who is the right person to interpret what’s happening at home and what the value is of that prosthesis for that person yeah you know if they if they are wearing their prosthesis for very specific tasks how is that any less or more valuable um than the person who wears the prosthesis all day and either uses it or doesn’t you know that the value for us you know that whole question of success and and um the amount of wear that a person you has their press thesis on it’s difficult I I get a little bit I I get scared thinking about what how people interpret that data yeah I have a comment also too I think it was your slide uh you talked about these expensive new devices and how to um to um justify the cost justify the costs and we in Sweden or in at least my Center we use the copm the Canadian occupation performance measure to to um to um justify this to the the ones who pay for the devices by by describing how the patient performs in daily life and that’s their own words how they they themselves Express the need for a typic or a specific function in a hand or so I think those comments reinforce um the issue that we do need to use multiple measures we won’t have one key so if you have a copm like the only reason to look at activity is if one of the goals was to increase the activity of usage right if it was to have a passive hand for B manual support then and they reached that goal and they’re not activating a Mayo that that doesn’t mean that they didn’t reach their goal so combining again the question what’s the and adding what’s the patient’s goal with choosing the appropriate outcome measure is is key definitely yeah and one last so so one interesting remark by vul again it says compensatory compensatory movements you feel um after one five days of wearing it neck and back gets stiff and painful yeah so yeah that’s a that’s a lack of functionality or dexterity on the proc and then he adds that in in his experience a well-built body powered arm is still better than still better than a my electric hand I get a last question so followup question by Mr zamani so the one who ask well how do you compare the two things and he asks so what should we do to have the same results the current literature is it focusing on this issue or not my view is sort of starting very very initially but not really focused so far even because there’s a so there’s an objective problem so how do you monitor people at home I mean you could Mount you know a motion tracking system at home at but but you can do it in one case as see we try we’re trying to find out statistics out of a number of different cases yeah there also a lot of ethical issues to measure things at home so it’s uh it’s not so easy because um preferably you would video a person uh for a certain amount of time in the home situation and uh have some uh some motion tracking system on uh on his body and then um and collect all the videos but it’s not that easy um what happens when you when the patient goes out yeah yeah for example yeah there are all kinds of constraints um so that is a bit bit of future music I think and um but um I think the fact that we are now talking about this and that this is in our minds that we should more look at our homes is the first step and I think this first step has been made already and now we are um we have to look more at how should we do this right so I was just also GNA put a plug for also like we’ve actually been drifting more towards M mixed methods and qualitative research because even if you give a patient a prosthesis and you ask them to do a daily diary I mean you’re you’re not going to get it’s been shown you’re not going to get accurate reporting necessarily and there’s the problems with interpreting things like um activity monitors what were they doing during the time but when you talk to patients after or or participants after they’ve had a change in a new system at home um or even if they’ve just tried it in the lab for a few hours the the open-ended questions and the data that they actually spontaneously give you are is the most valuable information that we ever get whether it’s to tell us how the system’s performing or to tell us what we should have done differently or how we could address things in the future so I think adding a proper qualitative methodology to any research studies um is is something that we should all be looking at in in in concert with our our quantitative methods yeah fully agree yeah yeah we agree as well yes that’s stuff that Engineers don’t like if you don’t if you don’t quantify then it doesn’t exist for us find your clinician collaborator find your Wendy good I I just one thing I I’m sad to hear about these overuse problems or or body problems from wearing or using his pesis the person that had this question and I wonder if perhaps having more contact with a therapist a physiotherapist or an occupational therapist to to find a way to to uh change motor Behavior to to uh activity performance to to prevent from this I think that’s also important I’m not sure if Wolf is still in the chat if so please give it in indication as far as I remember wolf has been he’s an amputated person yeah never mind since so I believe he has tried all possible solutions okay but not got a got a comment from him says no overuse comes from not wearing it my body power is perfect so wolf is a fan of body that’s good good I think well this is hour minutes I enjoy this tensely uh um is there’s any final remark that you want to uh throw on the plate if in case we have still time I can stay here until but don’t think our audience is gonna be so happy about that I don’t see any any feedback maybe I would maybe I would just say um I think all of us are always open to collaboration and trying to get people on board with focusing on outcome measures helping to validate outcome measures studying long-term term affects the proceses users so I think that’s the other thing that’s important it’s a small it’s a small group of researchers and a small population in in relation to some other um population issues so so I think collaboration across sites is always important as well okay got a couple people Thanking us so that’s great I think we could use the occasion to close this webinar dear colleagues has been it’s been a pleasure and honor to have you here and I hope that um that we’ll have more um engaging thought inspiring webinars such as this one um small reminder for the audience these webinars happen more or less every two months so the next one is scheduled I believe end of September beginning of October and it will be about lo and behold the cybathlon a very very controversial issue so good thank you very much for joining and see you soon bye-bye thank you bye bye you
  2. Introduction:
    Host: Claudio

    Welcome Note: Claudio introduced the webinar and highlighted the importance of functional assessment in prosthetics. He emphasized that while engineers often overlook functional assessment, it is crucial for determining the real-world effectiveness of prosthetic devices.

    Presentations:

    Professor Cory Fosl:
    Focus: Need for outcome measures in upper limb differences.
    Key Points: Discussed various types of outcome measures, their importance, and psychometric properties (validity, reliability, responsiveness, and interpretability).
    Challenges: Highlighted the lack of comprehensive outcome measures that cover all ICF domains (function, activity, participation).
    Future Directions: Suggested developing a core set of outcome measures for different ICF levels and involving service users in co-creation processes.

    Professor Liselotte Hanson:
    Focus: The ACMC (Assessment of Capacity for Myoelectric Control) test.
    Key Points: Described the development and application of the ACMC test for children and adults with upper limb prosthetics, focusing on measuring control and use of prostheses in daily tasks.
    Validation: Explained the Rasch analysis used for validating the test and ensuring its reliability.
    Expansion: Mentioned ongoing efforts to develop new items for multifunction hands.

    Professor Wendy Hill:
    Focus: Clinical tools and their application in assessing prosthetic use.
    Key Points: Highlighted various assessments like box and blocks, SHAP, ACMC, and self-report questionnaires (e.g., COPM, OPUS, PUFI).
    Clinical Relevance: Stressed the importance of using multiple measures to get a comprehensive understanding of a patient’s prosthetic use and satisfaction.
    Challenges: Addressed the need for functional tests sensitive to the features of multi-grip or multifunction hands.

    Dr. Jacqueline Hebert:
    Focus: Bridging the gap from lab-based assessments to real-world application.
    Key Points: Introduced the Gaze and Movement Assessment (GAMA) developed to measure visual motor actions and their integration into prosthetic use.
    Validation: Detailed the rigorous validation process and the importance of correlating lab-based measures with real-world performance.
    Future Directions: Emphasized the need for studying long-term prosthesis users and including qualitative methods to capture user experiences.

    Panel Discussion:

    Topics:

    Comparing body-powered prosthetics to myoelectric ones and the compensatory movements involved.

    The challenge of measuring real-world prosthetic use and the limitations of current methods.

    The importance of involving users in the design and evaluation process to ensure the relevance and effectiveness of prosthetic devices.

    Key Insights:

    Technology is advancing, making it possible to integrate more sophisticated measures into daily life without imposing a significant burden on users.

    Collaborations between engineers, clinicians, and users are essential for developing effective assessment tools and improving prosthetic design and function.

    Conclusion:

    The webinar concluded with a note of thanks to the participants and the audience. The importance of continuous collaboration and development in the field of prosthetics functional assessment was emphasized, with a call to action for more interdisciplinary and user-involved research.

    The next webinar is scheduled for the end of September or beginning of October and will cover the Cybathlon, a topic known for its controversy and importance in the field of assistive technology.


Cite this article:
Wolf Schweitzer: swisswuff.ch - User view on performance assessments and lab/clinic user communication [comment after ICORR Webinar #3 – Functional assessment]; published 03/06/2024, 21:31; URL: https://www.swisswuff.ch/tech/?p=13199.

BibTeX 1: @MISC{schweitzer_wolf_1761918963, author = {Wolf Schweitzer}, title = {{swisswuff.ch - User view on performance assessments and lab/clinic user communication [comment after ICORR Webinar #3 – Functional assessment]}}, month = {June}, year = {2024}, url = {https://www.swisswuff.ch/tech/?p=13199}

BibTeX 2: @MISC{schweitzer_wolf_1761918963, author = {Wolf Schweitzer}, title = {{User view on performance assessments and lab/clinic user communication [comment after ICORR Webinar #3 – Functional assessment]}}, howpublished = {Technical Below Elbow Amputee Issues}, month = {June}, year = {2024}, url = {https://www.swisswuff.ch/tech/?p=13199} }