Outlining the importance of post-commercialisation design reviews: A User Perspective
•Download as PPTX, PDF•
1 like•1,144 views
This talk was given at Informa Life Science's Medical Device and Development Conference on 18 October 2012. The focus is on learning from a user perspective, 'in the wild' beyond when the device has been deployed.
Recommended
More Related Content
Viewers also liked
Similar to Outlining the importance of post-commercialisation design reviews: A User Perspective
Similar to Outlining the importance of post-commercialisation design reviews: A User Perspective (20)
Recently uploaded
Recently uploaded (20)
Outlining the importance of post-commercialisation design reviews: A User Perspective
- 1. Outlining the importance of post- commercialisation design reviews: A user perspective Dominic Furniss (PhD)
- 2. • Appreciate the importance of a user perspective and learning about devices ‘in the wild’ • Appreciate how post-market input fits with design processes • Be aware of methods used for post-market studies • Be aware of the benefits of continuous and discrete post-market studies
- 3. Why a user perspective?
- 4. Why context? Tang, C. & Carpendale, S. (2008). Evaluating the Deployment of a Mobile Technology on a Hospital Ward. Proc. CSCW 2008.
- 5. Linear Design Cycles A simplified version of the waterfall model of software development (Sommerville, 1992 cited in Preece et al., 1994)
- 6. Linear Design Cycles Unknown author – flatearthcargoship.jpg
- 7. Non-linear Design Cycles The star life cycle (adapted from Hix and Hartson, 1993; cited in Preece et al., 1994 p. 49)
- 8. Non-linear Design Cycles Spiral model of software development (taken from Boehm, 1988)
- 9. UCD: User-centred Design Identify need for UCD Specify context of use System satisfies Specify Evaluate design specified requirements requirements Produce design solution
- 10. ISO 62366
- 12. Benefits • Learn about strengths and weakness of design • Provide ideas for evolving product – Small changes for version 1.x – Big changes for version y.1, z.1… – Changes to start a new line of product or service • Can improve customer relations • Give data on acceptance for sales teams
- 13. Post-market Study Dimensions • Summative Vs Formative • Continuous Vs Discrete • Shallow Vs Deep • Quantitative Vs Qualitative • Expensive Vs Cheap • Internal Vs External
- 14. Research Tools Helpline reports Customer complaints Surveys Diary studies Telephone interviews Auto-ethnographies F2F interviews Field studies and Focus groups ethnographies
- 15. Case Study 1: Day Care Unit • 5 days of observations – over about 2wks • 31 infusion observations • 2 interviews (was meant to be 5) • Main focus was on the infusion pump • Understand the context in which these work • Structured approach to analysis: DiCoT
- 16. Analysis
- 17. Redesign Highlight 1 I"haven’t"got"the"Volume"To" Be"Infused"(VTBI)."What"is"it" for"20ml/hr"over"15min?!"" Oh!"It"won’t"let"me"skip"the" VTBI"input"either."
- 18. Redesign Highlight 2 B EEP. B EEP. ! Wa it ! M y b a t t e r y c h a r g e w o n ’t l a s t t h e t ime o f t h e t r e a t me n t y o u w a n t .!
- 19. Patient Comments “I don’t really care just as long as it helps cure me.” Oncology Outpatients Unit “After you have been lying here for three weeks you tend to notice things that you might not usually… like these pointless alarms. They can go off at any time, sometimes 2 or 3 in the morning.” Haematology Ward
- 20. Case Study 2: Oximeter Typical, atypical and non-use of functions
- 21. Example 3: Emergency Call button Appropriation has led to service development
- 22. Post-market Process 1. Device to market 2. Gather information (opening continuous channels and conducting discrete projects) 3. Consider new data – validity, consequences, costs and benefits 4. Develop new requirements for v1.x and vY.1
- 23. Summary • Commercialisation/implementation is not the end • Rich source of feedback and learning from users ‘in the wild’ • Multiple methods to use, including continuous and discrete monitoring • Benefits for product and services
- 24. Acknowledgements • Prof Ann Blandford and Dr Astrid Mayer • Healthcare Human Factors for their video • CHI+MED is funded by EPSRC Programme Grant EP/G059063/1 • d.furniss@ucl.ac.uk • www.chi-med.ac.uk
Editor's Notes
- I’m Dominic Furniss and I’m a postdoctoral researcher at UCL on a project called CHI+MED. CHI+MED is sponsored by the EPSRC and across 4 universities in the UK. The project is focused on making medical devices safer and more user-friendly. We have researchers investigating regulations, formal modelling of medical devices, and the psychology around medical error. My role on the project is to go into hospital and learn from nurses and patients about what they like and dislike about the devices they use, and to investigate any issues they have with them – we have other researchers that are looking at medical devices in the home and evaluating mobile medical devices too.The philosophy that we take on the project is very much around usability – a User-Centred Design (UCD) approach. We want technology to be made that is as user-friendly, intuitive and safe as possible.
- So, in my talk I’ll cover the importance of taking a user perspective, we’ll see how post-market input fits with design processes, we’ll cover some methods that you can consider using for post-market input, and we’ll outline some of the benefits of continuous and discrete post-market studies.
- So, why take a user perspective?Has anyone seen this picture before? Please don’t shout out if you have, just raise your hand. The interesting thing about users is they see the world differently to what you do, and you will see the world differently to them. There might still be the same things in-front of our eyes but they see different things to what you would. E.g. my Dad was installing Skype, he had nearly finished the installation and he was at the stage of agreeing to the End User Agreement – he read this and was extremely confused! “End… User Agreement – no I don’t want to do that” OK and Cancel didn’t make sense to him as he wanted to progress but didn’t want to end the agreement.You might know your medical devices inside out but users don’t, and what seems obvious to you might not be obvious to them. To engage with the user perspective we can do user studies, observe users and talk to them about what they think of the device and its design.By engaging with real users we reveal more about how they see and engage with the world.Can anyone see the picture yet?
- Tang, C. & Carpendale, S. (2008). Evaluating the Deployment of a Mobile Technology on a Hospital Ward. Proc. CSCW 2008.We also think context is important. We can learn a lot away from context in research labs, design studios and simulation centres, but they do not mimic the realities of actual practice. If we don’t understand context properly we can have nasty surprises when products are actually released. This is a study of Computer on Wheels by Tang and Carpendale who identify a number of issues with their use… e.g.They are wheeled to where they can get wireless creativityThey encourage nurses to rush medication administration because of the time out functionThey clutter the corridors and are too large for the bedsideWe need to understand context to make appropriate design decisions.We can learn a lot when our designs are used ‘in the wild’.All of this is about ‘learning’ so things work properly in context. This might seem like common sense, and a lot of it is, but it is surprising that even simple things can go wrong or could be improved.
- So we’ve covered why it is important to engage with a user perspective and with devices in context, which are both important for post-market analyses - but how does this fit with normal conceptions of the design lifecycle?The problem is that most design lifecycles end once the product has been designed, which makes sense. Like this linear design lifecycle, but what happens after? We need a plan otherwise…
- We feel like we are going over the edge... If we don’t have a plan.
- These non-linear design lifecycles make envisioning the continuation of ‘learning’ and ‘design’ after the device or product has done to market.… and that’s the point that I want to make here, that we shouldn’t stop learning and improving once the device has gone to marker.
- This spiral model…
- … and this UCD model emphasise cycles and learning and adaption and these should continue after commercialisation.From the Usability Professional’s Association (UPA)
- This is the model in ISO 62366, which is about integrating usability engineering processes into medical equipment design.Again we have cycles of learning – and this specifically has a stage of ‘post-market surveillance – which could feed into further design iterations. These new design iterations might be small or they may be a step change.
- We covered a lot of complicated design processes in that section but the essential message is that we should not stop to ‘Look, Listen and Learn’ after commercialisation.We can learn a lot to improve our products, and in cases where the design has issues we can seek to improve them. Hopefully, we would have learnt a lot in the design process and so there is wide adoption and a smooth transition to patients and clinicians using your products in practice but sometimes this doesn’t happen, and things go wrong…
- So what benefits can we expect from post-commercialisation design reviews?Quality ImprovementInnovationCustomer relationsMarketing
- There are many different dimensions that might provide the basis from which to design a study…You might be seeking improvements and new ideas; or you may be seeing if customers/users are satisfied.You might go broad and shallow (survey), or you may go narrow and deep (ethnography)It might be cheap or expensiveIt might be internally led or it may be outsourced for an independent look.
- There are different research tools that you can employ for to gather post-market dataSome are continuous – top levelSome are more discrete, that might be triggered by quality control and improvement measures – bottom levelI do ethnographies and I thought I would talk about some of the things that I have found in my research ‘in the wild’.
- Haematology and Oncology Day Care Unit….
- I applied an analysis technique that allowed me to describe the how work is performed around the use of the infusion pump, including the physical context, the social context, and the work processes… Through my observations I saw quite a few unremarkable errors – they were recovered fro quickly. But some deviations from expected practice seemed to highlight a need for design intervention in the product and in how it is used.Furniss, D., Blandford, A. & Mayer, A. (2011). Unremarkable errors: Low-level disturbances in infusion pump use. Proc. British HCI.
- So the nurse should always have the Volume To Be Infused, the time for the infusion and the rate. The pump I was looking at required the user enter the VTBI first and then one of the other values. However, in one situation I observed the nurse did not have the VTBI but only the time and the rate, the problem is that the pump doesn’t allow her to skip it. Instead she struggled with the maths, guessed the answer a few times, turned the pump on and off and started again.Why can’t the pump skip the VTBI for these cases?Why has VTBI been prioritised? Furniss, D., Blandford, A., Mayer, A., Rajkomar, A. & Vincent, C. (2011). The visible and the invisible: Distributed Cognition for medical devices. Proc. EICS4Med.
- There is a need for patients to remain mobile in this outpatient unit – e.g. so they can go to the bathroom. So, ideally, the pumps will not be plugged in but be operated on battery power. Some patients have particularly long regimes where they are in all day. In such cases the nurse said that they need to remember to check the battery indicator to see if it will last or not.If the pump knew how long the battery was going to last, then could it give a warning if the requested infusion programme is longer than the remaining power in the battery? Furniss, D., Blandford, A., Mayer, A., Rajkomar, A. & Vincent, C. (2011). The visible and the invisible: Distributed Cognition for medical devices. Proc. EICS4Med.
- You get conflicting comments in practice… and sometimes you see that the same device can have quite different issues in different contexts. For example, the outpatient doesn’t care too much about the alarms as he’s going home to bed at the end of the day; this patient in the haematology ward is in a room on his own and has been disturbed by the pumps alarms during the day and during the night too.
- Haematology ward… story: Often staff tell me that there aren’t any issues with the devices and they work fine – this makes me doubt the quality of surveys and interview data BUT obviously they can be useful. Sometimes things happen when you’re present that are very interesting, that are part of the ebb and flow of ward life. This story is about an oximeter which measures the pulse rate and blood saturation. It is normally wheeled up to a patient, used for readings and then wheeled away again – no problem.I was in the ward one Saturday morning and a doctor came out of a side room and asked if anything could be done about the machine’s alarms. The nurse said she had tried everything. I thought this strange. She explained what she had done. I referred to instructions on top of the device. She said that she would try that. She did but didn’t know that it had worked – she should have done if the device ad given appropriate feedback.Could this have been more intuitive for the nurse?
- This isn’t my example but another researcher… he told me that some patients were using the emergency call button as a social call. Seeing how people were appropriating this device led the company to adapt the services that they offer people. Now, people can pay extra to use it as a social tool as well.