Oxford Open Grand Rounds

00:00:06.560–00:00:12.080 Olivia: Welcome everyone. Thank you for joining us. As usual, we'll give it the one minute courtesy and then we'll get started.

00:00:44.800–00:00:48.879 Olivia: Hello everyone just joining us. We will give a one-minute courtesy and then we'll get started. Good to see you all.

00:00:51.600–00:00:53.280 Olivia: I see we have some folks from different corners of the world which is great.

00:00:55.520–00:00:58.879 Olivia: Hopefully this time was relatively convenient for you all and it's not the middle of the night.

00:01:11.200–00:01:43.680 Olivia: All right. Well, let's go ahead and get started. It's wonderful to see everybody. Good morning, good afternoon, and good evening. We are very excited to welcome you to our Oxford Open Grand Rounds, which is titled, the quality improvement impact gap, where we go from here. And judging based on the questions we got in the registration, this seems like a very hot topic and we will certainly be addressing some of those questions in today's content. I'm very excited to welcome you to this Oxford Open Grand Rounds event. My name is Olivia. I am the Associate course director for our Masters in Surgical Science and Practice and our PGCert in Patient Safety and Quality Improvement.

00:01:45.439–00:01:54.159 Olivia: Really focused on all the nonclinical skills that clinicians need to be successful in their clinical career. I'm very excited to welcome Tom and to pass it over to him.

00:01:57.200–00:02:12.319 Tom: Thank you very much Olivia and welcome to everyone joining us. my name is Tom Revington. I'm a lecturer in the Medical Sciences Division here in the University of Oxford. Olivia and I together direct the Masters and PGCert programs that Olivia mentioned.

00:02:15.120–00:02:36.319 Tom: We'll come back and just me mention a little bit more about that at the end of this session for those that are interested in those programs. Within those programs, I run one of our modules focused on quality improvement. It's called Quality Improvement Science and Systems Analysis and that makes it a particular pleasure for me to introduce you to our guest today, April Taylor.

00:02:39.920–00:03:37.200 Tom: Who is joining us from the United States. I'm going to make a brief introduction to April and then pass over to her. So, April is the chief operating officer at Johns Hopkins Hospital in Baltimore, the United States. And in that role, she's responsible for the provision of care and for the quality of the care that is provided. She also is the president of the United States National Association for Healthcare Quality, NAHQ, which is an organization dedicated to setting standards in healthcare quality and safety and April has been involved in supporting, leading and teaching about improvement work for over 20 years. So great pleasure, April, to have you with us. Thank you very much. We are looking forward to hearing from you on the quality improvement gap.

00:03:41.200–00:03:59.599 April:  Well, thank you Tom. Thank you, Olivia, for the opportunity to really present to this audience today. I'm honored and excited to speak to this diverse group about the quality improvement gap that still exists across the world despite decades of improvement efforts and where ultimately we go from here to get to some sustainable results.

00:04:02.720–00:06:17.520 April: And just thinking about sort of my background and quality improvement and sort of you know what got me here as you mentioned my current role is as Chief Operating Officer. So, I'm spending a lot of my time in in hospital and ambulatory operations but I spent 25 plus years really growing up in quality improvement and the efforts that have been happening internationally over that time frame. My roles have ranged from you know boots on the ground quality and abstracting charts and looking for dangerous abbreviations to then eventually working my way up to be a vice president for quality and safety. Over that course of time, I spent a lot of my work not just operationally working in quality and safety but really focused in on the research and academic components. At one point in my career, I got a degree in applied statistics because this really all ties into how do we think about the right competencies that we need as we continue to to grow in our quality improvement journey. So, if you go to the next slide, one of the things that I would highlight is that over the course of our time, we're going to go through these different objectives and on the next slide, we'll talk about, you know, the outcomes in healthcare and how they continue to be variable across the world, right? And it's really demonstrated by a variety of rating and and ranking organizations like those shown on the screen. There's obviously a lot of debate sometimes around some of these metrics, so I'm not going to spend much time here, but really just to share that performance on safety measures show great progress in some organizations even within the same country or district, but then others continue to struggle. And so, the obvious answer that most people initially jump to is that the financial resources are different, right? Or resources in general are different. And it's true, right? When we think about lower- and middle-income countries, we might expect to see differential outcomes, but resources really aren't the only factor because what we find if you were to look at some of these reports is that organizations with seemingly endless resources and a real drive to improve have also really continued to lack in sustainability are promising interventions and then thus their outcomes. And so the question is really why is that? And so that's what I'm hoping that we're going to explore right over this next hour together.

00:06:20.240–00:07:00.960 April: So if you go to the next slide, when we really start to think about how we're building the future of healthcare quality, you know, I think we should start with sort of that shared understanding that quality is not a single initiative or metric or department. It's really an integrated system. And so, I often go all the way back to the basics where a lot of the axioms of our early pioneers of healthcare quality continue to ring true. So, I think about Edward Deming who said that every system is perfectly designed to get the results that it does or say it differently that organizational outcomes are a direct result of the underlying processes and structures not accidental failures.

00:07:04.000–00:08:52.240 April: And so, to me a useful framing comes from the classic sort of Donabedian model structure process outcome which is often referred to as the conceptual framework for quality. That model really reinforces that quality is built from different interconnected components evidence measures and outcomes and that they all work together within a defined structure. And so quality in this sense is not abstract. is really designed and operationalized and sustained through intentional systems. And you're going to hear me talk a lot about sort of structures and systems because I think that's part of the gap that ultimately we're trying to address because quality ultimately we don't operate in what I would call a controlled environment. If you go to the next slide, you'll see this graphic that we pulled together that you may have seen described in different ways.  there was an article by Lloyd Provis back in 2011 that was in BMJ quality and safety and he talked about the contrast between  research and healthcare and in that article he described research like a lake right controlled stable and bounded and then he talked about healthcare in particular health care quality like a raging river where it's dynamic it's fast-paced it's moving it's highly variable and this distinction matters when we think about the work that we're trying to do in healthcare quality whether it's in our clinics our wards you name it in a very highly variable environment because too often what I find even though I've   been working in healthcare quality for you know 25 plus years is that we still attempt to apply what I would call lake like thinking right controlled intervention static models to a system that is behaving like a river and so the result is that we have inconsistency volatility and gaps in our performance.

00:08:56.399–00:09:07.040 April: So here's the question, right, that we're wrestling with, right? And the question that we'll talk about during the rest of our time is how do we create quality in a system that is defined by constant motion and variability?

00:09:10.160–00:09:38.880 April: And the answer is that we need to focus on structure. We need to focus on building structures that can help us reduce some of that volatility. So, it's not about eliminating complexity because health care is complex. You maybe you've heard that term vuka. It's volatile, un uncertain, complex and ambiguous, right? That is healthcare. So, we're not going to eliminate the complexity. But what we can do is create stability within that. And so that's really where I think the field of quality and quality and safety must shift its focus a little bit.

00:09:40.640–00:11:58.160 April: Historically, we've been much more focused on just the outcome and initiatives piece, but not enough attention really thinking about the foundational structures that need to be present that are going to ultimately enable those outcomes to become reliable and sustained. So if you go to the next slide, some of these specific structures that I think are foundational which we need to really revisit to bridge this gap include incentives, education, standardization, and then competency development. So when I mention incentives, that's really looking at how do we ensure alignment between what is measured, rewarded, and prioritized. I worked for an organization earlier in my career and we were doing a lot of high reliability work and it was myself and a chief medical officer. We were meeting with a team of folks to really describe high reliability principles, how we wanted to embed them in practice and what some of our expectations were. And what we quickly identified was that the incentives for that group of staff was really around productivity and and sort of pushing out numbers because what they said to us was that they didn't have time to really focus in on some of the high reliability behaviors that we were trying to embed. They said that's not what they were incentivized to do. That's not what their leader ultimately rewarded them to do. And so as much as we there was agreement that yes, we should focus on these high reliability principles to be more safe in the care that we were providing when the rubber really met the road for them was that they were going to be incentivized by productivity and things like how quickly they could turn around their work or whatever their output was. So there was just a level of misalignment right where organizationally we needed to refocus or restructure education building workforce capability and not just enforcing compliance. And you'll see a theme here. How do we build capability and capacity standardization? How do we create consistency and how quality is defined and executed? That's still something that we are debating today right across organizations internationally, particularly competency development.

00:12:00.399–00:12:07.760 April: How do we equip professionals and not just healthcare quality professionals, how do we equip all professionals working within healthcare to operate effectively within complex environments.

00:12:11.760–00:13:27.600 April: So what I would offer is that these four incentives, education, standardization and competency development, they shouldn't be secondary considerations. I think often we sort of push them to the side. These are the infrastructure of quality. They are really foundational structures that we need to pay more attention to. And so Tom mentioned before that I’m currently the president for the National Association of Healthcare Quality. This is one of the areas where NAHQ (National Association for Healthcare Quality) has really stepped in to play a critical role particularly in the United States, but they also do have some footprint internationally as well. Really thinking about how do we advance the foundational components that ultimately allow quality to move from episodic improvement to sustained system performance. And a lot of the questions submitted in advance really ask the question around how do we get to this level of sustainability. And I would offer that a lot of the research shows that we need to really think more about these foundational structures because as I mentioned before ultimately systems produce the results that they are designed to get. So if we want better and more consistent outcomes we can't just rely on sort of these isolated interventions. We need to strengthen that underlying system, its incentives, and its capabilities. That's how we're going to move from reacting to variability to designing for liability.

00:13:31.360 April: So, next slide.

00:13:35.040 April: So, how do we really get there? Right?

00:13:36.560–00:15:08.399 April: Let's think about sort of this evolution of quality and safety. So, if we go to the next slide, when I think about quality and safety and again working in this field for 25 plus years, healthcare quality didn’t emerge fully formed. It's a little bit different in terms of how it's even emerged as a profession, right? And as a field. Healthcare quality really has evolved in response to complexity, risk, and rising expectations by consumers and patients for outcome, safety, and accountability. And so, for you know these last couple of decades improvement efforts have produced some progress but there's still a lot of inconsistency and variability across roles organizations and geographies. So, I share on this screen here a couple of different collaboratives because when I think back to the early days of quality and safety, I think about a lot of these clinical collaboratives that still exist today and how they have evolved over time and some of the promise of these clinical collaboratives that showed really really tremendous results. So, whether you're talking about the 5 million lives campaign or the 100,000 lives campaign by the Institute for Healthcare Improvement (IHI) or here in the United States there was this really big study called the Michigan Keystone study and you can see some of the other collaborative’s initiatives that were in the UK. A lot of these clinical collaboratives have showed tremendous promise and it's great right this idea that we should all learn and improve together and I think that's extremely important.

00:15:09.920–00:15:47.120 April: So many of these collaboratives, again mentioning things like that Michigan Keystone study in the US or the 5 million lives campaign, they showed some initial results, but what they also found was that even with the same sort of change package or interventions, there were often times variable results across organizations or across regions even when folks controlled for things like patient acuity or other factors. And so the question is why is that? If we're all doing the same thing, why don't we all get the same results within the same amount of time and then reach a point of sustainability and that's it.

00:15:49.279–00:16:49.920 April: And what the research might suggest here is that we need to think more about these sort of structural components as I mentioned before. So in the next slide, this gets us into this idea of organizational context. And hopefully this is something that you've seen or experienced in the literature when you've been looking at doing your own quality improvement work. But this idea around organizational context is that our organizational structures are different, right? Our cultures are different, our resources may be different, the incentives may be different. And so how do we get to the next level where it's not just improvement for some but improvement for all and not just short-lived improvement but sustained improvement? I think we need to think more about context. And so what do I exactly mean by context? So, there is this model for understanding success and quality improvement sometimes referred to as MUSIQ (Model for Understanding Success in Quality).  It's a paper that was written by Heather Kaplan et al.  probably 10 plus years ago. And in that article, they identify a number of contextual factors.

00:16:52.240–00:17:02.079 April: And these contextual factors are really sort of the environment, the organization, everything that is surrounding the improvement sort of intervention or initiative itself that could impact the success or the outcome.

00:17:05.280–00:17:10.559 April: And so, in this article, they identify a number of contextual factors such as knowledge, competency, and capability.

00:17:13.439–00:17:24.319 April: And I'll talk through those three specifically although there are a number of other contextual factors because I think they really play a role in what I would say is maybe the first step in bridging this gap. So capability for improvement at the micro system level.

00:17:27.119–00:17:36.080 April: Basically they describe whether a team or care setting has the knowledge the skills and the competencies as well as the experience to execute quality improvement effectively.

00:17:38.240–00:19:22.799 April:  They also describe as a contextual factor team skill or knowledge and so team QI skill is whether or not the quality improvement team possesses the technical competency and know-how to conduct improvement work and they also note workforce focus on QI and subject matter expertise that's beyond just the improvement team itself but that relates to organizational knowledge and expertise and supporting improvement so the research would suggest suggests that contextual factors can influence the outcomes that we're trying to achieve in our quality improvement work. They can be both facilitators, but they could also be barriers. And so really to truly improve, we need to understand how these factors can have an impact if you look on the next slide from the micro system level all the way to the macro system level and then across all of our different stages of improvement. As you can see here in this graphic from Coles, Anderson et al. that it looks at what's happening at the micro system level through more of the organizational level through maybe the country or sort of policy level and these different contextual factors such as technical infrastructure or readiness for change or culture competency, skills and capability, leadership that these may have an impact either positive or negative on the success of our quality improvement efforts. The amount of impact may vary depending on where an improvement effort is in its course. Is it more in the early sort of pre-planning phase or how does this have an impact as we get closer to sustainability.

00:19:24.320–00:20:22.720 April: So, as we move to the next slide, that's really where we get to okay, we know the history, we understand the problem. How do we get to this level of sustainability? And although there are lots of contextual factors and lots of things that impact sort of that structure piece of structure, process outcome that I think is so critical, I would say that it really starts first with the workforce sustainability and getting to where we want in QI and bridging that gap so we can have less sort of peaks and valleys and variable improvement relies on the workforce. So as I described before, you know, music discusses capability for improvement, team QI skill and workforce focus on QI as really important contextual factors. So if you look at the next slide, sustainable quality really depends on our people, our staff who not only care about improvement but are actually prepared to do the work.

00:20:24.640–00:21:03.120 April: And I would sort of ask you all to sort of think about your own organizations or the work that you're doing and how much do you think that your teams, your staff are really prepared to do some of the quality improvement work? Do they have the skills and competencies to be able to not only carry that work out but sustain it once the formal improvement intervention or project is actually complete? So, this is where I share with you NAHQ, the National Association for Healthcare Quality’s competency framework. This competency framework has been in development for years. It's been refined. I think it's only the second or third iteration at this point.

00:21:05.440–00:21:54.400 April: It has eight domains, 28 competencies, and hundreds of skills that describe what professionals, not just healthcare quality professionals, but all professionals working in healthcare need to be able to know and do in modern health care. As we think about getting to a level of sustained improvement and these competencies apply universally across the care continuum, the difference in some cases is going to be the scope and the depth but not in the actual definition of quality. And so what I would propose here is that by standardizing competencies, organizations can move from sort of the individual heroics to more systemwide capability. And if we think back to that MUSIQ framework and their hypothesis, this is critical to not only achieving initial success, but really getting to a level of sustainability.

00:21:57.600–00:22:26.880 April: On the next slide, NAHQ's taken a step further. So, as I mentioned, they developed this healthcare quality framework. And in developing that framework, they start off by then testing it out and doing some workforce development reports. They started initially with quality professionals and what they found is that there was a lot of variability in performance not just across organizations but within organizations with respect to how folks were performing as related to these healthcare quality competencies.

00:22:29.360–00:24:00.720 April: From there they undertook a project to begin to sort of build out the workforce. To workforce development starting with healthcare quality professionals but then expanding that outside to all health care professionals and that's really important because when we think back to this idea about building organizational capability and capacity it can't just be quality professionals alone there will never be enough healthcare quality professionals to do all of the work of healthcare quality but starting first with what are the competencies let's get this right within the folks that day by day or should really charge to do the work and then let's look at how to make it sustainable by broadening that impact and really bringing these competencies up down and across the organization. So, in this report that just came out I want to say last year maybe last fall took these competencies they worked with a subset of health care systems within the United States to really do some baseline measurement in these competencies embed workforce development education and then continue to measure and monitor not just how folks were performing with respect to these competencies but then what was the actual ual sort of ROI, right? That return on investment, tangible outcomes and areas of interest across these organizations. And you can see that on this slide that there were improvements in things like cost avoidance. There were improvements in pay for performance programs which are really common here in the United States.

00:24:02.640–00:25:15.600 April: There's a reduction in patient harms, right? So serious safety events. And in the US, there's different quality ratings programs. You know, I showed some of the earlier slides that were sort of international on on some of the very specific quality rating programs within the United States. There was also an improvement in what we call the star ratings. Like think about a fivestar sort of Google rating for example. But this was one of the first reports that really was able to tie together workforce development competency focus not just with limited short-term improvement but with long-term improvement. Because for many of these organizations that are shown here on these slides, the work started at this point maybe four or five years ago and the improvements have sustained or even continue to improve over time. So these tools related to workforce competencies are intentionally designed to work across roles, care settings and systems because ultimately this work of quality is universal. Right? even when the context differs the work of the classic sort of competencies should really be that level playing field across organizations.

00:25:17.919 April: So let's go to the next slide.

00:25:20.960–00:26:02.400 April: So what are we trying to really get to when we think about where we've been in quality and safety? I'm just going to say over the last 25 years because here in the United States about 20 - 25 years ago there were some critical publications that really highlighted the quality improvement and patient safety challenges. Some of the critical foundational books and articles related to Err is Human or crossing the quality chasm that were put out in the US by the Institute of Medicine at that point in time really laid the foundation for two plus decades of focus on quality improvement and patient safety.

00:26:04.880–00:26:19.360 April: But just a couple of years ago, there were some international articles that really pointed to it 20 - 25 years later and although there have been starts and stops in terms of improvement, that improvement hasn't been sustained. And why is that?

00:26:21.760–00:26:34.960 April: And what I would suggest is that we need to shift from healthcare quality as just this sort of peripheral activity to a core organizational competency and really getting back to the basics, right? That structure, process, outcome.

00:26:37.200–00:26:44.880 April: In order to get to outcomes, we absolutely   need to focus on process. But critical to our processes working is really the infrastructure that we have.

00:26:47.440–00:27:18.880 April: And a key component of that infrastructure that's helping us to build organizational capability and capacity is going to be workforce competencies. So when I think about the work that we've done in the US and trust me there's a lot more work to do and where this extends to an international audience is that international health care systems operate under varied regulatory structures, varied resource constraints, different workforce models and cultural expectations. But our drivers for improvement are all the same, right? We want patient safety. We want trustworthy outcomes. We want workforce stability and sustainability.

00:27:22.720–00:28:56.399 April: And a competency based approach can be especially valuable internationally because it transcends job titles. It transcends national role definitions and creates a shared professional language that supports workforce mobility and development across borders. So although context may differ, confidence travels and we can ultimately get there. If you look at the next slide, my three main takeaways is going back to the basics really focusing in on structures that will help us to support sustained improvement moving beyond a single institution. So intraorganizational learning and collaboratives to me really are the way to go. Thinking back to some of those collaboratives that I showed on an earlier slide, we did see lots of promising improvement when we were all working together. So increased collaboration, not competition. The challenge was how do we get to a level of sustainability and that is really tied to looking at what are the structures that we have in place. So, once we move beyond that intervention or project, the work that we've done doesn't change with that the next iteration of staff to come through because we've ultimately built that stable foundational structure. So, the takeaway message before we move on to questions, I think I have about 2 minutes before we move to Q&A is that quality is universal competencies and competence is what is going to make it sustainable and education is what's going to make it possible for us to get there.

00:28:59.279–00:29:13.520 April: All right I think I've taken up just about my 30 minutes. There were a ton of questions that came in before the session and hopefully some, while we were chatting here and so really wanted to spend some time engaging a bit more with the the audience.

00:29:16.720–00:31:30.480 Tom: Thank you very much, April. A really, really thought-provoking discussion presentation. You're absolutely right, we had a lot of questions that came in from those of you registering for the talk today. So, I'm going to come to some of those questions., and, also invite, those of you listening, if you have questions, that are in your head now or that come to mind as we go through the, the Q&A, then I hope you're able to add those into the, the Q&A function. You may need to dig around a little bit deeper on your screen to find that but if you have a question, you can put it there. I think April before I come to our questions that have been submitted I'm going to take the liberty of putting one of my own to you coming from what you said you put a lot of emphasis on context and that absolutely chimes with my own experience in quality improvement in healthcare and within context you then highlighted workforce you know within that I know the model you mean the MUSIQ model quite a complex looking model it always reminds me of a kind of metro map of a of a city there are lots of lines moving around it it does illustrate the fact that that it's a complex system but within that you highlighted workforce as the area that you would you would focus on I wonder if you were to pick one other aspect of context as either the next most important or perhaps the one that would be your pick for a complement to the workforce. You know, let's say you've you've kind of got the workforce piece going into place. What's the the other thing that you would highlight from context as most enabling sustainable improvement in quality?

00:31:32.880–00:32:19.279 April: So I'll answer your question in two different ways because I think the first answer actually still does tie to workforce but workforce a little bit differently, you know, my focus really that I highlighted today was more on workforce development competencies, but the other piece of workforce that I find within context is more around sort of staffing and resources and understanding what might be different between different units, different organizations, different districts. So, to give you a very concrete example, I was working on a project related to sepsis. I think sepsis is like a global worldwide issue, right? Like the many organizations have. And on the specific project related to sepsis, we were doing work within one of our emergency departments.

00:32:20.799–00:32:23.120 April: And one of the emergency departments was a large academic center, so 1,200 beds.

00:32:26.640–00:32:36.240 April: And then several of the other hospitals that we were also working with as part of a collaborative were much smaller. So, they were more community- based organizations some had as few as 100 beds, others maybe two or 300.

00:32:39.279–00:33:08.000 April: And what we ultimately saw is that the smaller organizations were improving a lot faster, right, than the organizations that were larger. And the question is why was that, right? What was it about sort of the organizational or environmental context that maybe was different? Because the interventions were the same, right? We were doing the same intervention at each location. So it wasn't the intervention itself. And we even tested, you know, is there something about the fidelity, right, of the implementation of that intervention. Were we doing it the same?

00:33:09.679–00:34:32.079 April: Did it vary? Nope. Everyone's doing the exact same thing. But what it came down to as thinking about sort of the sort of staffing and sort of resource component is that in our larger organization, we had a lot more trainees. So we had a lot of different folks that were coming through who were learners in the organization. So the staffing wasn't as consistent. and the smaller sort of community organization where they knew each other, sort of rallied around. And so that ended up being an important contextual factor or difference. Now, that doesn't mean that the intervention wasn't going to work, but what we needed to do was to think a little bit differently around how could we make it work in that setting where the staff were different. The staff were more transient than in some of these smaller organizations where they didn't have these learners, these trainees. And that is ultimately where we landed is that we would need to adapt a little bit our implementation approach and strategy to make it work for these more transient staff and learners in order to be able to get to that same level of sustainability. So still workforce related but workforce related in maybe a slightly different vein than what I mentioned. The other big piece I would say though if I were to focus on something outside of that it really is around leadership culture and incentives.

00:34:33.599–00:35:08.000 April: When you look at organizations that have re reached a level of improvement quickly and sustained improvement, it was often due to a leadership imperative and incentives around improving in a very specific area or very specific domain. That's where you know lots of health systems lots of countries have moved towards sort of these pay for performance systems. The challenge with the pay for performance systems where they shown some maybe initial improvement and then sort of like have topped out is usually twofold. One there end up being too many incentives.

00:35:09.680–00:35:16.640 April: So there were just too many priorities for organizations to focus on. And then it moved from being a system where they're going to be rewarded for performance to really a penalty system.

00:35:18.640–00:36:05.760 April: And when it became penalties for everyone that made it a lot more challenging to get to a level of improvement and sustainability. It also became more about the competition versus really working together. And when you think about the beauty of what worked for some of the more collaborative efforts, it really was around learning, growing, and changing culture together. And the organizations that were able to do that are the ones that not only saw the initial improvement, but really saw the sustained improvement over time. So, thinking about the incentives and how those are embedded in the culture and not a way where it is a one-time but something that is really ongoing, I think would be the next element within sort of that that contextual framework that I would focus on.

00:36:07.760–00:36:58.240 Tom: Thank you. Yeah, both of those sound like very significant factors alongside workforce capability like you say that the kind of difference in staffing structures. very interesting point, you made about trainees coming through as well and the kind of faster cycle time that those have. And then incentives and the leadership culture are so important but so hard to get right. Thank you. Let me come to some of the questions that we had coming in. No surprise some about in quality improvement. So, have you seen examples of worthwhile AI applications in QI? What use cases do you see for the future and what do we need to be wary of?

00:37:00.720–00:37:37.839 April: So I would say that right now when you look at most of  AI and where it's focused it's actually focused I would say less on quality improvement itself and more on the clinical applications right now the clinical applications of AI may absolutely have an impact on  quality improvement but not surprisingly  at least in in my frame of reference we're in the US I feel like our sort of operational environments tend to lag behind innovation compared to more of the medical sort of system and and research.

00:37:40.480–00:39:17.359 April: So in more academic settings there's a lot of innovation around AI and so where that usually docks in first into the healthcare enterprise is more on the clinical side right so we have a lot more clinical innovation AI adoption that is looking at diagnostic accuracy diagnostic safety  whether that's in things like radiology and pathology and how it can help assist our clinicians in speeding up time to diagnosis having better differential diagnosis and ultimately being more accurate right in diagnosis that's where a lot of the work in AI has focused so will that translate into impacts from a quality and safety perspective absolutely if we can speed up time to diagnose, if we can reduce waits and delays in care, if we can reduce some of the waste because we're not needing to do maybe as many  tests because we can have more specificity and sensitivity right at the beginning when I think about the applications in sort of quality improvement I haven't seen as many directly within I would say the quality improvement space again they're more on the clinical side where I've seen limited application from an improvement perspective is really starting to help more with measurement and there is a lot of measurement burden because there are so many things to measure within healthcare and within healthcare quality. So, some of the AI applications that are making it a lot easier to identify patient harms to collate some of that measurement so we don't need to do as much of it by hand.

00:39:19.119–00:39:35.119 April: I think that will ultimately be helpful in allowing a lot of folks that are working in quality improvement to do the true work of improvement instead of spending as much time doing data abstraction and analysis because that can maybe be more easily curated and pulled from out of our clinical records.

00:39:37.760–00:39:45.760 April: Whether you're talking about the clinical applications or applications in QI or applications more on the operation side, good governance across the board is going to be what's really important.

00:39:47.599–00:41:25.520 April: I think that there's a lot more governance systems that have emerged over I would say the last like two to three years. When you think back about five years when AI was really picking up, it was mostly a lot of startup groups, right? Very many startup groups that had very unique and sort of boutique products. And so that really became hard I think for organizations to know what organizations to trust, if their models were as well developed, if their models are going to continue to sort of grow over time. Would that company even still be in business a year or two later? And so ultimately, you know, a lot of organizations in healthcare, given that healthcare has a lot of embedded risk associated with it, they sort of had to put on the brakes, right, and really start to develop, more AI governance and really looking at what should we consider if we're bringing AI into the organization. So that way we can have parameters as it relates to quality and safety to ensure that's actually going to help us with care and not harm us, right? And sort of do the reverse of what it is that we're planning. So, as we look at applications that move more beyond clinical to operational and or data quality safety sort of abstraction, I think the same principles apply with good governance and multi-disiplinary groups. So when I say multi-disiplinary, yes, we need clinical folks, we need business folks, we need our research folks, we need our data folks all involved in that governance process to ensure that tools we're bringing into the organization have been fully vetted and that they'll continue to undergo evaluation over time.

00:41:27.520–00:42:16.480 Tom: Thank you. And and we've got a question that's come in which I I think would be good to put to you at this point. April, it's from Jad Abdelsattar a candidate for our Oxford MSC. My question is as healthcare increasingly integrate AI assisted workflows, do you think that quality improvement frameworks are sufficient current quality improvement frameworks are sufficient or do we need new competency domains specifically around human AI collaboration, oversight and governance? I suppose one thing it makes me think about April is that those domains that you showed us with your kind of hexagon of competencies in, you must be thinking about well how is AI going to be influencing what what we expect and what's required.

00:42:19.200–00:43:22.240 April: Absolutely. So that is a great question and definitely something that we've been thinking about the National Association for Healthcare Quality. And what you'll find though is that AI could work two ways, right? AI could be its own domain. But the reality is that AI crosses all the domains within the competency framework. So, if you were to look at the competency framework in current state, one of our domains is related to sort of regulatory and accreditation. There's a patient safety competency domain. There's a performance and process improvement domain. And so AI is actually layered right within all of those. So, I would actually anticipate instead of it just being called out as AI as a single bucket that AI is integrated across the competency framework across all the domains that currently exist. But that's something that yes, we're already thinking about how to incorporate and include that AI. One of the challenges is that it it is evolving at a very fast pace compared to anything that we've really seen in history.

00:43:24.400 Tom: Yeah.

00:43:26.079–00:43:58.240 Tom: Thank you. I'm going to move us on from AI. Just mindful of we've got we got other questions on other areas that people submitted in advance. So, I'll take us to the next area where there were a few questions and it was about quality improvement across silos. So how can we encourage cross specialty or cross institution collaboration and you spoke a lot about that to prevent effective applications of QI existing in in silos.

00:44:01.119–00:44:20.319 Tom: So how do we encourage that cross specialty or cross institution collaboration? And related to that, how do you see the trade-off between the added complexity and potential points of failure when you're doing that kind of collaboration versus the potential for tap tackling more systemic problems and and achieving greater scale?

00:44:25.280–00:45:04.240 April: That's a great question. There's probably a couple different ways to approach how to best do sort of these collaborative efforts, but it all depends on starting with the problem and how unique the problem is to a specific organization or location versus something that more folks are tackling globally. So going back to something like sepsis or you know we're doing a lot of work right now on pressure injuries because we have patients that are sitting in beds for a long time and they often get pressure injuries or or patient falls right. A lot of those we know are global problems that organizations across the world are all struggling with those same problems.

00:45:06.720–00:46:21.359 April: In addition to that we know that there are pockets where they've already had really great success. And so rather than coming up with novel new interventions on our own, the benefit of that cross collaboration is to identify how did you make it work or sort of stick in your environment where we've been trying the same interventions and you couldn't quite make it work. So that gets back to some of these contextual related factors. So, when you have quality problems or initiatives such as those where the problem itself is more universal, there are clearly identified and codified interventions that we know that work. But the challenge is really figuring out how to make it work more locally. That's where the collaboratives I think can best help in networking and benchmarking and working up down across these silos with your peers whether they're in a neighbouring hospital, a neighbouring district or ultimately in a neighbouring country and breaking down some of those competitive pieces. In that case, what you're really doing is moving from that that model of this has already worked in a couple of pockets and now we're just scaling and spreading, right? So, it's really about moving something that's worked in a small area to that scale and spread.

00:46:23.760–00:46:35.119 April: Where that may not always work is in two different cases. One, if there is an initiative or a problem where it's not as sort of universal or widespread.

00:46:37.200–00:47:14.079 April: So, it really is something that is very unique to a site or a location where that cross entity work is not going to necessarily have the same benefit, at least not yet. Right? So, you may not want to start with that approach out of the gate if there's not going to be enough value and sort of that cross organizational work. Where it's also probably going to be less effective is when there's not yet a willingness to really work together. So when you think about a lot of quality improvement work and a lot of quality improvement interventions, part of it is the buy in right around there being a challenge around there being something to solve.

00:47:15.839–00:48:10.079 April: And if organizations aren't ready, and this could be even within an organization, right? So if your units and wards aren't ready to actually accept that there is a problem to be able to work together, then you're sort of forcing them to work together instead of saying this particular unit, this particular hospital is a early mover, an early adopter. Well, let's let the early adopter go ahead and start to work on the project instead of making it a system or a countrywide project where there's so much lack of buy in that we just can't get the inertia to get started. In that case, maybe we do want to start with those couple of pilot areas, let them really sort of fine-tune the work and the interventions. Once now you can get some results and buy in, then we can move back to this idea of being able to scale and spread more broadly and not slow down those early movers or early adopters from getting some of the work done.

00:48:12.400–00:48:16.480 Tom: Thank you. That's really helpful pointers on when the conditions are there for a collaborative to work well.

00:48:20.000–00:49:33.200 Tom: You also got me thinking there about comparing if you like the incentives you were talking about earlier and that idea of the leadership saying you know this is what matters and we're going to have pay for performance or other incentives versus I think that the motivation you're tapping into often with the collaboratives is if you like it's more of an intrinsic motivation. It's it's the idea that amongst the clinicians and other healthcare workers that are doing the work, they have an intrinsic desire to see their patients getting the best possible care and if somebody else has cracked a problem. If somebody else has got something that works better then they just want to know that. So, there's there's something which we won't have time to explore now April about you know under what conditions do you use that more kind of leadership top-down driven approach to say hey as a whole organization this is what we're going after kind of like it or not right we're going after it versus you know we're a group of cardiologists or we're a group of elderly care nurses and and we're going to listen to other elderly care nurses or other cardiologist, so very thought provoking.

00:49:34.960–00:49:51.200 April: Well, I would say that's a great that's a great point. I know we have other questions to get to, but there are different cases where yes, that top down approach is going to work. And there's other cases where it's more of the bottom up. And sometimes they sort of meet in the middle, right? Where it's a more grassroots sort of micros system effort versus more of the macro system.

00:49:54.400–00:50:09.599 April: Ideally, these sort of macro in macro system incentives can help support some of that ground swall where it's like, man, we're really excited about doing this this thing for our patient population. And now we can tap into some additional incentives or resources to do it.

00:50:10.800 April: Yeah.

00:50:12.880–00:50:37.040 Tom: As you mentioned though, we we've got a couple of other areas to try and squeeze in, in our last 10 minutes. Workforce, so this is back to a theme that you did already talk about but the question was how can we make improvement at the heart of professional identity and not just professional identity the skill set and ethic for every clinician and manager.

00:50:40.640–00:53:16.319 April: So, this is a a challenging problem and one that I think we've been trying to tackle for quite some time with quality improvement as like a discipline. When you think about quality improvement and sort of patient safety and where it's grown up within health care, honestly, I would say it's still maybe a bit more in its infancy. When you think about the professions of nursing or medicine, they have a much longer history and they have an identity, right, that sort of surrounds them. How we got to sort of this discipline of quality and safety was in a much less direct way. I think I said earlier in the talk that there's been sort of this evolution and as a result of that this tends to be seen as this added activity versus a core set of competencies and domain that every health care professional right should really see as a core component of their role and a core component of their work. I think that really ties into that healthcare quality comp framework that sort of like hexagon as you mentioned before because the intention there is how do we really embed this as the core capacity and capability builder for all health care professionals and not just have it be it's for a specific subset or subgroup you know I sometimes in talks have a slide where I do show sort of the progression where if it's someone who is working specifically in quality and patient safety like where I've worked for 25 years I'm expecting that my level of knowledge is going to be different so I think about something like I don't know Bloom's taxonomy not only do I know it and can apply it but I can teach it right I'm not expecting most of our folks in healthcare to be there but do I expect them to have general recall ability right are they more sort of in that that domain of recall and application needing to get everyone there. So, I think what's been holding us back from getting there ultimately is that we haven't had really a codified framework because quality and safety has evolved over time. And so even if we were to go right now and then look online, there are so many different frameworks and education paradigms and things that are out there that for many organizations they have struggled with well where do I start and or I need to sort of create it myself.

00:53:18.960–00:53:22.640 April: So, you know, direct response to that question, one, we need to focus in on structure.

00:53:25.040–00:53:26.480 April: And I said part of that structure really is workforce and workforce development.

00:53:28.720–00:53:54.319 April: Having this framework and I think NAHQ has done a really great job of trying to put together something that could be used broadly internationally to say let's stop trying to like continue to like piece together multiple disparate things. Thinking about governance, let's have a standard. Have a framework and if we can leverage that that may help us to bridge this this gap that was mentioned in that question.

00:53:56.400–00:54:06.640 Tom: Yeah, makes makes a lot of sense.  I'm going to come to our last area return on investment. I I've been burnt by this one, so I'm interested.

00:54:10.240–00:54:26.880 Tom: How do you use standardized documenting quality improvement return on investment? And how important is this given that it can be difficult to do well? And how do those ROI calculations feed back on investment in and attention to quality improvement?

00:54:31.839–00:54:45.200 April: So, ROI is difficult. As much as I shared that there's that ROIQ report that NAHQ put together and they've been able to demonstrate ROI in some very specific use cases, I would propose that we can look at ROI two different ways.

00:54:47.280–00:55:08.079 April: Part of ROI is going to be local, right? And I would say that's probably the best way to start is looking at ROI locally. When you start to look at ROI from like a macro perspective that makes it a lot harder because there is not right now, I think those standard definitions around how to really define the return on investment for quality.

00:55:10.160–00:58:29.920 April: If you see one definition, you've seen one definition. And where you mentioned that, oh, you've been burned before, it's probably because if you were to go and try and calculate ROI across multiple groups for a large project and then put that out there in the literature, there's going to be a lot of folks, right, that are going to have different definitions around how to calculate it. They're going to poke holes in your approach. So I don't know that we are there yet from a sort of macroeconomic approach. Although certainly there are countries and districts that have tried that. If they have a public reporting system where maybe they are having organizations report in harms and then they can calculate a reduction in harms and tie that to this is how many more tests or bad days or something that the patient would have experienced had they had that harm, right? And so, they're doing it as sort of a cost avoidance. here's the cost of caring for a patient with an infection. If this many fewer patients get infections, here is really the cost of care that we avoided. Here's how many bed days we opened up that allowed access to care for other patients, right? But you're trying to tie that together at like a very sort of macro level. Where I think I would recommend that organizations start and where I think there's a lot more success in really starting at more the micro or meso system level. looking within your own organization because there it's going to be a little bit easier to tie the numbers together as real dollars with respect to improving quality and or preventing harm. For those of you that are working in the quality and improvement space given that my current role is the chief operating officer, I was a vice president for quality and safety. As a chief operating officer, yeah, I'm working with, you know, our our data daily on finance, productivity, you name it, often times I find that a lot of our quality improvement professionals and teams, they don't come to the leaders of the organization to say, What would ROI look like to you? How would you define it? And so just at sort of that micro meso system level, go to the leaders, right, that are running your organization and say what would outcomes look like that tie into ROI and how could we define those? And once you define those at a more sort of local level that allows you to now set up your measurement system and setting up a measurement system within your own organization or health system is going to be a lot easier than trying to extrapolate outcomes from multiple hospitals or multiple health systems on a macro level. So, for ROI it is important because it continues to sort of fuel the work. You want to be able to see outcomes and ideally tie it back to something tangible to the organization for ongoing investment. But I would start small and remember that when I say ROI, it's not always the finances, right? They're going to be these other intangibles or maybe it is very tangible, but these other things that we can also measure that going to show that return on investment that are really going to matter to patients and families and reputation. That is also something that you can document, but much easier to do it on the local level than doing it sort of on the broad scale.

00:58:32.000–00:58:38.160 Tom: I really like your suggestion of doing this at the start of a piece of improvement work rather than at the end.

00:58:40.559–00:59:10.480 Tom: And I can certainly think of cases where, you know, we've had presentations to a board at the end of a piece of improvement work.  and you can see that what they're really kind of thinking is okay, but where's the bottom line? Where where has this release savings that we can reinvest for? For example, whereas if we had at the beginning gone to those people and said, what is it that you would want to see? Because you're right, sometimes it is a financial impact. You know, the the the saying good quality costs less. Great.

00:59:12.720–00:59:47.839 Tom: Well, then let's see where the financial impact really is. But other times, you're right, they might say, if you've improved the patient experience, if you have dealt with some of the safety problems we're having, that's impact for us. We'll count that as return on investment, but I really like the idea of getting that from those people that will be kind of judging the outcomes at the beginning. Look, I know we're out of time. Olivia is back with us to close us out. April, thank you so much. I've certainly found it a really fascinating conversation, and Olivia, I'll hand over to you to just close off the session.

00:59:49.680–01:00:00.319 Olivia: Absolutely. And a huge thank you to you both for facilitating this discussion. I just am revisiting and I underlined multiple times April's quote, how do we achieve quality in an environment that is characterized as the raging river?

01:00:03.440–01:00:15.440 Olivia: And I think this was a really strong strong step to bridge that gap. So, thank you, I want to point you towards where you can get some more resources on this topic. So, first and foremost, we are going to be sending this recording as well as all of the links that you saw in this presentation.

01:00:17.839–01:00:52.160 Olivia: That includes the NAHQ framework in an email. So, if you've registered for this, you will be receiving that in your inbox shortly. If you are interested in more of these nonclinical concepts that clinicians need to be successful in their clinical careers, have a look at some of the resources from which this session emerged. As I mentioned, Tom and I are heavily involved in our masters in surgical science and practice, our PGCert in patient safety and quality improvement as well as a number of short courses, one of which is quality improvement. So do have a look at that and it will be sent to your inbox shortly.

01:00:53.760–01:01:15.920 Olivia: I want to also say that if you're interested in staying in the loop, you know, please do follow us on social media, you can visit our website, you can visit the NAHQ website when we send those resources out later. And if you're not already on our mailing list or maybe you want to sign up with a different email so that you get it straight to your main inbox, please go ahead and scan the QR code now. Otherwise, thank you all for joining us and we will see you for our next Oxford Open Grand Rounds event.