Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

New reporting guidelines, jointly published in Nature Medicine and the BMJ by Oxford researchers, will ensure that early studies on using Artificial Intelligence (AI) to treat real patients will give researchers the information needed to develop AI systems safely and effectively.

© Shutterstock

Artificial Intelligence in medicine has shown promising results in numerous simulation studies, but very few AI systems have yet been used in patient care. But as a growing number of AI-driven clinical decision-support systems are progressing from development to implementation, better guidance on the reporting of human factors and early-stage clinical evaluation is needed.

Researchers from Oxford have led the development of a new guideline, DECIDE-AI, which aims to improve the reporting of research on AI systems when they are used for the first time in real clinical settings by doctors treating actual patients. It was developed based on opinions and feedback from over 150 experts across 18 countries, including computer scientists, clinicians, ethicists, patient representatives, and entrepreneurs. 

Baptiste Vasey of Oxford University’s Nuffield Department of Surgical Sciences (NDS) said: 'While AI has shown promising results for clinical application in simulations, very few AI systems have had any significant effect on patient care so far. A major problem has been a lack of robust scientific evidence to back widespread use in clinical practice.

‘This study is the first to clearly define the minimum reporting standards for the early stage evaluation of AI-based decision support systems in clinical settings.’

Professor Peter McCulloch, the senior author, also based at NDS, said: 'These guidelines take the principles developed for evaluating new surgical operations by the Oxford-based IDEAL Collaboration, and apply them to an important new field where guidance is lacking. The idea that innovation and evaluation should proceed in tandem at every stage in the development of new treatments applies just as much to AI as it does to surgery. 

Benjamin Hornsby, Project Coordinator for the IDEAL Collaboration at NDS, said: ‘DECIDE-AI is only the second reporting guideline dedicated to AI systems research, and the first ever to focus on the development stage of this technology. 

‘We chose to focus on early-stage evaluation results to emphasise the need for researchers to report transparently about four key aspects of AI research, to build confidence in the technology and highlight the practical benefits and improved efficiencies the technology can bring to healthcare.’

The factors DECIDE-AI evaluates are:

  • proof of clinical value in a live care environment
  • the AI system’s safety profile and potential risks for patients
  • human factors around the use of the system, such as how the AI system is integrated into the existing care pathway and how it influences clinicians’ behaviour
  • the steps needed to develop the larger-scale trials which will be needed to provide conclusive evidence necessary for an AI system to be accepted into routine patient care.

The full paper, ‘Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI’, can be read in Nature Medicine and the BMJ.

Similar stories

Life-changing results from DRWF Human Islet Isolation Facility at NDS

A new documentary film launched by the Diabetes Research and Wellness Foundation (DRWF) on World Diabetes Day 2022 features an interview with Professor Paul Johnson, as he describes life-changing results from the DRWF Human Islet Isolation Facility at the Nuffield Department of Surgical Sciences (NDS).

Dimitrios Doultsinos wins Young Investigator Award from the Prostate Cancer Foundation

Dr Dimitrios Doultsinos, a John Black Research Fellow in Prostate Oncology at Oxford University's Nuffield Department of Surgical Sciences (NDS), was one of this year’s recipients of the highly coveted Prostate Cancer Foundation (PCF) Young Investigator Award.

Oxford gets £122m funding for healthcare research

Health and care research in Oxford is to receive £122 million in government funding over the next five years to improve diagnosis, treatment and care for NHS patients.

Blog posts

Staff Census: We need you!

All Medical Sciences staff are invited to log into HR self-service and ensure their contact, diversity background and disability details are up to date. Between 6% and 27% of staff in the Division are missing their diversity data which makes it difficult to complete statutory reporting, understand or track our population changes and to plan actions around staff diversity and equality – make sure your data is complete today!

Oxford MedSci goes silver: 10 Years of Athena SWAN

The Medical Sciences Division is celebrating 10 years since its first Athena Swan bronze application, and the first year in which all 16 of its departments have achieved a silver award. The silver award recognises commitment to gender equality, understanding culture and context, and more. Read about our department’s hard work and innovation.

Lights, camera, action! My journey into video production

Dr Hannah McGivern provides a 'behind-the-scenes' account of her experience producing the video 'Journey of a QUOD Sample: Donating to Transplant Research', supported by the funds from the University of Oxford Public Engagement with Research (PER) Seed Fund.