Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

In a correspondence to Nature Medicine, a team of Oxford-led academics describe upcoming new guidelines to improve the reporting of early clinical stage (or first-with-human) evaluation of decision support systems driven by artificial intelligence.

Red light with silhouette of a man

As an increasing number of clinical decision-support systems driven by artificial intelligence progress from development to implementation, better guidance on the reporting of human factors (ergonomics) and early-stage clinical evaluation is needed.

Therefore, Dr Baptiste Vasey and Professor Peter McCulloch from the Nuffield Department of Surgical Sciences, together with a steering group of experts in the fields of machine learning, human factors, and guidelines development from Oxford University and other universities in the UK and abroad, aim to develop new guidelines to improve the reporting of the early-stage clinical evaluation of AI-based decision support systems.

“We are convinced that human clinicians should and will remain at the centre of patient care, and therefore aiming to improve the way in which AI-based clinical decision support systems are evaluated when used to enhance rather than replace human intelligence. A critical phase of this process is when such systems are assessed when first used by clinicians in real life settings,” says Dr Vasey.

The article outlines a Delphi process, comprising two rounds of experts’ feedback and one consensus meeting, and the rationale for the new Developmental and Exploratory Clinical Investigation of Decision support systems based on Artificial Intelligence (DECIDE-AI) guidelines being developed. The authors hope to address some of the important issues hindering the translation from in silico algorithm development to clinical impact.

The current literature is lacking of clear recommendations for this stage of clinical algorithm development, which has led to low quality research and poor reporting, as well as a gap between the number of algorithms proposed in scientific publications and the number of algorithms actually impacting on patient care.

The focus of the DECIDE-AI guidelines will be on the algorithm’s actual performance when used by clinicians at small scale, its safety profile, the human factors evaluation, and collecting the algorithm’s characteristics necessary to inform the study design of larger-scale trials.

“The early clinical stage of evaluation takes place between the purely computational validation of an algorithm and its large-scale evaluation through clinical trials. It can be compared to phase I/II trials in the drugs development pathway or IDEAL stage 2a/2b studies for surgical innovation,” says Professor McCulloch.

A Delphi process was launched at the end of January 2021. The DECIDE-AI Steering Group welcome expression of interest from experts in related fields wishing to contribute. Please contact Dr Baptiste Vasey.

Read the correspondence to Nature Medicine 

Meet the NDS researchers

Similar stories

Life-changing results from DRWF Human Islet Isolation Facility at NDS

A new documentary film launched by the Diabetes Research and Wellness Foundation (DRWF) on World Diabetes Day 2022 features an interview with Professor Paul Johnson, as he describes life-changing results from the DRWF Human Islet Isolation Facility at the Nuffield Department of Surgical Sciences (NDS).

Dimitrios Doultsinos wins Young Investigator Award from the Prostate Cancer Foundation

Dr Dimitrios Doultsinos, a John Black Research Fellow in Prostate Oncology at Oxford University's Nuffield Department of Surgical Sciences (NDS), was one of this year’s recipients of the highly coveted Prostate Cancer Foundation (PCF) Young Investigator Award.

Oxford gets £122m funding for healthcare research

Health and care research in Oxford is to receive £122 million in government funding over the next five years to improve diagnosis, treatment and care for NHS patients.

Blog posts

Staff Census: We need you!

All Medical Sciences staff are invited to log into HR self-service and ensure their contact, diversity background and disability details are up to date. Between 6% and 27% of staff in the Division are missing their diversity data which makes it difficult to complete statutory reporting, understand or track our population changes and to plan actions around staff diversity and equality – make sure your data is complete today!

Oxford MedSci goes silver: 10 Years of Athena SWAN

The Medical Sciences Division is celebrating 10 years since its first Athena Swan bronze application, and the first year in which all 16 of its departments have achieved a silver award. The silver award recognises commitment to gender equality, understanding culture and context, and more. Read about our department’s hard work and innovation.

Lights, camera, action! My journey into video production

Dr Hannah McGivern provides a 'behind-the-scenes' account of her experience producing the video 'Journey of a QUOD Sample: Donating to Transplant Research', supported by the funds from the University of Oxford Public Engagement with Research (PER) Seed Fund.