Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

An in-house annotation platform, developed in Oxford by: Alan Aberdeen, Nasullah Khalid Alham, Clare Verrill, Jens Rittscher

The validation of quantitative image analysis algorithms as well as the development of strongly or weakly supervised machine learning methods depends on well-curated collections of annotated images. Especially the automated analysis of histology images requires the input of highly trained experts who are able to discern the high visual complexity and relevance of minute details in such images.

While a number of annotation systems have been developed, we propose an approach that structures the image annotation task into a set of well-defined assignments. Our server based annotation system allows the definition of such a prescribed annotation task for a given study. The user interface has been designed so that it is intuitive and does not require any special training. We pay special attention to the fact that human experts often disagree and provide a set of tools that allow highlighting of differences in annotation patterns and extracting a consensus from various experts.

Although the system has been designed for algorithm validation and method development we demonstrate the utility of the proposed approach in a specific annotation study.

The ‘’Immunoscore’’ is a concept developed in colorectal adenocarcinoma samples, that evaluates inflammatory cell infiltrates in CD3 and CD8 immunohistochemical stained sections at the invasive margin (IM) and centre of the tumour (CT).  Guidance is not given on the definition of the IM or CT.  The Immunoscore potentially provides additional prognostic information, above that provided by traditional TNM staging.

6 pathologists will be asked to assess 8 radical prostatectomy H&E images and draw the following regions of interest: tumour (centre of tumour - CT), tumour (invasive margin - IM) and non-tumour. They were also asked to annotate glands.  The annotations provided by individual pathologists will then be compared to one another in order to determine how the IM and CT had been interpreted.