Three-class ROC analysis--the equal error utility assumption and the optimality of three-class ROC surface using the ideal observer.

Full Abstract

Previously, we have developed a decision model for three-class receiver operating characteristic (ROC) analysis based on decision theory. The proposed decision model maximizes the expected decision utility under the assumption that incorrect decisions have equal utilities under the same hypothesis (equal error utility assumption). This assumption reduced the dimensionality of the "general" three-class ROC analysis and provided a practical figure-of-merit to evaluate the three-class task performance. However, it also limits the generality of the resulting model because the equal error utility assumption will not apply for all clinical three-class decision tasks. The goal of this study was to investigate the optimality of the proposed three-class decision model with respect to several other decision criteria. In particular, besides the maximum expected utility (MEU) criterion used in the previous study, we investigated the maximum-correctness (MC) (or minimum-error), maximum likelihood (ML), and Nyman-Pearson (N-P) criteria. We found that by making assumptions for both MEU and N-P criteria, all decision criteria lead to the previously-proposed three-class decision model. As a result, this model maximizes the expected utility under the equal error utility assumption, maximizes the probability of making correct decisions, satisfies the N-P criterion in the sense that it maximizes the sensitivity of one class given the sensitivities of the other two classes, and the resulting ROC surface contains the maximum likelihood decision operating point. While the proposed three-class ROC analysis model is not optimal in the general sense due to the use of the equal error utility assumption, the range of criteria for which it is optimal increases its applicability for evaluating and comparing a range of diagnostic systems.

Learn Faster Today Improve your study skills

Author information

Author/s: He, Xin (X); Frey, Eric C (EC);

Affiliation: Department of Radiology, Johns Hopkins School of Medicine, Baltimore, MD 21287, USA. xinhe(-atsign-)jhmi.edu

Grants: R01-EB000288 (Agency:NIBIB NIH HHS)

Journal and publication information

Publication Type: Evaluation Studies; Journal Article; Research Support, N.I.H., Extramural

Journal: IEEE transactions on medical imaging (IEEE Trans Med Imaging), published in United States. (Language: eng)

Reference: 2006-Aug; vol 25 (issue 8) : pp 979-86

Dates: Created 2006/08/09; Completed 2006/12/05; Revised 2007/12/03;

PMID: 16894992, status: MEDLINE (last retrieval date: 12/26/2008)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article (including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

Algorithms
Data Interpretation, Statistical
Image Enhancement - methods
Image Interpretation, Computer-Assisted - methods
Information Storage and Retrieval - methods

Information Storage and Retrieval - methods
Observer Variation
ROC Curve
Reproducibility of Results
Sensitivity and Specificity

These are the highest related articles currently in the database:

Random field model for integration of local information and global information. 30 Jul 2008
Restoration of DWI data using a Rician LMMSE estimator. 29 Sep 2008
A generic framework for internet-based interactive applications of high-resolution 3-D medical image data. 30 Aug 2008
Script-independent text line segmentation in freestyle handwritten documents. 30 Jul 2008
Extension ratio of depth of field by wavefront coding method. 16 Aug 2008
Spatio-spectral color filter array design for optimal image recovery. 29 Sep 2008
Concurrent computation of attribute filters on shared memory parallel machines. 29 Sep 2008
Design of multimodal dissimilarity spaces for retrieval of video documents. 30 Aug 2008
A wavefront reconstruction method for 3-D cylindrical subsurface radar imaging. 29 Sep 2008
Cost aggregation and occlusion handling with WLS in stereo matching. 30 Jul 2008

See 100+ related articles.