The Wisdom of the Crowd in Medicine

The wisdom of the crowd relies on the diversity and collective intelligence of individuals.

Centaur Labs

Centaur Labs harnesses the power of the crowd and applies it to a crucial area of medicine: diagnostics. By aggregating the collective opinion of medical students, doctors, nurses, and others from around the world, Centaur Labs is capable of labeling medical images with a very high degree of accuracy.

How Accurate Is The Crowd?

In this post, we will take a look at how many users, on average, must review a case in order to have high assurance of accuracy using a dataset of skin images, each labeled as with or without psoriasis by a large number of DiagnosUs users. This project was undertaken in collaboration with LEO Pharma’s Innovation lab. The “correct answers” of the dataset are determined by a panel of 2–12 professional dermatologists.

This is an example image of skin with psoriasis, similar to those in the DiagnosUs app. Suppose that we have 30 people vote that this image looks like psoriasis, and 2 vote that it does not — in that case, we can say that the simple majority vote dictates that it is a psoriasis image.

An Interesting Mathematical Question

How many opinions do we need on a particular image in this dataset to get a reasonable assurance of accuracy?

We want to see how many ways there are to choose x of the m total objects of one variety, and the rest of the other variety, divided by the total number of ways to choose a subset of objects if we ignore the categories.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Centaur Labs

Centaur Labs

12 Followers

Centaur Labs (centaurlabs.com) is powering medical AI through accurate and scalable data labeling. On our app DiagnosUs, users compete to analyze medical data.