A ’90s Pop Sensation and the iPhone App Powering Medical AI
Users of the iPhone app DiagnosUs typically spend their time identifying things like clumps of cancer cells in pathology slides and hemorrhages in brain CTs.
But sometimes they could use a break — especially if they’ve just watched hundreds of clips of colonoscopy videos. We compiled a dataset of images of the 1990s pop group 98 Degrees and created a DiagnosUs contest where users competed to be the best at distinguishing between Drew, Jeff, Justin, and Nick.
We left the contest open for 24 hours, with a first-place prize of $10 and another $10 split among the runners-up.
A day later, more than 200 users had competed. Together, they reviewed images of the singers a total of 49,315 times.
Usually, DiagnosUs serves a far more serious purpose. (The 98 Degrees dataset was initially just used internally by our team to test the app, but we had too much fun with it to keep it to ourselves.) Medical AI companies and researchers upload large datasets, and the app’s users compete to label them most accurately. AI learns from large datasets, which must have accurate labels to teach an algorithm what to look for. For instance, a skin lesion might be labeled benign or malignant.
DiagnosUs users compete against each other in a variety of contests, analyzing skin lesions or brain aneurysms or ultrasounds. The competitive format yields tens of thousands of labels per day — and our team at Centaur Labs aggregates the opinions of the highest-performing users. We deliver our results back to the owners of the datasets, so they can train their AI.
Though medical tasks tend to require a lot more skill than distinguishing between four faces, the same principles of collective intelligence are at play.
Drew or Jeff?
Even for those of us who remember the heyday of 98 Degrees, it may have been 20 years since we last laid eyes on the group. So it’s not uncommon to mistake Jeff for Drew, or Drew for Jeff, or Drew for Nick.
This particular DiagnosUs contest rewarded users based on how fast they could get 12 correct answers in a row, encouraging speed over sheer accuracy. (Most contests on DiagnosUs reward users for accuracy alone, so users can take their time and be as accurate as possible.)
Still, even while optimizing for speed, DiagnosUs users were overwhelmingly accurate. Justin Jeffre was the most easily distinguishable member, with DiagnosUs users correctly identifying him 88.8 percent of the time. Nick Lachey was correctly identified 86.9 percent of the time.
And a simple majority of the answers submitted through DiagnosUs matched the correct answer on every image.
Drew Lachey and Jeff Timmons were the most commonly mixed up. About 15 percent of the times that users were shown Jeff, they thought it was Drew, and about 13 percent of the times that they saw Drew, they thought it was Jeff. This surprised us, since Drew and Nick are brothers.
Estela Cárdenas, one of the top finishers in the DiagnosUs contest, admitted she wasn’t a big 98 Degrees fan before playing. “It’s just not the kind of music I listen to,” she said in an email to us. But the competition got the best of her, and Cárdenas played the contest more than 50 times — reviewing 698 images of the band — before securing her winning spot.
We asked Cárdenas what her secret was. How did she get so good at identifying these nearly-identical 1990s sweethearts? “I totally did Google research to get to know them because I was IN for the challenge,” she said. “I started seeing the images of the members over and over again, memorizing their faces…I had a great time improving my response time and getting to know the band.”
“Believe me, Justin, Jeff, Drew and Nick’s faces will never leave my brain,” she said.
Zach Rausnitz is a co-founder and CTO at Centaur Labs, and Kira Prentice is a product manager at Centaur Labs. Learn more about Centaur Labs at centaurlabs.com and download DiagnosUs free from the App Store.
