WHY THIS MATTERS IN BRIEF

Millions of people around the world go untreated because doctors can’t diagnose, or sometimes even see, their symptoms, but now AI can use photos to identify health conditions in seconds not days, weeks or years.

Patient number two was born to first-time parents, late 20s, white. The pregnancy was normal and the birth uncomplicated. But after a few months, it became clear something was wrong. The child had ear infection after ear infection and trouble breathing at night. He was small for his age, and by his fifth birthday, still hadn’t spoken. He started having seizures.

Brain MRIs, molecular analyses, basic genetic testing, scores of doctors; nothing turned up answers. With no further options, in 2015 his family decided to sequence their exomes, the portion of the genome that codes for proteins, to see if he had inherited a genetic disorder from his parents. A single variant showed up: ARID1B.

The mutation suggested he had a disease called Coffin-Siris syndrome. But Patient Number Two didn’t have that disease’s typical symptoms, like sparse scalp hair and incomplete pinky fingers. So, doctors, including Karen Gripp, who met with Two’s family to discuss the exome results, hadn’t really considered it.

Gripp was doubly surprised when she uploaded a photo of Two’s face to Face2Gene. The app, developed by the same programmers who taught Facebook to find your face in your friend’s photos, conducted millions of tiny calculations in rapid succession – how much slant in the eye? How narrow is that eyelid fissure? How low are the ears? Quantified, computed, and ranked to suggest the most probable syndromes associated with the facial phenotype. There’s even a heat map overlay on the photo that shows which the features are the most indicative match.

“In hindsight it was all clear to me,” says Gripp, who is chief of the Division of Medical Genetics at A.I. DuPont Hospital for Children in Delaware, and had been seeing the patient for years, “but it hadn’t been clear to anyone before.”

What had taken Patient Number Two’s doctors 16 years to find took Face2Gene, a new Artificial Intelligence (AI) program, just a few minutes.

An example of using AI to recognise health conditions

Face2Gene takes advantage of the fact that so many genetic conditions have a tell-tale “face” – a unique constellation of features that can provide clues to a potential diagnosis, and it is just one of several new technologies taking advantage of how quickly modern computers can analyze, sort, and find patterns across huge reams of data.

Built using a field of AI known as Deep Learning these new AI platforms look set to revolutionise medicine by helping doctors recognise and diagnose disease faster than ever before.

Genetic syndromes aren’t the only diagnoses that could get help from machine learning.

The RightEye GeoPref Autism Test can identify the early stages of autism in infants as young as 12 months – the crucial stages where early intervention can make a big difference. Unveiled earlier this year the technology uses infrared sensors test the child’s eye movement as they watch a split screen video – one side fills with people and faces, the other with moving geometric shapes. Children at that age should be much more attracted to faces than abstract objects, so the amount of time they look at each screen can indicate where on the autism spectrum a child might fall.

In validation studies done by the test’s inventor, San Diego State University researcher Karen Pierce the test correctly predicted autism spectrum disorder 86 percent of the time in more than 400 toddlers.

That said, it’s still pretty new, and hasn’t yet been approved by the FDA as a diagnostic tool.

“In terms of machine learning, it’s the simplest test we have,” says RightEye’s Chief Science Officer Melissa Hunfalvay, “but before this, it was just physician or parent observations that might lead to a diagnosis. And the problem with that is it hasn’t been quantifiable.”

Meanwhile elsewhere, a similar tool could help with early detection of America’s sixth leading cause of death – Alzheimer’s. Often, doctors don’t recognize physical symptoms in time to try any of the disease’s few existing interventions but machine learning hears what doctor’s can’t, signs of cognitive impairment in speech.

This is how Toronto-based Winterlight Labs is developing a tool to pick out hints of dementia in its very early stages. Co-founder Frank Rudzicz calls these clues “jitters,” and “shimmers,” high frequency wavelets only computers, not humans, can hear.

Winterlight’s tool is way more sensitive than the pencil and paper based tests doctor’s currently use to assess Alzheimer’s. Besides being crude, data-wise, those tests can’t be taken more than once every six months. Rudzicz’s tool can be used multiple times a week, which lets it track good days, bad days, and measure a patient’s cognitive functions over time. The product is still in beta, but is currently being piloted by medical professionals in Canada, the USA, and France.

If this all feels a little scarily sci-fi to you, it’s useful to remember that doctors have been trusting computers with your diagnoses for a long time. That’s because machines are much more sensitive at both detecting and analysing the many subtle indications that our bodies are misbehaving. For instance, without computers, Patient Number Two would never have been able to compare his exome to thousands of others, and find the genetic mutation marking him with Coffin-Siris syndrome.

But none of this makes doctors obsolete. Even Face2Gene, which, according to its inventors, can diagnose up to half of the 8,000 known genetic syndromes using facial patterns gleaned from the hundreds of thousands of images in its database, needs a doctor, like Karen Gripp, with enough experience to verify the results, so in that way, these new machine capabilities are an extension of what medicine has always been – a science that grows more powerful with every new piece of data.