129 views
0

WHY THIS MATTERS IN BRIEF

Cameras that can see around corners are getting better, and the technology is beginning more commercially viable but there’s still a long way to go before you can take pictures around corners.

 

Building cameras that can see around corners is nothing new and over the past few years there have been a good number of attempts, by both MIT and the Chinese, but up until now none of those attempts have been able see around corners using just an ordinary smartphone setup, and while to many it might sound like something out of a science fiction movie that’s the idea behind a new algorithm out of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), and the discovery has implications for everything from emergency response to self-driving cars.

 

RELATED
Scientists grow the world's first human pig chimera

 

The CSAIL team’s imaging system, which can work with smartphone cameras, uses machine learning and algorithms to analyse information about the light reflecting back off of objects or people in a hidden scene in order to measure their speed and trajectory, all in real-time.

 

Can you see round corners?
 

To explain, imagine that you’re walking down an L-shaped hallway and there are some objects around the corner. Whether or not you can see it those objects reflect a small amount of light on the ground in your line of sight, think of this as the difference between seeing a strong shadow, depending on the strength and direction of the light source, a weak shadow, or, taking it one step further, nothing visible at all – in the latter case just because you can’t see anything doesn’t mean that there isn’t any light being reflected on the floor in front of you, it’s just too weak for your eyes to sense. These “fuzzy shadows” are called “Penumbra.”

 

RELATED
Windows are the perfect lens-less cameras for AI and VR headsets

 

Using a video of the penumbra, CSAIL’s new smartphone system, which the team has dubbed “CornerCameras,” can stitch together a series of one-dimensional images that reveal information about the objects around the corner.

“Even though those objects aren’t actually visible to the camera, we can look at how their movements affect the penumbra to determine where they are and where they’re going,” says Katherine Bouman, who was lead author on a new paper about the system, “in this way, we show that walls and other obstructions with edges can be exploited as naturally-occurring ‘cameras’ that reveal the hidden scenes beyond them.”

Bouman says that the ability to see around obstructions would be useful for many tasks, from firefighters finding people in burning buildings to drivers detecting pedestrians in their blind spots.

 

RELATED
Doctors will soon be able to track your health through walls using your wifi

 

Most approaches for seeing around obstacles involve special lasers. Specifically, researchers shine cameras on specific points that are visible to both the observable and hidden scene, and then measure how long it takes for the light to return. But, these so-called “time-of-flight cameras” are expensive and can easily get thrown off by ambient light, especially outdoors. In contrast, the CSAIL team’s technique doesn’t require actively projecting light into the space, and works in a wider range of indoor and outdoor environments and with off-the-shelf consumer cameras.

By viewing and analysing video of the penumbra, CornerCameras can produce a one dimensional image of the scene but 1D images aren’t especially helpful and can contain a lot of noisy data, but by observing the scene over several seconds the system can stitch together dozens of distinct images to determine an objects motion and trajectory.

 

RELATED
Mark Zuckerberg shows off Jarvis, the AI that runs his home

 

“The notion to even try to achieve this is innovative in and of itself, but getting it to work in practice shows both creativity and adeptness,” says Professor Marc Christensen, who serves as dean of the Lyle School of Engineering at Southern Methodist University and was not involved in the research, “this work is a significant step in the broader attempt to develop revolutionary imaging capabilities that are not limited to line-of-sight observation.”

The team was surprised to find that CornerCameras worked in a range of challenging situations, including weather conditions like rain.

“Given that the rain was literally changing the color of the ground, I figured that there was no way we’d be able to see subtle differences in light on the order of a tenth of a percent,” says Bouman, “but because the system integrates so much information across dozens of images, the effect of the raindrops averages out, and so you can see the movement of the objects even in the middle of all that activity.”

 

RELATED
The sixth Game of Thrones book is being written by an AI

 

The system still has some limitations. For obvious reasons, it doesn’t work if there’s no light in the scene, and can have issues if there’s low light in the hidden scene itself. It also can get tripped up if light conditions change, like if the scene is outdoors and clouds are constantly moving across the sun, and with smartphone quality cameras the signal also gets weaker as you get farther away from the corner.

The researchers plan to address some of these challenges in future papers, and will also try to get it to work while in motion. The team will soon be testing it on a wheelchair, with the goal of eventually adapting it for cars and other vehicles.

 

RELATED
Israeli Robo-Lawyer trashes top US lawyers in head to head competition

 

“If a little kid darts into the street, a driver might not be able to react in time,” says Bouman, “while we’re not there yet, a technology like this could one day be used to give drivers a few seconds of warning time and help in a lot of life-or-death situations.”

The work was supported, in part, by the DARPA REVEAL Program and will no doubt one day make its way out of the lab and into the real world.

About author

Matthew Griffin

Matthew Griffin, described as “The Adviser behind the Advisers” and a “Young Kurzweil,” is the founder and CEO of the 311 Institute, a global futures and deep futures consultancy working between the dates of 2020 to 2070, and is an award winning futurist, and author of “Codex of the Future.” Regularly featured in the global media, including AP, BBC, CNBC, Discovery, RT, and Viacom, Matthew’s ability to identify, track, and explain the impacts of hundreds of revolutionary emerging technologies on global culture, industry and society, is unparalleled. Recognised for the past six years as one of the world’s foremost futurists, innovation and strategy experts Matthew is an international speaker who helps governments, investors, multi-nationals and regulators around the world envision, build and lead an inclusive, sustainable future. A rare talent Matthew’s recent work includes mentoring Lunar XPrize teams, re-envisioning global education and training with the G20, and helping the world’s largest organisations envision and ideate the future of their products and services, industries, and countries. Matthew's clients include three Prime Ministers and several governments, including the G7, Accenture, Bain & Co, BCG, BOA, Blackrock, Bentley, Credit Suisse, Dell EMC, Dentons, Deloitte, Du Pont, E&Y, GEMS, HPE, Huawei, JPMorgan Chase, KPMG, McKinsey, PWC, Qualcomm, SAP, Samsung, Sopra Steria, UBS, and many more.

Your email address will not be published. Required fields are marked *