As leader of one of the two OFFSET integrator teams selected in October 2017, Raytheon has announced that they’ve created a Virtual Reality interface that allows a single user to control large groups of inexpensive unmanned vehicles like these Locust drone swarms that Raytheon and pals showed off a year or so ago. And Northrop Grumman, the other lead integrator, has also announced they’re “designing, developing and deploying an open architecture” for so called drone swarm technologies that use “game based architectures to enable design and integration of swarm tactics,” according to DARPA.
So far Raytheon has tested the technology to control swarms with as many as 50 drones and they plan on growing that number significantly over the coming months, according to Shane Clark, a scientist at Raytheon and principal investigator for the company’s OFFSET efforts.
“The goal is to allow a single user to actually control, in real time, up to hundreds of air and ground vehicles that have different capabilities, or are different models,” said Clark.
To manage the swarm Raytheon had to develop an entirely new VR interface and during tests the drones communicated with the “Swarm Tactician” using a laptop over Wi-Fi link, although the eventual plan is to make the technology communications platform agnostic. The tactician then interacted with drones and the simulated environment with a HTC Vive VR headset and a pair of controllers, which one day could be replaced by HTC’s brain controlled VR headset that I showed off a while ago.
“Right now, the data [the platform produces] is just used for real time decision-making, so there isn’t a storage component, though that could be added in the future,” Clark said. “And because interacting with hundreds of individual drones would be complicated, they’re managed in groups.”
Grouped drones can show their target area, given task, battery life and status of the communications link all seamlessly, and operators can drill down to see information on the individual drones too, added Clark.
The drone swarms themselves are designed to act as a mobile ad-hoc mesh network with each drone acting as a link connecting the entire swarm together.
At the moment Raytheon’s interface is currently capable of simple commands like selecting a subset of the swarm and tasking it to move to a particular area, or asking drones to spin in place to get a sustained view of the surrounding environment. Many of the drones will be outfitted with electro-optical cameras capable of image recognition, and others will be equipped with LiDAR to allow for 3D high resolution mapping and reconnaissance.
Raytheon is also working on a capability that would allow the operator to use the VR environment to draw around an area to be mapped, select the drones to complete the task and then issue a voice command to map the area, Clark said.
OFFSET is being conducted in a series of “sprints,” and groups of “sprinters” will be selected through the solicitation process to develop applications in several technology areas. The first sprint, released last year, addressed advancements in swarm tactics for a mixed swarm of 50 air and ground robots in an urban environment over a period of 15 to 30 minutes, and relevant capabilities included mapping abilities, locating entry and exit points, deploying sensor networks and maintaining connectivity for war fighters, according to the broad agency announcement.
Raytheon’s VR environment will be leveraged by different “sprinters” who will be using it along with the AirSim open source flight simulator developed by Microsoft to test the different tactics they’ve been chosen to develop.
“In simulation it’s easy to postulate a new sensor that gives you a particular capability and see how that might inform what sorts or tactics you could accomplish,” Clark said. “One of the examples they gave in the BAA was what if you had a camera that could see through walls, how would that change things?”
And speaking of cameras that see through walls, what about if there was one that could see round corners too… like this one.
“We’re really interested in understanding some of that more speculative technology,” he said, “so that we’re not just building a pile of parts that will become obsolete, but be forward looking.”
Matthew Griffin, described as “The Adviser behind the Advisers” and a “Young Kurzweil,” is the founder and CEO of the World Futures Forum and the 311 Institute, a global Futures and Deep Futures consultancy working between the dates of 2020 to 2070, and is an award winning futurist, and author of “Codex of the Future” series.
Regularly featured in the global media, including AP, BBC, Bloomberg, CNBC, Discovery, RT, Viacom, and WIRED, Matthew’s ability to identify, track, and explain the impacts of hundreds of revolutionary emerging technologies on global culture, industry and society, is unparalleled. Recognised for the past six years as one of the world’s foremost futurists, innovation and strategy experts Matthew is an international speaker who helps governments, investors, multi-nationals and regulators around the world envision, build and lead an inclusive, sustainable future.
A rare talent Matthew’s recent work includes mentoring Lunar XPrize teams, re-envisioning global education and training with the G20, and helping the world’s largest organisations envision and ideate the future of their products and services, industries, and countries.
Matthew's clients include three Prime Ministers and several governments, including the G7, Accenture, Aon, Bain & Co, BCG, Credit Suisse, Dell EMC, Dentons, Deloitte, E&Y, GEMS, Huawei, JPMorgan Chase, KPMG, Lego, McKinsey, PWC, Qualcomm, SAP, Samsung, Sopra Steria, T-Mobile, and many more.
FANATICALFUTURIST PODCAST! Hear about ALL the latest futures news and breakthroughs!SUBSCRIBE
1000's of articles about the exponential future, 1000's of pages of insights, 1000's of videos, and 100's of exponential technologies: Get The Email from 311, your no-nonsense briefing on all the biggest stories in exponential technology and science.