Matteo Shapira, Co-Founder & CXO of XTEND – Interview Series

Matteo Shapira, is the Co-Founder & CXO of XTEND, a revolutionary telepresence platform that enables humans to virtually extend themselves into action via a human-centric machine interface model. The model combines both humans and machines' best qualities to perform disruptive, complex tasks.

Could you begin by explaining how Xtended Reality (XR) differs or merges virtual, augmented, or mixed reality?

At XTEND, we allow the users to ‘teleport’ to a remote location. This telepresence isn't necessarily a ‘virtual reality’ but rather a real reality, where users can perform various professional or hobbyist activities. We refer to this as ‘XR’ or ‘XTENDED REALITY’ since it allows users to quickly and intuitively extend themselves into a remote location. We use Augmented Reality technology to present spatial 3D graphics on top of the user's view to enhance and empower the experience (i.e., an architect can see the building's design; a child can see Star-Wars-like shops integrated into their local environment; a soldier can see data about the battlefield)

Your previous startup, Replay Technologies, was eventually acquired by Intel. Could you describe the technology that you pioneered at Replay Technologies?

At Replay, we developed an Emmy-award winning Volumetric Video format & technology called ‘freeD.’ FreeD worked by capturing reality with three-dimensional voxels instead of two-dimensional pixels like regular video formats. The technology was used by most major sports leagues globally (NBA, NFL, MLB, Euro Soccer, US Open, and more) to create 360 replays – creating the illusion of a camera where there was no physical camera. Our beginnings at Replay inspired the concept of “telepresence” – since we could allow the user to ‘teleport’ themselves right inside the sports arena and run around with their favorite team players. Except that with XTEND, we do this via drones.

What is the genesis story behind XTEND?

XTEND was born out of Replay and later sold to Intel Sports when founders Aviv & Matteo decided to move on to new endeavors. The brothers met with Rubi Liani, Founder of the Israeli Drone Racing League (FRIL | DRL) and Adir Tubi (a defense specialist), where they discovered that in terms of U(X), drones resemble what computers were 30-40 years ago.  There was no easy way to experience this ‘reality’ with ease of use or a modern immersive experience; only ‘geeks’ could customize and operate them effectively.

XTEND set out to change all this. They decided that defense should be the first market to penetrate, helping soldiers, police, and first responders answer the call of duty. The technology allows said operators to perform their tasks via a flying machine, which drastically reduces the risk to human lives (*see the link to “before | after” clips of XTEND in various disruptive ecosystems). XTEND's first proof of concept (POC) was a system called SKYLORD HUNTER. It proved to be a game-changer in the field of aerial interception, as it allowed any soldier patrolling the southern Gaza border to easily intercept any threat that came from the sky (drones, incendiary balloons, etc.) efficiently and effectively.

How will the XR Platform enable a user to control a range of autonomous robots?

The idea behind XTEND is game-changing. Instead of having a completely automated “team” of autonomous robots performing completely independent tasks, we aim to integrate humans and machines to assimilate both entities' best advantages. Instead of replacing humans with teams of autonomous machines, we are developing a Human-Centric Machine Operating System that allows us to easily interface and ‘teleport’ into a remote reality and perform various functions. Part of this OS's core capability is to be able to ‘multi-presence’ – That is, control several machines simultaneously. For example: Think of a team of 10 fire-fighting drones reaching a disaster site, allowing a single human fire-fighter to easily switch between them to control the fire and rescue survivors. Each drone (or “node”) can have different skill/toolsets such as breaking a window, dropping a fire deterrent, communicating with victims, relay data inside a complex building, and more…

Can you discuss some machine learning or other technologies that facilitate the experiencing of remote realities?

We mainly use machine learning to assist the drones in ‘transparent’ and accurate environment agnostic navigation. For a drone to operate seamlessly and accurately in any environment (indoors, outdoors, light, dark, complex, simple), the machine needs to “teach itself” to behave accurately in unpredictable and dynamic situations. We are developing an assortment of algorithms to enable such accurate human-centric machine operability (where the person is agnostic to any limitations or operations of the machine – They are ‘transparent’ to them much in the same way that an Operating System masks all its internal processes from the user)

Additionally, our technology uses what we called ‘stacked AI layers,’ allowing drones to ‘service’ the human operator behind the scenes: From path prediction and collision avoidance to enabling ‘child nodes’ in a swarm team to follow the leader and patrol effectively.

XTEND’s initial product launch is SKYLORD, which assists the military and first responders. Can you discuss what SKYLORD is and what makes it so disruptive?

SKYLORD is a revolutionary platform. It allows any non-professional drone operator to easily and accurately use a toolset of drones. This technology effectively reduces the risk and loss of human life on the battlefield, as a physical presence is not required. Today drones are either 100% autonomous or very difficult for a user to operate effectively. XTEND’s technology makes using tactical drones a breeze. For example, why storm a house with terrorists if a drone can perform the same task as if the operator was right there physically? The technology is continuously evolving to maximize the potential of AI and SWARM based technologies to allow a single operator to effectively control a team of drones with various payloads and abilities with a human-centric versus a fully autonomous approach. 

What are some of the potential gaming applications of XTEND technology?

The potential is enormous. Today, if we bought a drone to fly around, it’s virtually useless, and to attain the Drone Racing League (DRL) excitement levels, it requires years of practice. We enable an XR experience “out-of-the-box,” allowing anyone to play exciting real life first-person-view games without needing to become proficient at operating a drone. This potential can enable new kinds of theme park rides and new ways to have casual fun outside – To new interactive remote tourism forms.

Is there anything else that you would like to share about XTEND?

The novel concept of combining humans with machines presents many exciting possibilities. Whether it's creating a rapid citywide first responder fleet or enabling the next generation of GoPro-like technology, XTEND's ultimate goal remains the same. We strive to construct a reality where each one of us could be anywhere, at any given moment, interacting fully with that remote environment.

Thank you for the interview, this sounds like it could potentially be game changing technology. Readers who wish to learn more should visit XTEND .