How Mixed Reality is Transforming Collaborative Cancer Research

The ability to visualize big data in 3D and use that data to collaborate across disciplines through Mixed Reality is transforming cancer research.

As opposed to Virtual Reality (VR), Augmented or Mixed Reality (AR, MR) works by mapping holograms over real, physical space through the help of a head set. With Mixed Reality, you can still see and interact with your immediate environment instead of being absorbed completely in a simulated world like with VR.

According to Fortune, “Microsoft refers to AR as mixed reality, and it is still relatively new phenomenon that generated much interest after the popularity of last summer’s blockbuster mobile game,” Pokemon GO.

Mixed Reality for cross-discipline collaboration

However, when it comes to cancer studies in the medical field, the Microsoft HoloLens Mixed Reality headset is helping researchers across disciplines collaborate in real-time in order to come up with better cures faster.

According to Microsoft’s HoloLens for Research Proposal, “High-definition holograms integrated with your real world will unlock all-new ways to create, communicate, work, and play.”

The collaborative function of the HoloLens has garnered praise from those who work directly with it in the medical field.

“I think the thing that stands out with HoloLens, at least in our experience with it, is the ability to look both at the data and at the people at the right time,” said Alex Sigaras, research associate in computational biomedicine at the Weill Cornell Medicine in New York.

Mr. Sigaras’ research interest focuses on data visualization and interaction of healthcare and genomic data for cancer research and care using new emerging technologies including augmented and virtual reality.

In an interview with DECODED CHAT, Sigaras explained to Microsoft’s Christian Heilmann the importance of the HoloLens’ collaborative functionality as it allows computational biologists, oncologists, lab technicians, and physicians to visualize and analyze the data while voicing their own insights in real-time with other members of the team.

“I would say that each member has a key contribution to their field and to the team, and devices such as HoloLens allow us to keep everybody on the same page,” said Sigaras.

Mixed Reality for visualizing and interacting with gene sequences

Apart from its collaborative functionalities, Mixed Reality allows researchers to visualize huge amounts of data, such as a gene sequence, in a way that is neatly organized and compressed. For example, if researchers want to know what drug would work best with a problematic gene or to identify a mutated structure, this can be visualized through Mixed Reality to make those connections.

The video below demonstrates how researchers visualize the cancer drug network using the HoloLens. Towards the end of the clip, you can see how the researcher is interacting with red, holographic bubbles. Each one of these bubbles represents a cell containing 3.2 billion results of sequenced DNA. That’s a lot of data to process, but it is made accessible through Mixed Reality.

In January Sigaras gave a presentation on How HoloLens Transforms Healthcare at Microsoft Reactor.

Sigaras said that when it comes to researching cancer treatments, “You want to see how a drug connects to a gene,” and, “if you see in the sequencing that there is a gene that has a problem, and there’s a drug that targets it, that’s a Bingo! and I can do something with that to help our patient.”

All of that is now possible with the latest breakthroughs in Mixed Reality.

Where to keep up on the latest Mixed Reality cancer research

To find out more about how Sigaras and his colleagues are using Mixed Reality and the Microsoft HoloLens for cancer research, you can check out the DECODED CHAT video on the Microsoft DECODED Blog, the Weill Cornell Medicine YouTube channel that includes Sigara’s Microsoft Reactor presentation, and SlideShare for the slides from that same presentation.

For up-to-date news, studies, and reports, be sure to check out the Weill Cornell Medicine Sandra and Edward Meyer Cancer Center website.