‘Hacking on Hololens’: Microsoft CSE Sync Week

At this year’s fourth annual Microsoft Sync Week in April, 644 people from 46 different countries hacked together in teams on unique customer challenges, including HoloLens collaborations.

Microsoft’s Sync Week is put on by its Commercial Software Engineering (CSE) team, which has set itself apart by working directly alongside its customers through a process called Open Hacks.

Open Hacks leverages direct evangelism, which puts customers at the heart of product development strategy.

Read More: Open Hacks: understanding customers by working alongside them

The hacking teams at this year’s CSE Sync Week leveraged the latest in Microsoft tech to take on 50-plus specially selected projects, each undertaken with the expectation of immediately driving value for third-party partners.

Among the technologies hacked were:

• Mixed Reality: Hacking on HoloLens to explore ways of harnessing the power of HoloLens’ in industrial environments
• Image recognition: Best practices for the use of an image recognition machine learning model for personally identifiable information
• WiFi Networks: Setting up high-quality Gigabit WiFi for 750 heavy network users and 1000-plus cutting-edge devices
• Basketball Performance: Hacking a machine learning algorithm designed to determine the optimal arc for shooting and dribbling a basketball, using sensors attached to players’ wrists and the ball itself.

“We send our own developers to work side by side with our customers to build technology that solves their biggest problems together. And in doing so, it has transformed the way we do business entirely,” Microsoft Technical Communications Director Dan Behrendt, wrote in a recent article on Entrepreneur.

Collaborating with Mixed Reality Using HoloLens

As opposed to Virtual Reality (VR), Augmented or Mixed Reality (AR, MR) works by mapping holograms over real, physical space through the help of a head set. With Mixed Reality, you can still see and interact with your immediate environment instead of being absorbed completely in a simulated world like with VR.

This technology is particularly useful when it comes to collaboration among teams whether the people are physically present together or not.

At CSE Sync Week, the teams leveraged the HoloLens on a Mixed Reality Remote Assistance project. A very common use case that Microsoft hears from partners is that when they are doing something on the HoloLens, they are able to look at something in their own space while somebody thousands of miles away can drop a 3D model in their space, and everyone can collaborate on the same project in real-time.

These types of collaboration using Mixed Reality were highlighted in Season Two of Microsoft’s DECODED Show on YouTube.

Read More: Microsoft DECODED Show Season 2 to kick off with special screening in SF

One team was transforming the world of medicine while another was changing the way we visualize the complex industries of oil and gas.

The Microsoft HoloLens Mixed Reality headset is also helping cancer researchers across disciplines collaborate in real-time in order to come up with better cures faster.

The collaborative function of the HoloLens has garnered praise from those who work directly with it in the medical field.

“I think the thing that stands out with HoloLens, at least in our experience with it, is the ability to look both at the data and at the people at the right time,” said Alex Sigaras, research associate in computational biomedicine at the Weill Cornell Medicine in New York.

Mr. Sigaras’ research interest focuses on data visualization and interaction of healthcare and genomic data for cancer research and care using new emerging technologies including augmented and virtual reality.

“I would say that each member has a key contribution to their field and to the team, and devices such as HoloLens allow us to keep everybody on the same page,” said Sigaras in an interview with DECODED CHAT.

Read More: How Mixed Reality is Transforming Collaborative Cancer Research

Microsoft CSE Team Projects

In April Arccos Golf teamed-up with Microsoft’s Commercial Software Engineering (CSE) team “to develop a novel method for rapidly pre-labeling training data for image segmentation models,” according to Clemens Wolff on the Microsoft Developer Blog.

Microsoft Software Development Engineer Margaret Meehan explained the results succinctly:

“We helped them develop a machine learning algorithm, so that a golfer can figure out their best course of action and how to get from tee to cup in the best possible way.”

“By making more training data available, we generalized Arccos’ deep learning model and thus improved the virtual caddie’s performance,” wrote Wolff.

Read More: Arccos Golf launches Coaches Dashboard with powerful visualization features

Another project led by the Microsoft CSE team involved making sense out of handwritten documents. Many documents are handwritten, and searching through them to find keywords and subjects of interest can be an extremely laborious task when done manually.

According to the Microsoft CSE team, “One of the great advantages of the shift from physical to digital documents is the fast and effective search and knowledge extraction methods now available. Gone are the days of reviewing documents line-by-line to find particular information.”

Data scientists Patty Ryan and Tingting Zhao note on the Microsoft Developer Blog that some of the main challenges in achieving acceptable extraction accuracy when applying traditional search and knowledge extraction methods to handwritten documents are poor document image quality and handwritten annotations.

The team; however, leveraged a service called Microsoft Cognitive Services that was able to save researchers “hours of review time” when applied to handwritten documents.