Microsoft Build 2017 begins Wednesday. Many weeks ago we had the opportunity to host a webinar with Microsoft General Manager and producer of the Decoded Show Dave Mendlen on digital disruption and transformation, which we felt was more relevant today than ever before.
With Microsoft Build 2017 starting this week, we revisit our webinar from earlier this year with Microsoft Decoded Show executive producer Dave Mendlen, who was also a speechwriter for both Bill Gates and Steve Ballmer, about what disruptive technologies Microsoft has been working on ahead of the conference.
Here we discuss developments in conversational technology, machine learning, bots, and Mixed Reality, and how these technologies are transforming industries and societies.
We also get a glimpse of a secretive project that Microsoft is developing in the next 18 months that will change how we experience reality.
Conversations will be the new platform
One thing that Dave believes in strongly, echoing the words of Microsoft CEO Satya Nadella, is that conversations will be the new platform. Instead of typing on a computer keyboard or swiping your finger on a mobile device, interactive conversations between you the user and bots will be the platform of the future.
“We think conversations are going to be the new platform. This idea that you’ll be able to have conversations back-and-forth with a piece of software that will be able to help make decisions for you and with you will be a critical component of the next generation of computing,” says Mendlen.
Sometimes the conversation is between you and technology and sometimes the conversation is just between technologies as when bots communicate with each other or a back-end service.
Transformational User Experiences will change society
The way we use our technology in our computers hasn’t changed in a long period of time. Although smart phone technology came about some years ago, we still interact with them in basically the same way when they first came out.
One of the technologies that Mr. Dave Mendlen is most excited about and will transform user experience is Mixed Reality — something that he believes will become more mainstream on the not-too-distant horizon.
The difference between Mixed and Virtual Reality is that in Virtual Reality you only see the universe that was created by the software developer, and Mixed Reality allows you to interact with your actual, physical environment.
“The problem with Virtual Reality,” according to Mendlen, “is that it doesn’t know about your physical reality, so it’s very easy for you to wander around and walk into walls or to trip or hurt yourself.”
That is a problem Microsoft is trying to fix with Mixed Reality and one that should be on display at Microsoft Build 2017 — how to make the experience safer with technology that understands proximity by making Virtual Reality look like your real world.
That’s why Microsoft’s Mixed Reality headset HoloLens “is about bringing the world that’s out there and creating holograms into it, so it projects holograms into the headsets, and what you see or what you think you see, are these three-dimensional holograms that have been brought into the world that you see all around you.”
A secretive technology that Microsoft has in the works
On the set of Decoded
When I asked Dave if Virtual or Mixed Reality could one day be used for reporting from war zones, he touched upon a partnership Microsoft was working on, but he couldn’t give the exact specifics.
“It’s funny you say that,” said Dave, “because we’re working with a partner — one of my favorite partners that I can’t talk too much about, but this experience you just described is exactly what they’re trying to do. You can imagine that even at a concert, and by the way I think this will be coming in the next 18 months as we’re already working on it, but you can be at a concert and be able to experience and record not only the music, but also what you are seeing with a 360 degree view of what that experience is really like. So you can imagine bringing that experience back and being able to share it with friends and family.”
But this technology could be used for almost any experience from warfare to concert-going, or even something as mundane as a fender-bender. When trying to deal with an insurance company, you could simply share the experience exactly how it happened without bias.
Perhaps this will come forward at Microsoft Build? What other disruptions can Mixed Reality bring?
Mixed Reality will disrupt many industries
Mendlen believes that Mixed Reality will provide a unique perspective on the inner workings on how many industries conduct business. There will be almost 20 sessions on Mixed Reality at Microsoft Build, and here Dave lists a few industries it can disrupt.
Construction – Mixed Reality will allow you to fix things in dramatic ways that have never been done before. It would allow you to go to a building site and see things in a virtual way that would allow you to see potential problems and make improvements even before construction is underway.
Medical Field – Training doctors to operate on live organs using Mixed Reality as opposed to using cadavers can be revolutionary. Studying a virtual heart can enable a professional to be much more proficient in diagnosis. And it’s not just a rendering of a generic human heart, but rather taking a live heart and projecting it in 3D to make decisions in real time.
Media – Although the technology may be 15-20 years down the road, Mixed Reality along with Virtual Reality and Artificial Intelligence has the potential to present journalism in a way that the world has never seen. Reporting from a war zone, for example, is one thing, but adding a technology that makes you feel like you are actually there has the power to evoke human emotion such as empathy with the lives that are being reported upon. Additionally, AI one day may be able to even generate content, and who knows if a bot may one day win a Pulitzer Prize? The jury is out.
Legal Forensics – Anything from getting in a car wreck to police investigations into more serious crimes, having a Mixed Reality experience captured can allow for investigators to re-live the experience and draw conclusions.
There are many, many other industries that can be disrupted through Mixed Reality that are too many to list.
Conversational technology
On the set with Decoded host John Shewchuk
Going back to the theme of “Conversation as the New Platform,” we should probably talk a little bit about what bots are and how they operate as there will be over a dozen sessions on bots at Microsoft Build.
“When you think about training a bot, one of the ways that we do that is through a technology that Microsoft has called LUIS — Language Understanding Intelligent Service,” Dave explains.
LUIS aims to be the most comprehensive & easiest-to-use cloud-based service for conversational understanding for apps & devices that draws on technology for interactive machine learning & language understanding from Microsoft Research and Bing.
However, before someone starts having a conversation with a bot, that bot has to be trained, and that means a lot of back-end work on machine learning for developers to program these bots to “bring the best information forward,” as Mendlen says.
Back-end Machine Learning and programming
Machine learning, which will be a hot topic at Microsoft Build, is something that happens on the “server side,” according to the Decoded Show producer, and Microsoft Azure is one example of such a server that is based on the cloud and powered by machine learning.
Here’s what Dave had to say about the relationship between back-end machine learning.
“Let’s take this technology and enable it to learn on its own, and put that in the back-end for developers to take use of. If you are building an application, you can use that to do amazing things. They tend to be things that I’ll call back-end things or processing things that the user doesn’t necessarily see directly.”
A lot of the technology that is disrupting industries has some element of machine learning, where the machines can “learn” how to better optimize themselves.
As mentioned earlier, machine learning is mostly used by software developers behind the scenes, but the product of machine learning that we users interact with are the conversational bots.
Bots as the user interface to compliment back-end machine learning
This notion that conversation will be the new platform is taken very seriously at Microsoft. After developing a solid back-end using machine learning, it is now ready for that information to be made accessible to the user through a bot.
Bots are not only for conversation with us mortal humans, but they also “communicate” with other software on the back-end to make sure we receive the most relevant and useful information we are looking for.
“Bots on the other hand tend to be more about interfacing with the user, and oftentimes that conversation — that bot — is interacting with a back-end service,” according to Mendlen.
In other words, machine learning is how the machines can get better at the back-end to bring the best information forward, so that bots can communicate that to real people or other bots.
What to expect at Microsoft Build
The conversation between Dave and I touched upon many themes that Microsoft is currently working on and how these technologies will disrupt industries and transform societies.
This week’s Microsoft Build conference is sure to bring more of these ideas to the surface. All of the sessions can be found here.
The conference will run from May 10-12 and tickets have been sold out for months.
