What you will learn?
I don’t work for a startup but I do work for Microsoft and we have a start-up culture within our organizations. I’m going to tell you a little bit about that, but I’m mainly going to spend a lot of time telling you about what I believe is going to be the new boom, and it’s this clash between the creative and the innovator and the technical. So on one hand you have the rise of AI, mixed reality and IoT, and on the other hand, you have the creative, the designer, the soul food, the maker, the artist. And it’s this collision that’s driving the next phase of innovation. So I’m going to spend a bit of time trying to convince you of this idea.
So we all know that there’s an invisible revolution coming. In the future, you’re not going to be able to buy a washing machine or a dryer, a toaster or a toothbrush or a hair dryer that doesn’t have IoT connected to it. And that’s because product designers know that they can get a whole lot more benefits out of having an IoT-connected device. And you start to see it already in our industry with people like child and elderly monitoring, and medication adherence, sports and fitness, smart appliances, sleep tracking, home security, leak detection. And to tell you a really good IoT story in a minute as well. Smart lighting.
I mean I was at home the other day, my daughter said why is my house so dumb, I want to talk to the lights. I’m like, geez! But the thing is that not only do you get the ability to have this new way of being conversational for example, but the predictive maintenance and this whole automating intelligent decision based actions based around it.
And IoT’s not just about putting a sensor in a device. It’s about connecting it back to the cloud. So I’ll spend a bit more time sharing a little bit more about that with you.
I did tell you I was going to tell you about the crazy things that we do at Microsoft and I was going to wear this this amazing cape, but we have a start at the world’s largest hackathon every year and thousands of employees get involved in it. And this year, my favorite one was this IoT solution which was designed for hospitals and I actually have one of the creators in the audience. Bo Fluke, you’ll be able to talk to him afterwards.
But in hospitals, nurses spend 30% of their time looking for devices like they’re looking for that ECG or they’re looking for the wheelchair or they’re looking for the pump. And what they did is they used IoT, connected all the devices to an equipment tracking system, and then they were able to actually use a mobile app to find the devices they were looking for. And this is a complete transformation for what’s going on in hospitals today.
So it’s amazing what you can achieve in one day. Imagine what you could do with your new startups with IoT.
And another company that I really love, and they’re a Sydney company, and the product manager for Azure IoT actually met them in Barcelona because we get around as Australians. But they have this concept of the digital twin. So imagine you build a building and then later on you actually want to do stuff to anyone understand it and therefore they can actually turn all the plans and all the systems into a digital twin of that building. And then they start to add other different IoT services to it that really improve your employee, the experience that you’re having in the building. And this is, and you probably are, does anyone actually work in the building here? Anyone? No? Yeah. Do you use the bank, the app that comes with this whole, yeah, so they designed that and allows you to navigate around the building and know what’s going on, book rooms, etc. So it’s part of this whole emergence of the smart connected buildings.
So the other thing that’s going on at the same time as IoT is mixed reality. So what you’ll see here is a lady wearing a HoloLens. It’s actually a computer on your head. You don’t need a keyboard, you talk to it or you can use gestures. This reboots it just in case you have a go and you accidentally go like that, it reboots it. But you’ll see that this lady here is in Germany and she’s building a car at Volvo. This is a doctor who’s using it for surgery.
This is actually the biggest use case. What a lot of people find is that when they Skype, they have difficulty with being able to actually use their hands. So this is someone who’s trying to build something. They’re using the HoloLens and on the other side, someone’s showing them exactly how to build this lamp that they’re trying to build and this is a really big use case in the remote field service and a whole lot of blue collar stories and I’ll tell you a bit more about that.
And the other area that’s huge for the HoloLens is design. So this is actually an amazing architecture student in Melbourne who, one of the lecturers actually started wanting to teach. His whole idea was that architects build things that are too hard to construct. And that, you know, construction hasn’t changed for hundreds of years. So they wanted to use the HoloLens to actually make construction more digital.
So what she’s doing is she’s wearing the HoloLens and it was actually providing a digital grid so that she can build this really complex structure and they’re finding that some complex structures can be reduced in construction time by up to 90% and in accuracy and they are now a huge startup called Fologram. You can follow them on LinkedIn or Instagram. They’re traveling the world. They’re selling HoloLens by the bucket load because they actually come into your construction scenarios and help you to really reduce the time it takes to build complex objects.
It’s not just happening in construction and you know those different areas but this is a friend of mine, Mikaela Jade. She actually just won the Veuve Clicquot Woman of the Year award for her work in taking cultural heritage into a holographic environment. This is the first time ever she meets Namande who she used to look after when she was a ranger in the Kakadu National Park. And she managed to work with the indigenous, she’s indigenous herself, and put it into the HoloLens and now she can continue to maintain their cultural heritage and looking at doing it for a whole range of different other aboriginal insights.
So when we think about mixed reality, I think often people think about it as though it’s just VR, but it really isn’t just VR. It’s got a whole lot of stuff behind it. So what we call the intelligent Cloud so when you use a hollow lens, you can also do natural language interaction. You can do object recognition. You can overlay the real world into the mixed reality environment and start to do some really amazing stuff.
For example, these are some of the intelligent edge devices that we’re working in across a range of different industries. Drones, HoloLens, other kinds of mixed devices, mobile phones to be able to do things like the object recognition, the real world perception and the cross device collaboration.
So the same time as mixed reality, IoT, you have AI. So this is a guy who’s blind. He’s a blind engineer who works at Microsoft and he developed the seeing AI software and what it does is it actually uses cognitive services. To take photos of what you see around you and tell you and give you audio description of what’s going on. So he’s in a meeting and he can tell the age of the people in the meeting and it can tell the gender. And this is my friend Kenny who also works at Microsoft and is very low sighted but he uses it to read labels, etc.
And you can actually download this app on your iPhone if you have it. It’s called Seeing AI and start playing with it. My favorite party trick on a Saturday night. You can start finding out how old your friends really are. And just got to get a feel of the kind of things that cognitive services can do.
Uber’s using it. For example, in the US, they got a trial at the moment when the biggest issues with Uber drivers is that they’re not the actual person driving the car that’s licensed to drive the car. So this uses facial recognition to ensure that you are the driver of that car before you start your session.
And you can also use it for emotions. So this is me using the emotion photo booth challenge. Imagine if you actually could build some kind of emotional awareness into your software. So it, so there’s a whole emotional component that uses the emotion API to understand your emotion.
So let’s talk about mixed reality in retail, really big in the whole idea of retail spatial planning, retail logistics and inventory management, creating virtual stores, consumer preference, heat maps so you know where people are going. A lot of people love this story. This is someone who’s collected all the pictures they love of their kitchen and put it on Pinterest and then they walked into Lowe’s which is like the Bunnings in America and it’s created your perfect kitchen based on your Pinterest. And not only has it created your perfect kitchen based on your Pinterest, it also allows you to modify components. So you might say I don’t like those tiles, I want a higher bench or a different bench color. And when you’re done, it provides a whole lot of analytics back to your salespeople to say which bits did you like and which bits you didn’t like based on your dwell time and your sentiment analysis. So what a new way to go shopping.
Manufacturing is a really big area and that’s where the digital twin is really taking off where you’re able to get a deeper understanding of what’s going on in your manufacturing environment and do start to do things like remote collaboration, which I was talking about previously.
This is an amazing story of a telecommunication company in Canada. They were wasting a lot of money digging up the ground. I don’t know how often you’ve seen Telstra digging up the ground. They can’t find things. So what they’ve got is a digital over layers, actually what’s going on in the ground? I’ll put the HoloLens on and actually digitally tell them where to dig and where to find the components. Totally changing their business model.
And the final area for mixed reality as you know as an architecture and construction engineering, but just a whole new area where you can start to have more immersive workplace safety, contextual maintenance and repairs, you start to bring in a lot of that predictive maintenance and those kind of things.
Okay, so we talked about AI, we talked about mixed reality. Let’s talk about speech.
So just last, I think just two months ago, we, there was a human, there was a study done by Stanford that showed that we have reached human parity. Computers now understand language better than people. So what does that mean? So it means that we’re going to see a lot more conversational activity going on.
This is for example Skype. This is a guy from Bing, we all love Bing. He’s actually getting training but it’s being done in Sydney. He speaks Chinese, but he, you can speak English and he’ll hear it in Chinese. So why not put these services into your software like, you know, there are cognitive services that you can use into your apps.
This is Xiaoice, she has 40 million friends in China. She’s an intelligent bot. She has EQ, she remembers your conversations. The average conversation with Xiaoice is up to two hours. She’s so popular that she even reads the news weekly on the big Chinese TV station, and we’ve now launched her in Japan and she’s got 29 million friends and it’s the technology that we’re using to dramatically improve our contact centers.
So if you are using the, if you’re using Microsoft Help, you can start talking to a virtual agent and that virtual agent will triage and remember and help you and we’re getting so much more satisfaction from that than call-in people. So I think that’s way more human. Get an answer instantly instead of being on hold and it’s this kind of learning that’s really helping us to develop that.
So you can start using the bot framework today. I have how many of you have played with bots? Yeah. We got a couple. Yeah, I call this the bot framework the executive bot framework because it’s really easy to use. Did you find that? Relatively? So, you know in a couple of days, I’ve Ryan that was here before, that was his first project. You know in a couple of days you can put a FAQ in and you can start having, answering questions. But what a great way to start learning about the stuff that everybody is going to want their own digital assistant.
And you can wrap all of these Services up into the cognitive services APIs. So not only do you get language also get the ability to start to use enterprise search. Also use speech, machine learning, vision, the whole ability to recognize objects and the search components.
So here’s some more ones you can play with if you’re bored. CaptionBot, you can start sending pictures to it. You can start playing with ProjectMurphy.net. So you send up pictures and it’ll like change. You know, that’s me as Madonna. You might have your own combination that you come up with. You know, what if I looked like Rocky, I don’t know, and then you can start talking to our own English bot as well. So in the same way we have Ziaoice, you can still talk to Zo on messenger and she’s a bot that remembers your conversations.
So, where’s it all going? So we all kind of know that, I know back in the old days when I worked for Telstra, I worked there before Frank, they had an internet fridge and you could go up to the Internet fridge and you can surf the Internet on the fridge. I thought that was pretty lame. But this is what I call an internet fridge so you can use object recognition and tell you what’s in your fridge and where in the fridge it is. And is this kind of technology that we’re starting to use to drive things like autonomous vehicles, autonomous cars, all those all those kind of object recognition and spatial at the same time.
So Microsoft also really really focused on taking AI into new places. So we’ve got AI for Earth where we’re working really hard to solve some of the most biggest challenges around climate change, AI for genomics, we’re looking at ways that we can move the organism into an algorithm and start to really solve some of the problems around health as well as intelligent healthcare, smart agriculture, and we just announced AI for accessibility which is a really big part of what Microsoft center around diversity and inclusion. And you probably saw that in our Seeing AI product but you know, this is actually a grant that’s open to anyone in the world who wants to use AI to make more inclusive software. Have a look at it and just Google AI for Accessibility. It’s a great way to be able to amplify human capability.
So I’m going to invite you guys to go and visit the Microsoft for startup stand. You’ll be able to talk to Ben who built that amazing app in the hospitals and you’ll also be able to sign up. So Microsoft actually got a pretty cool startup offer where if you sign up you get $200 of free Azure credits and a whole range of other development support. So you can start to use some of these cognitive services, also start to use the Azure IoT platform. You know, you might start download and an emulator for the HoloLens and start to see what’s possible with that you know. You might even buy the device, they’re only $2000. You never know. So a lot of these things are really available to startups today and you can make a massive impact in this space. And there’s also the Premium next level offer, which I’m not going to explain but I think it’s more about you know, really getting to know that platform and then being able to connect with a Microsoft person and build that connection.
And so you just go to startups.microsoft.com and off you go. So that’s my last slide. I’m actually going to leave 3 minutes for questions because I know sometimes I speak very fast and there might be some questions. So did anybody want to ask any questions?
Man from audience:
Yeah, quick question about IoT. It’s becoming more and more popular. However, read a little while back, there’s some pretty strong criticisms of the MDM for example not being able to facilitate the increase in traffic to happen from everybody having IoT devices throughout the home. Just want to get your opinion and view on that if you have one.
Yeah, I kind of do. You know everyone loves MDM bashing? I don’t even have the MDM. But the way the devices work is that they send really low amounts of data. So some of them don’t have to pull all day long. So they can pull one, you know and every hour that kind of stuff. So I think it’s about learning how to use the devices. They’re like, you know, I’m working with a company that’s putting sensors in every single chair in an office. Because they want to know occupancy of chairs, but those will only poll when someone sits on it and gets off it. So yeah, it really depends on how you set it up. So I think we’re the first ones where the devices were like sending data all the time. Yeah, but it’s about kind of trying to figure out. And there’s also different types of devices that will do different types of sensing. So there’s ones that will do water, one that’ll do temperature or they’ll do light, they’ll do motion. So, yeah, it’s all, I don’t think it’s going to be that bad. I’m not scared.
Man from audience:
Yeah, it’s me. You said the HoloLens is 2000? Okay, and how much would it work, will it cost to build…
It’s a very approximate price. It’s probably got a few numbers at the end.
Man from audience:
To build the VR environment and twins?
Well on average, you know those architect guys did it in a couple of months, so they connected, so it depends what you’re trying to do. So one of the great things about the HoloLens is it scans the room. So in the old days when you were building VR things you had to like figure out how to get the data into the VR environment whereas if you wear the HoloLens, it will scan the room. And so that will be one of the beginnings, so that will put it into that space and then you can start building things within that. So yeah, it’s a really hard thing to put a figure on. I’m sorry. It’s like yeah, but there’s lots of great, I’ve got a lecturer at RMIT who’s launched the world’s first mixed reality course, so he’s developing a lot of content. There’s also a lot of content on dev to hololens.com so you can start reading about it. As I said, you can download emulators. It’s all you need to use unity and visual studio. So if you can code it’s kind of fun to get getting into.
Okay. Thank you so much.