AI in Microsoft Office apps

February 20, 2020

– Coming up, we give you a roundup of the latest infusion of artificial intelligence in Office to find relevant content expertise as you search for information
on author in Office, up-level your presentations and documents through automated design
assistance in PowerPoint and the integration of mixed reality, uncover hidden insights in your data, focus your inbox and use Cortana at work to manage your calendar, and even use AI to
proactively protect yourself and your data at work. (techno music) I’m joined once again by Jack Elmore. Welcome back. – Good to be back, Jeremy, thanks. – So AI is really infusing everything now, everything from our device at home and things in our car, in the office, and we’ve been infusing AI, really, in our applications at
Microsoft for a while now under the covers. – Yeah, Office has been
focused on intelligent features for 20 plus years, and so what’s exciting is AI and the cloud-powered capabilities start to really unlock creativity, reason over large amounts of data, and do some fantastic things
that come alongside your work, help you create professional results without getting in the way. – Alright, so let’s make this real for the people watching at home. What can we do now as
an everyday Office user? – I’ve got a couple scenarios that I’ll show you today, Jeremy. First, I’m gonna start
with, well, my resume. As you know I’m looking for
my next opportunity here. And so I’m gonna pull up my resume and what do you notice
on the right-hand side? There’s the Resume Assistant coming up. It’s powered by LinkedIn.
– [Jeremy] So AI has detected that the resume is a resume file and opened that up? – [Jack] It’s done more than that. Detected that it’s a
resume and that the name matches the user name. So it’s mine. It knows that this is, I’m not looking at someone
else’s but this is my resume. And so it’s actually
captured my job title here and you’ll notice some
of Word’s Cloud-powered editor features, like for example, ‘think outside the box’
it’s suggesting you know maybe I should use ‘think freely’, instead of saying ‘young
blood’ maybe I should… so it’s helping me
improve my writing style that extends beyond just resumes. But the resume assistant is actually sorting out some similar resumes. Now LinkedIn has profiles on
over a half a billion people. – [Jeremy] It’s a lot of people. – [Jack] So you can start to
use AI to say, you know what, here are some resumes and ways
that others have described their work experience similar to yours. A lot of people suffer
from blank-page syndrome when they start working on a resume. It’s not something that we do everyday. – [Jeremy] Right, this will
really help people get started. – [Jack] Yeah so as I click into these and I say ‘Hey I wanna
see some more examples.’ I can filter those examples by different related disciplines, as I scroll down I get some
ideas like hey you could see additional examples on LinkedIn. And then if I wanna get really creative I could say, you know what, I wanna hire a professional to do this, frankly I just, kinda want somebody else to take care of it for me. So there’s a lot of great
information that’s available here right inside Microsoft Word. But as I scroll down a little bit further, what I can start to see is something really interesting around skills. So LinkedIn tracks a lot great
information on job postings and the way that the
winning job applicants are describing their skills and then translating that to titles. So here we can start to say, these are some skills that I might have, but I might not have
described them in a way that’s gonna generate the most hits, and create the most matches. So this is helping my resume to get seen. Articles on writing your resume, and even suggested jobs. So you’re in Seattle, we’ve got Starbucks, we’ve got, you know, Amazon, Time Inc. A lot of folks have got job postings and I can even with a single
click let recruiters know that I’m actually looking
for the next opportunity. So whether you’re brushing up your resume for your next job internally, or looking for your next opportunity outside the organization, Microsoft Word has a lot do to
help you publish your resume in a way that’s gonna
be exciting and engaging using the power of AI. – [Jeremy] Right, and
really what’s happening here behind the scenes is we’re
taking some of the things like search APIs and cognitive
services, we’re doing the look-ups against the information
that’s in LinkedIn, and then serving up those
results, so you’re able to really get some things that really match what you’re trying to do
in the Resume Assistant. And the nice thing is
when I open up my resume, it’s just going to just
directly detect it’s a resume and if I don’t find that, and maybe I’m starting from scratch, I can get straight to that
through the, which tab was it? – [Jack] So under Review,
you’ll see the Resume Assistant here on the far right. And that feature can be turned
on right under File, Options, ‘Show LinkeIn features in
my Office applications’. – [Jeremy] So you can get
it running right from there, even if you start from scratch. So, now, let’s say you’ve landed your job, you’re on the next journey,
the next chapter of your life, now how do I stay in the
job once I’ve got it? – Yeah, that’s right Jeremy.
So you’ve landed your role. And you’ll notice I’m using
Teams as my hub for teamwork. Now the great thing about
this is I can go back in time as the newest person joining the team. I can see the relevant
chats and conversations and get caught up. I don’t have to have people
send me a bunch of prior emails. I can get connected
exactly to what’s going on. I want to show you one example
here with Microsoft Stream. Now this is like a corporate YouTube where you can publish videos. But it has some really
unique capabilities. Here’s an example of the
Quarterly State of Contoso. This is the CEO addressing the company. I’m gonna click in and
see what Patti Fernandez had to say at the last company All Hands. So here I can consume this All Hands right from within Teams. But if I activate the full
Microsoft Stream experience, you’re gonna start to see
some additional capabilities that are fueled by AI. Let me show you a couple of those. Here’s Patti. She’s gonna begin speaking in a moment. And as I scroll down, do you see here three different people? So it’s actually sorting out
that like different people are speaking during this video. AI is determining who
those unique speakers are and where they’re speaking
inside the video itself. So I can use it as a navigational element. More importantly, I can filter this relatively
lengthy CEO address and say hey, I got hired to work on the Mark 8 Quadcopter, so I really just wanna kinda
see what Patti has to say about the Mark 8. When I type “mark 8”, that transcript is generated
automatically from the spoken text and shows me it’s
seven minutes and 43 seconds that’s where she… – And that’s quite cool cause it actually, you did a search for the letter eight and it actually found the
spelled out version of eight. So it’s actually done
some fuzzy matching there as it transcribed the voice to text. – Pretty intelligent. – Really what we’re seeing here from an intelligent standpoint is we’re seeing computer vision. So it’s detecting the speaker
and who’s actually talking and then letting you jump
between those different speakers. We saw with the heads
earlier in the different parts of the video. It’s also transcribing the
video from voice to text. And then based on that text
that’s been transcribed, it’s enabling these search capabilities that Jack just showed
by looking for “Mark 8”. – That’s exactly right. – Alright, so you’re back in Teams now, and I know you’re working on Mark 8. What else have you got going on? – So, we’ve got a crew
working around-the-clock. As the sun goes, so goes
the work around the world. And so a team in France here
is working on a presentation about the latest capabilities and the status on the Mark 8. So I’m gonna pull up
that presentation now. The first thing you
gotta do is translate it to a language so they can understand. I don’t know how to consume French, but with PowerPoint’s
Cloud-powered translator, I can simply translate these
slides from French to English, and in a matter of seconds, I’ve got that same presentation now translated using cognitive
services from French to English. I’m gonna take these four
pictures here of our Mark 8 and in just a moment the
Cloud-powered designer pops up and says, you know what, here are some alternate examples or ways you could lay out that slide. That’s pretty nice looking right? – [Jeremy] It’s not bad. – [Jack] Better than what we had before. – [Jeremy] A few other
recomendations you can make there. This looks really good. – [Jack] Yep. So I’m gonna
move on to the next slide. And here you see that this
very wide angle photo, clearly you need to
focus on the woman there, just to the left of center. And with salient region detection, this is happening automatically, intelligent cropping of the photo to make this woman the hero of the slide. As we move further, we’ve got
our development milestones and when we plan to launch. You can see November of this year. And I can very quickly turn that into an intelligent piece of smart art. Now because this is just smart art, I can visualize this information in a number of different ways. Including one of my favorites, the arrow pointing toward the upper left. So it’s starting to look pretty good. Yeah, Jeremy? – [Jeremy] Much better so far. – [Jack] And then, finally,
for a slide like this that has the series of
call-to-action steps that are prepended by a verb, I can represent that as a numbered list, or I can even go further down and call out that specific verb to draw attention to the action that I’m asking for. – Very cool stuff. So we saw you translate from
French to English earlier. Is there anything else we can do there? – Well I mentioned we’re
a global organization, so we’ve got a group in Asia that we’re gonna present this content to. And I understand you may
speak a little bit of Chinese. – (Jeremy speaking in Chinese) – [Jack] Alright. So what we’re gonna do is take advantage of the other
national language capability powered by cognitive services, and we’re going to
subtitle our presentation as you present it. So if you’d kindly put on
– Alright, let’s see if this your headset.
– works the other way around. – [Jack] You’re gonna to
speak simplified Chinese. And we’re gonna to subtitle it in English. – I’m gonna try to speak
simplified Chinese. – It’s gonna be fantastic. Okay, so before we get started, here you can see a QR code. And if you have the Microsoft
Translator app on your phone, you can actually choose from
a library of over 60 languages that you speak, and have
real-time simultaneous subtitling consume it on your phone in, for example, Portuguese or
some other language as well. So what I’m gonna do now is
begin the slide presentation. And I’ll go off mute and
you can begin speaking. – (Jeremy speaking in Chinese) – How about that. Pretty accurate? – Not bad. That’s exactly
what I was trying to say. – [Jack] Fantastic. So what you can see here is in the global organizations
of today and of tomorrow, look at all of the powerful
ways that we can communicate and keep that cycle going across the time zones, across
the world, across languages. – Very cool. So we just saw some, a lot of intelligence being applied within the context of PowerPoint. But I know you’re working
on some other things in terms of being able to
work with mixed reality also. So what can we do there? – So, I’ve shown you can
example of like an internal status update presentation, right. What I really wanna be able
to do is generate excitement and demand for my Mark 8 before it’s actually even
available on the market. I’d like to sell some pre-orders. So I’ve got an example here of
a two-customer presentation. And you know we’ve been
able to insert pictures inside of PowerPoints for many many years but what is new is this
ability to add 3D models. So I’ve got a 3D model
here of my quadcopter, and you can kinda see I
can rotate that there. I’ve got a variety of
different ways I’ve positioned that same image across multiple slides to call attention to different features. – [Jeremy] And the nice thing
is if you’re using PowerPoint now you can test that by using 3D images. Even if you don’t have things handy, just by going through and
finding those through the kinda the remix archive. – That’s exactly right. For example, if I went
up here to 3D Models, and I said I wanna pull
from online sources, you’ll see a series of very much like the online
pictures gallery as well and search for something
that you’re looking for. – [Jeremy] But we can actually do some pretty cool things. We’ve shown morph in
the past with 2D images. – We can do that here with
the 3D as well, right? – Right, so the morph transition
automatically calculates differences between slides. And what I’m gonna do here is take a copy of this quadcopter image and
simply paste it on slide two. But I’m gonna make some adjustments. Maybe I’m gonna rotate it a little bit and move it down here
into the lower right, to talk about the long-range control. And then, as I animate my presentation with the morph transition, watch what happens. I automatically calculate
differences between the slides in a cinematic movie-like effect. – [Jeremy] That looks amazing. – [Jack] So when you combine
that morph transition with all the other animations
that are in PowerPoint, anybody can create a presentation
that looks like this, incredibly professional.
Yes? Would you agree? – Very cool. So you’ve
shown now, with designer, showing how you can
make those other slides go from kind of zero to hero in that case. We’ve shown now with
morph and the 3D images. But I know you can do
a bit more in terms of mixed reality and what’s
built-in with Windows 10. – Yeah there’s nothing like you seeing it in the real world, to
create a sense of realism and also to grab your
checkbook and buy that first one that’s coming
off the assembly line. – Right, right. I’d love
to have a drone myself. – That’s right. So here I’ve got that
same 3D quadcopter model inside Windows 10 with mixed reality. Now at the moment I’m
just viewing the model, and of course I can, you know, look at it from a variety
of different perspectives. The same types of things we
observed there in PowerPoint. But if I take this Surface device, and I hold it here. And I’ll just point at the table and tap the mixed reality button. In just a moment, what
you’re gonna see is, we actually detect that
there’s a hard surface there and I can drop that quadcopter
right here into real space, adjust it, and you’ll notice as I rotate the surface device, it stays pinned in space and
I can even take a picture and then e-mail that picture to you to make sure that you’re
as excited as I am about purchasing one of these quadcopters. – Alright. That looks amazing. Yeah, I’d love to have
that here in the studio. So, very cool. So we saw how PowerPoint and kind of the Office
experience gets much better, a lot of people though,
when they think about AI, and they think about using intelligence, they’re really thinking about how they can apply that to data. So can we do anything with
Excel and AI together? – Yeah, you bet. So let me show you an example here of a feature called Insights. I’m looking at a bunch of data here. In fact, it’s a pretty sizeable amount. If I go to the bottom here
it’s about 4,466 rows of data. – [Jeremy] It’s quite a bit. – [Jack] It’s quite a bit. And so what you typically
do here is start creating some pivot tables and trying to find trends in that information. With a series of Cloud-powered algorithms with Insights, I can simply
click that Insights button, and you’ll notice on the right-hand side, we’re doing some analysis, we’re running some
heuristics on this data, and bringing back a series of
suggested things you might, or trends that you may
or may not have observed. So here I can see, for
example, spend by fiscal month, spend by category, ooh… Marketing and people seem to be declining. I hope I joined the right
team here at Contoso. So I wanna take this chart
and say, you know what, I wanna summarize that. By the way 28 different results. So there’s quite a few more
we could browse through. But in the interest of time, I’m just going to simply take this chart and we’ll see that the
intelligence in Excel has automatically created
a summary pivot table and a chart that goes along with it. – [Jeremy] That might
explain why the sales have been dropping for the normal cameras. – They need to double down
on my product I think. – Right. So we really
wanna help save time. What else can we do then, in
the context of work with AI? – So I’ll show you some great examples on a PC and a desk environment,
or a mobile environment. But, when you’re mobile, you’re also using your phone, right? More and more today. And so it’s really important
to be able to be productive on the go as well as safe on the go because you’re also more
distractible when you’re mobile. So I’m gonna bring up my mobile phone here and I’m gonna bring up
probably the application I use more than any other, which is Outlook. And in my inbox I’ve got
an email from Megan Bowen and she’s saying hey I’ve been looking at
this Contoso Q2 sales data, it’s a power BI dashboard, I’ll tap the link, and here I can see that, for example, our camera sales have been declining, quadcopters are increasing. That’s part of the business strategy. You’ve got a mature market around cameras and we’re trying to launch a new category to get to some new customers. I wanna have a meeting with the team to be able to discuss this in more detail and how to capitalize
and take advantage of it. But I don’t have a personal
assistant back at the office, so I’m gonna use Cortana to
help me schedule this meeting. You’ll notice I’ve copied Cortana and I’m just gonna @ mention Lee and add Lee to our discussion and here you can kinda see I said ‘Hey can you set up some time
tomorrow for us to meet?’ I’m gonna go ahead and send
that message to Cortana and she’s gonna, in the
background, consult the free busy, find out when people are available, and automatically schedule
this meeting for me. And sure enough, there it is. So now when I tap on the
message from Cortana, we’ve got the right time set up, we’ve got a meeting
that’s established for us. Pretty straight-forward. – Saves a lot of time.
Especially when you’re on mobile, if you have to kinda
compare everyone’s schedule. So one of the things I
noticed in your inbox is you’re getting emailed already directly from the CEO of the company? – I like to make a big
impact fast on a new job. But as I open this email from Patti, what you’ll notice is
intelligence in Outlook is actually highlighting
the fact that this email address is rather suspicious. This is not the one Patti typically uses to correspond with me. Likely a phishing attack. And so I think I’m going to hold off on responding until I can
investigate a little bit further. – It doesn’t look too legit. Of course we’ve built all of
this intelligence into Office, now if you’re a developer, you can use as your cognitive services, which includes things like
machine learning models, and it’s all hosted as a service, so it’s a really great way to add valuable AI capabilities right
into your apps and websites. Now, if you saw with Office, each of those cognitive
services really provides AI functionality spanning
vision, speech, language, knowledge and search like we just saw. So, Jack, are all of these
capabilities available now? – Most are available today
are being rolled out as part of the Office Insiders builds. So sign up as an Office Insider, and if you’re on Office
365 and you’re an admin, you can enable targeted
release in your tennant for everyone, or selected users today. – And of course, if you want to see the latest updates in action, subscribe to the Mechanics
channel on YouTube and follow us on Twitter. That’s all the time we have for this show. Thanks for watching,
we’ll see you next time.

No Comments

Leave a Reply