Microsoft’s new Copilot will change Office documents forever

Microsoft’s new AI-powered Copilot summed up my meeting just yesterday (the meeting was with Microsoft to discuss Copilot, of course) before listing the questions I asked just seconds before. I’ve been watching Microsoft demo the future of work for years with virtual assistant concepts, but Copilot is the closest thing I’ve ever seen them come to fruition.

“In our minds this is the new way to compute, the new way to work with technology and the most adaptive technology we’ve seen,” says Jon Friedman, corporate vice president of design and research at Microsoft, in an interview with The Edge.

I spoke to Friedman on a Team call when he activated Copilot in the middle of our meeting to perform its AI-powered magic. Microsoft has an eye-popping marketing video that shows the potential of Copilot, but seeing Friedman demonstrate it in real time across Office apps and in Teams left me convinced that it will forever change how we interact with software, create documents, and ultimately, how we work .

Copilot appears in Office apps as a useful AI chat on the sidebar, but it’s much more than that. You could be in the middle of a Word document, and it will gently pop up when you highlight an entire paragraph – just like Word has UI prompts that highlight your spelling mistakes. You can use it to rewrite your paragraphs with 10 suggestions of new text to browse and freely edit, or you can have Copilot generate entire documents for you.

This adaptability is what sets it apart from Microsoft just shoving ChatGPT into a sidebar in Office. Copilot doesn’t just offer a chatbot interface – you can use it to command Office apps like Excel and PowerPoint. If you’re looking at a slide deck and wish every title was orange instead of blue, just ask Copilot instead of having to dig into PowerPoint features.

In Excel, you can have Copilot generate a PivotTable, create a graph, or simply help you understand the rows and columns of data in front of you. “One of the ways we start with Copilot is to help analyze and understand data,” says Friedman. “You can ask Copilot what it’s doing from the data, you can get graphs from Copilot based on trends it sees in the data, and you can put those trends into a spreadsheet.” Excel even has a “show me” feature for Copilot, which will let this AI teach you how it just completed a command so you can improve your Office knowledge.

It feels like Microsoft is slowly building on the vision it had for its Cortana assistant or even Clippy decades ago. “I love that our heritage is full of products that try to adapt to people,” says Friedman. “Copilot shares some similarities with some things we’ve done in the past, but it’s much more capable, it’s humble, and it’s there to serve things up for you that help you save time.”

Microsoft has customized this Copilot system for each Office application, so there are different ways to command it. Friedman showed me how Copilot can help you write emails in Outlook, offering short or long message drafts with options to change the tone. It even works in the mobile version of Outlook, which got me thinking about the ways this could speed up work on the go.

“Outlook mobile is the first place we’re making a big push,” explains Friedman. Outlook can summarize all your emails on the go, generate drafts, and generally make it easier to third-party your inbox. But imagine creating entire Word documents from your phone without having to type on a tiny on-screen keyboard. “We’ll have more to talk about mobile in the coming months,” Friedman says. But you can imagine where things will go.

As impressive as Copilot is, we’ve seen the many ways that great language models can fail, including inserting racial or gender bias into text and simply making things up. Those features are alarming enough in a search engine, but when you’re talking about Excel (which probably powers the world economy) or your email inbox, it’s a whole different level of ethics, privacy and data concerns.

“It gets things right a lot of the time but not always,” admits Friedman. “In the user experience we do things like put in costs to not send something until you’ve read it, or encourage you to try again, edit and discard.”

Microsoft also has some warnings within Copilot that pop up while you’re using it. In PowerPoint, you’ll see a message that says, “Content is generated by AI and may contain errors for sensitive material. Be sure to verify information.” Elsewhere, there are prompts that say, “AI-generated content may be incorrect.” Microsoft tries to design the system in a way that reminds you of that you are carefully

“We give you tools to report it when it’s wrong. We create prompt suggestions to help you write good invitations. Everything we do in the user experience is to make it conversational and give you agency,” says Friedman.

We’ve seen what happens when it goes wrong in Microsoft’s Bing search engine. The AI-powered chat hallucinated several times, and Microsoft had to put limits to control its outbursts. In one conversation with The Edge, Bing even claimed that it spied on Microsoft employees with webcams on their laptops and manipulated them.

“Everything we learn from Bing in advance helps us mitigate those risks,” Friedman says. “We are applying this learning and thinking to Copilot as well.” Microsoft is also starting small with its launch of Copilot. It will initially be available to just 20 businesses before Microsoft opens it up to more when it’s ready. Microsoft is also starting with enterprise customers first before it expands to consumers.

“We feel pretty good about what we have as a starting point, but we still don’t know if it works the way we want it to and helps really empower people to do their jobs,” Friedman says. “We will quickly iterate, we built quickly, very quickly. But we are taking a break and we are learning a lot and we will update very quickly. Our plan is to move as quickly as possible to scale to more enterprise in a thoughtful and responsible way and make sure the experience is amazing.”

Is Microsoft moving too fast, though? Google announced its own AI features for Gmail and Docs earlier this week, and the AI ​​race has many experts worried that the tech giants aren’t properly considering the impact of these new tools.

“In our mind we are thoughtfully fast,” says Friedman. “We’re thinking we’re rolling it out to 20 clients and working side by side with them.” Despite reports of Microsoft firing its ethics and society team, which taught employees how to make AI tools responsibly, Friedman says Microsoft is growing the people working on these concerns. “Based on our investment in ethics and AI, we’ve grown more and more ethics and AI experts in all the product teams working on this issue,” Friedman says. “We have to scale much bigger, so we invested more and it continues to grow year after year.”

Microsoft knows that Copilot is not perfect and that it will take some time to get there. While it impressed me during the Teams meeting summary, it could easily confuse my voice with someone else’s if I used a bad microphone, or Outlook could pull the wrong summary in an email thread. There are big challenges ahead, but Microsoft hopes that the work it’s doing to make it easier to edit responses, correct sources, and publish suggestions will ultimately improve the system.

“We know AI makes mistakes, we know it hallucinates and we know it does it confidently,” admits Friedman. “We keep working to improve it by doing less of that, but also that the user experience really empowers people and puts them in the driver’s seat.”

For all the challenges, the future of the Copilot system will not be just a text-based generation either. Microsoft has a clear vision of using Copilot to generate images, videos and more when large language models can handle these functions well.

Microsoft has already integrated OpenAI’s DALL-E model into its Designer application, which allows people to generate images based on text. Designer will also help PowerPoint choose the best images for AI-generated slides. “We’re going to continue to bake Designer into Copilot, so you can change things in Designer,” says Friedman. “The Designer stuff you’ve seen today only scratches the surface. I fully suspect that we will be using Copilot to do amazing multimedia things.”

So where else could we see Copilot appear? I asked Friedman about Windows integration. “We’re looking at all kinds of places and ways to expand (Copilot). I believe this is the next major wave of computing and it will change the way we work with all devices in the coming years,” says Friedman.

The future will also include a multiplayer experience for Copilot as well. Loop components, one of the biggest changes to Office documents in decades, are available in Teams and Outlook. Loop components, the trademark for Microsoft’s Fluid workflow, are blocks of collaborative text or content that can live independently and be copied, pasted and shared freely.

Now imagine copying a Loop component from text into an email and having multiple people edit it and interact with the Copilot. “In the as-we-edit component, the conversation is a clickable history of the content created that you can go back and forth through,” says Friedman. “What’s so great about it is that it feels like a whole new mental model of how to work with Copilot with a group of people.”

All of these Copilot features for Office and Microsoft 365 feel like they will forever change how we work and communicate, especially as these big language models evolve in the coming years. Microsoft’s push to integrate this AI deeply into its products could have a lasting impact on the job market.

“Every time there’s a new technical advance, there are both opportunities and things we have to consider,” says Friedman. “We believe that infusing this AI will create new job opportunities in the long term and increase job satisfaction in the short term. We expect it to change the nature of many jobs and create new jobs that didn’t exist before. That’s why empowering people and building this shared design system is so important to us.”