MS-700 certification - Managing Microsoft Teams Updated: 2023

Microsoft is best known for its Windows operating systems and Office software. But the company has a much broader product portfolio that includes online services (Bing, MSN, advertising), gaming (Xbox 360), hardware (tablets, PCs, keyboards and mice) and more. The company also has a sizable certification program that turns out qualified administrators and technicians to support its system and application products.

Achieving IT certifications through the Microsoft Certification Program shows a person’s competence in a specific IT role, and it can result in all kinds of work-related and personal benefits. For example, studies show that IT certifications increase the chances of landing a job (or getting a promotion), and over 80 percent of hiring managers report IT certifications are medium to high priority in hiring decisions.

Microsoft is moving to strengthen its computing infrastructure play with the announcement of two new in-house chips for enterprises: Azure Maia 100 and Azure Cobalt 100.

Showcased at this week’s Microsoft Ignite 2023 conference in Seattle, the tech giant’s largest annual global event, the chips provide enterprises with efficient, scalable and sustainable compute to take advantage of the latest cloud and AI breakthroughs.

Microsoft says they represent the final piece of the puzzle in its mission to deliver flexible infrastructure systems – with its own and partner-delivered hardware and software – that can be optimized to meet different workload requirements.

Maia, as the company explained, is its AI accelerator, designed to run cloud-based training and inference for generative AI workloads. Meanwhile, Cobalt is an Arm-based chip designed to handle general-purpose workloads with high efficiency. Both offerings will be deployed in Azure next year, starting with Microsoft’s own data centers driving its Copilot and Azure OpenAI services. 

“We are reimagining every aspect of our data centers to meet the needs of our customers,” Scott Guthrie, executive vice president of Microsoft’s Cloud + AI Group, said in a statement. “At the scale we operate, it’s important for us to optimize and integrate every layer of the infrastructure stack to maximize performance, diversify our supply chain and supply customers infrastructure choice,” he added.

What to expect from Azure Maia and Cobalt?

While Microsoft has not shared specific performance stats, the company does note that the Maia AI chip can handle some of the largest AI workloads running on Microsoft Azure, right from training language models to inferencing. The silicon has been designed specifically for the Azure hardware stack, which enables it to achieve absolute maximum utilization of the hardware when handling the workloads.

It developed the accelerator over the years by working in conjunction with OpenAI. Specifically, the company tested it on models built by the Sam Altman-led generative AI unicorn and used the feedback to make necessary changes.

“We were excited when Microsoft first shared their designs for the Maia chip, and we’ve worked together to refine and test it with our models. Azure’s end-to-end AI architecture, now optimized down to the silicon with Maia, paves the way for training more capable models and making those models cheaper for our customers,” Altman said in a statement.

Like Maia, the capabilities of Cobalt are also largely under the wraps. However, one thing’s pretty clear: this chip will handle general-purpose workloads on Azure with a focus on energy efficiency. The Arm-based design ensures it is optimized to maximize performance per watt, making sure that the data center gets more computing power for each unit of energy consumed. 

“The architecture and implementation are designed with power efficiency in mind. We’re making the most efficient use of the transistors on the silicon. Multiply those efficiency gains in servers across all our data centers, it adds up to a pretty big number,” Wes McCullough, corporate vice president of hardware product development at Microsoft, said.

More importantly, since both of the chips have been designed in-house, Microsoft will install them on custom server boards, placed within tailor-made racks that fit easily within existing company data centers. For the Maia rack, the company has also designed “sidekicks” that direct cold liquid to the cold plates of the chips, ensuring that it does not heat up when dealing with heavy power usage situations.

Partner integrations expanded

As part of its flexible systems approach, Microsoft is also backing its custom chips with expanded support for partner hardware. The company said it has launched a preview of the new NC H100 v5 virtual machine series built for Nvidia H100 Tensor Core GPUs and will soon add the latest Nvidia H200 tensor core GPUs to its data center fleet. It also plans to add AMD MI300X accelerated VMs to Azure to accelerate the processing of AI workloads for high-range AI model training and generative inferencing.

This approach gives customers of the company multiple options to choose from, depending on their performance or cost needs.

As of now, Microsoft plans to roll out the new chips next year. It has already started working on their second generation.

Microsoft has announced a project it has been "refining in secret for years;" Its own custom silicon in the form of two new server chips. The company unveiled the fruits of its labor at Microsoft Ignite, showing off the Azure Maia AI Accelerator and the Azure Cobalt CPU. The latter of which, at least, the company is happy to admit is ARM-based, which can still feel unthinkable to eyes so used to Microsoft and Intel's hand-in-glove dominance of the computing market.

The company turned to OpenAI to receive feedback on Azure Maia and to use the company's models for testing. OpenAI CEO Sam Altman said the updated Microsoft's Azure will also provide the opportunity for training improved models and making them more affordable for customers.

The custom-designed chips can further optimize Microsoft's infrastructure instead of relying on third-party options. "Much like building a house lets you control every design choice and detail, Microsoft sees the addition of homegrown chips as a way to ensure every element is tailored for Microsoft cloud and AI workloads," a blog post from the company explained. "The chips will nestle onto custom server boards, placed within tailor-made racks that fit easily inside existing Microsoft datacenters. The hardware will work hand in hand with software — co-designed together to unlock new capabilities and opportunities."

The company plans to use the new Maia 100 AI Accelerator to power some of Microsoft Azure's biggest internal AI workloads. Microsoft claims both the accelerator and Azure Cobalt CPU will Strengthen efficiency and performance. The chips will make their way to Microsoft's data centers early next year for powering services like Microsoft Copilot (now encompassing Bing Chat) and Azure OpenAI Service.

Microsoft is expanding its policy to protect commercial customers from copyright infringement lawsuits arising from the use of generative AI -- but with a caveat (or several).

Today during Ignite, Microsoft said that customers licensing Azure OpenAI Service, the company's fully managed service that adds governance layers on top of OpenAI models, can expect to be defended -- and compensated -- by Microsoft for any "adverse judgements" if they're sued for copyright infringement while using Azure OpenAI Service or the outputs it generates.

Generative AI models such as ChatGPT and DALL-E 3 are trained on millions to billions of e-books, art pieces, emails, songs, audio clips, voice recordings and more, most of which come from public websites. While some of this training data is in the public domain, some isn't -- or comes under a license that requires citation or specific forms of compensation.

The legality of vendors training on data without permission is another matter that’s being hashed out in the courts. But what might possibly land generative AI users in trouble is regurgitation, or when a generative model spits out a mirror copy of a training example.

Microsoft's expanded policy won't apply by default to every Azure OpenAI Service customer. To be eligible for the new protections, subscribers are on the hook for implementing "technical measures" and complying with certain documentation to mitigate the risk of generating infringing content using OpenAI's models.

TechCrunch asked Microsoft to elaborate on these measures, but the company declined to provide specifics ahead of the announcement this morning.

It's also unclear if the protections extend to Azure OpenAI Service products in preview, like GPT-4 Turbo with Vision, and whether Microsoft is offering indemnity against claims made over the training data used by customers to fine-tune OpenAI models. We asked for clarification.

Late this afternoon, a Microsoft spokesperson told TechCrunch via email that the policy applies to all products in paid preview and Microsoft's -- but not a customer's -- training data.

The new policy comes after Microsoft's announcement in September that it’ll pay legal damages on behalf of customers using some -- but not all -- of its AI products if they’re sued for copyright infringement. As with the Azure OpenAI Service protections, customers are required to use the "guardrails and content filters" built into Microsoft's AI offerings in order to retain coverage.

Perhaps not coincidentally, OpenAI recently said that it would begin paying the legal costs incurred by customers who face lawsuits over IP claims against work generated by OpenAI tools. Microsoft's new Azure OpenAI Service protections would appear to be an extension of this.

Beyond indemnity policies, a partway solution to the regurgitation problem is allowing content creators to remove their data from generative model training data sets -- or to supply those creators some form of credit and compensation. OpenAI has said that it'll explore this with future text-to-image models, perhaps the follow-up to DALL-E 3.

Microsoft, in contrast, hasn't committed to opt-out or compensation schemes. But the company has developed a technology it claims can help "identify when [AI] models generate material that leverages third-party intellectual property and content." A new feature in Microsoft's Azure AI Content Safety tool, it's available in preview.

We asked for background on how the IP-identifying tech works, but Microsoft demurred -- simply pointing to a high-level blog post. We'll keep our eyes peeled for more details at Ignite.

Why it matters: Microsoft had been rumored to be working on custom silicon for its data center needs for years. As it turns out, the rumors were true and this week the company unveiled not one but two Arm-based processors. The new chips will be integrated into Azure server farms starting in early 2024, to be used as the workhorses of AI services like Microsoft Copilot.

This week, Microsoft announced it has built two "homegrown" chips that will handle AI and general computing workloads in the Azure cloud. The announcement was made at the Ignite 2023 conference and confirms previous rumors about the existence of "Project Athena" – a custom-designed Arm-based chip that would reduce Microsoft's reliance on off-the-shelf hardware from vendors like Nvidia, especially in the area of artificial intelligence training and inference.

The first chip is called the Microsoft Azure Maia 100 AI Accelerator and is the direct result of Project Athena. As its lengthy name suggests, the Redmond giant designed the chip specifically for running large language models such as GPT-3.5 Turbo and GPT-4. Built on TSMC's 5nm process and featuring no fewer than 105 billion transistors, the new chip supports various MX data types, including sub-8-bit formats for faster model training and inference times.

For reference, Nvidia's H100 AI Superchip has 80 billion transistors, and AMD's Instinct MI300X has 153 billion transistors. That said, we have yet to see any direct performance comparisons between the Maia 100 AI Accelerator and the existing chips used by most companies building AI services. What we do know is that each Maia 100 compute unit has an aggregate bandwidth of 4.8 Terabits thanks to a custom Ethernet-based network protocol that allows for better scaling and end-to-end performance.

Also read: Goodbye to Graphics: How GPUs Came to Dominate AI and Compute

It's also worth noting that Microsoft developed the Maia 100 chip using extensive feedback from OpenAI. The two companies worked together to refine the architecture and test GPT models. For Microsoft, this will help optimize the efficiency of Azure's end-to-end AI architecture, while OpenAI will be able to train new AI models that are better and cheaper than what is available today.

The second chip introduced by Microsoft at Ignite is called the Cobalt 100 CPU. This one is a 64-bit, 128-core Arm-based processor based on the Arm Neoverse Compute Subsystems and brings performance improvements of up to 40 percent for more general Azure computing workloads when compared to current generation hardware found in commercial Arm-based servers. Cobalt 100-based servers will be used to power services like Microsoft Teams and Windows 365, among other things.

Rani Borkar, who is the head of Azure infrastructure systems at Microsoft, says the company's homegrown chip efforts build on top of two decades of experience in co-engineering silicon for Xbox and Surface. The new Cobalt 100 CPU allows the company to control performance and power consumption on a per-core basis and makes it possible to build a more cost-effective cloud hardware stack.

Pictured above is a custom-built rack for the new Maia 100 AI Accelerator at a Microsoft lab in Redmond. The "sidekick" on the left is used to cycle cooling liquid to and from the rack.

The cost part of the equation is particularly important. In the case of the Maia 100 AI Accelerator, Microsoft had to come up with a new liquid cooling solution and a new rack design that provides more space for power and networking cables. That said, the cost of using the new chip is still significantly lower than using specialized hardware from Nvidia or AMD.

Microsoft seems determined to make a Copilot "for everyone and everything you do," and that is reflected in the release of Copilot for Windows, GitHub, Dynamics 365, Microsoft Security, and Microsoft 365. The company just rebranded Bing Chat to "Microsoft Copilot," so it's clear it wants to bolt ever more advanced AI models into every service it offers moving forward.

Rani Borkar holding samples of Microsoft's custom-designed chips

AI training and inference get expensive fast, and running an AI service is estimated to be up to ten times more expensive than something like a search engine. Making custom silicon could also alleviate supply issues and help Microsoft get a competitive advantage in a crowded landscape of AI cloud providers. Some like Amazon, Meta, and Google also have their own homegrown silicon efforts for the same reasons, and companies like Ampere that once dreamed of becoming the go-to suppliers of Arm-based data center chips will no doubt be forced to adapt to these developments if they want to survive.

That said, the Redmond company says it will keep using off-the-shelf hardware in the near future, including the recently announced H200 Tensor Core GPU from Nvidia. Scott Guthrie, who is executive vice president of the Microsoft Cloud + AI Group, says this will help diversify the company's supply chain and supply customers more infrastructure choices.

Today at Microsoft Ignite, the company unveiled two custom-designed chips and integrated systems resulting from a multi-step process for meticulously testing its homegrown silicon, the fruits of a method the company's engineers have been refining in secret for years, as revealed at its Source blog.

The end goal is an Azure hardware system that offers maximum flexibility and can also be optimized for power, performance, sustainability or cost, said Rani Borkar, corporate vice president for Azure Hardware Systems and Infrastructure (AHSI).

“Software is our core strength, but frankly, we are a systems company. At Microsoft we are co-designing and optimizing hardware and software together so that one plus one is greater than two,” Borkar said. “We have visibility into the entire stack, and silicon is just one of the ingredients.”

The Chips

The newly introduced Microsoft Azure Maia AI Accelerator chip is optimized for artificial intelligence (AI) tasks and generative AI. For its part, the Microsoft Azure Cobalt CPU is an Arm-based processor chip tailored to run general purpose compute workloads on the Microsoft Cloud.

Microsoft said the new chips will begin to appear by early next year in its data centers, initially powering services such as Microsoft Copilot, an AI assistant, and its Azure OpenAI Service. They will join a widening range of products from the company's industry partners geared toward customers eager to take advantage of the latest cloud and AI technology breakthroughs.

“Microsoft is building the infrastructure to support AI innovation, and we are re-imagining every aspect of our data centers to meet the needs of our customers,” noted Scott Guthrie, executive vice president of Microsoft’s Cloud + AI Group. 

Guthrie emphasized, “At the scale we operate, it’s important for us to optimize and integrate every layer of the infrastructure stack to maximize performance, diversify our supply chain and supply customers infrastructure choice.”

The Microsoft Azure Maia 100 AI Accelerator is the first chip designed by Microsoft for large language model training and inferencing in the Microsoft Cloud. The chip will power some of the largest internal AI workloads running on Microsoft Azure. 

Additionally, the company said its partner OpenAI is providing feedback on the Azure Maia platform as to how its workloads run on infrastructure tailored for its large language models, which will help to inform future Microsoft designs.

“Since first partnering with Microsoft, we’ve collaborated to co-design Azure’s AI infrastructure at every layer for our models and unprecedented training needs,” explained Sam Altman, CEO of OpenAI. 

Altman added, “We were excited when Microsoft first shared their designs for the Maia chip, and we’ve worked together to refine and test it with our models. Azure’s end-to-end AI architecture, now optimized down to the silicon with Maia, paves the way for training more capable models and making those models cheaper for our customers.”

The Maia 100 AI Accelerator was designed specifically for use with the Azure hardware stack, pointed out Brian Harry, a Microsoft technical fellow leading the Azure Maia team. 

That vertical integration – the alignment of chip design with the larger AI infrastructure designed with Microsoft’s workloads in mind – can yield huge gains in performance and efficiency, Harry said. “Azure Maia was specifically designed for AI and for achieving the absolute maximum utilization of the hardware,” he added.

Importantly, according to Wes McCullough, Microsoft's corporate vice president of hardware product development, Microsoft's new Cobalt 100 CPU is built on an Arm architecture which embodies a new type of energy-efficient chip design, optimized to deliver greater efficiency and performance in cloud-native offerings. 

Choosing Arm technology was a key element in Microsoft’s sustainability goal, McCullogh said, adding that the company aims to optimize “performance per watt” throughout its data centers - which essentially means getting more computing power for each unit of energy consumed. 

“The architecture and implementation is designed with power efficiency in mind,” he said. “We’re making the most efficient use of the transistors on the silicon. Multiply those efficiency gains in servers across all our data centers, it adds up to a pretty big number.”

Send In the NVIDIA GPUs

To complement its custom silicon efforts, Microsoft said it is also expanding on its industry partnerships to provide more infrastructure options for customers. 

For example, the company has launched a preview of its new NC H100 v5 Virtual Machine Series, built for NVIDIA H100 Tensor Core GPUs, offering greater performance, reliability and efficiency for mid-range AI training and generative AI inferencing. 

Microsoft also said it will add the latest NVIDIA H200 Tensor Core GPU to its fleet next year to support larger model inferencing with no increase in latency.

New Racks with a Side of Liquid Cooling

Meanwhile, no racks existed to house the unique requirements of the Maia 100 server boards, so Microsoft built them from scratch. 

These new racks are wider than what typically sits in the company’s data centers. That expanded design provides ample space for both power and networking cables, it said, essential for the unique demands of AI workloads.

As duly noted by Microsoft's blog, such AI tasks come with intensive computational demands that consume more power, and traditional air-cooling methods often fall short for these high-performance chips. As such, liquid cooling employing circulating fluids to dissipate heat has emerged as the preferred industry solution for alleviating such thermal challenges, ensuring that chips run efficiently without overheating.

But Microsoft’s current data centers weren’t designed for large liquid chillers. As a workaround, the company has developed a “sidekick”element that sits next to the Maia 100 rack. 

How the sidekick works is analogous to the operation of a car's radiator. Cold liquid flows from the sidekick to cold plates that are attached to the surface of Maia 100 chips. Each plate has channels through which liquid is circulated to absorb and transport heat. 

That flows to the sidekick, which removes heat from the liquid and sends it back to the rack to absorb more heat, in a circular process. Microsoft's McCullough said the tandem design of rack and sidekick underscores the value of a systems approach to infrastructure. 

"By controlling every facet — from the low-power ethos of the Cobalt 100 chip to the intricacies of data center cooling — Microsoft can orchestrate a harmonious interplay between each component, ensuring that the whole is indeed greater than the sum of its parts in reducing environmental impact," he added.

The Best Set of Options

Before 2016, most layers of the Microsoft cloud were bought off the shelf, noted Pat Stemen, partner program manager on the company's AHSI team. Then, the company began to custom build its own servers and racks, the better to reduce costs while furnishing a more consistent customer experience, before, over time, silicon became the system's primary missing piece. 

Today's announcements go a long way toward shoring up that gap. The silicon architecture unveiled today lets Microsoft not only enhance cooling efficiency, but also optimize the use of its current data center assets, whie maximizing server capacity within its existing footprint, the company said. 

Meanwhile, Stemen noted that Microsoft has shared design discoveries gleaned from its custom rack, which can be used those no matter what piece of silicon sits inside, with its industry partners. 

“All the things we build, whether infrastructure or software or firmware, we can leverage whether we deploy our chips or those from our industry partners,” he said. “This is a choice the customer gets to make, and we’re trying to provide the best set of options for them, whether it’s for performance or cost or any other dimension they care about.”

Keep pace with the fast-moving world of data centers and cloud computing by connecting with Data Center Frontier on LinkedIn, following us on X/Twitter and Facebook, and signing up for our weekly newsletters using the form below.