Last Updated on October 26, 2023 by Nahush Gowda
Microsoft and OpenAI, which are the leading names when it comes to current AI era, are thinking of developing their own custom AI chips. The tech sector and in particular AI technologies are progressing at insane speeds.
When we talk about AI (Artificial Intelligence) we cannot miss naming Nvidia.
Most of the modern AI tech like Popular LLM (Large language model)programs – ChatGPT, Google Bard, Midjourney, Microsoft Bing (Uses ChatGPT underneath)etc. All are developed on hardware which includes over thousands of GPU chips. Most of these (except Google and some others) have Nvidia hardware in the form of NVIDIA H100 and Nvidia A100 GPU chips.
At a high level, it seems the AI companies are easily able to take advantage of easy to deploy Nvidia AI hardware. But, the pricing?, H100s retail at around $30,000 a piece. Given the scale at which AI data centers operate, the costs can quickly spiral into hundreds of millions of dollars.
Enter Project Athena. Microsoft’s audacious plan to replace this Nvidia dependence with its own custom AI chips. Why?
Firstly, it aims to cut down the astronomical costs associated with AI infrastructures. By leveraging its own in-house silicon, Microsoft could become less dependent on external vendors, driving costs down and perhaps passing some of the savings to consumers.
Secondly, with its own custom Hardware, Microsoft will be able to design its AI algorithms more efficiently.
Athena’s performance claims are still under wraps, but even matching NVIDIA’s current-gen chips would be a considerable feat.
Microsoft is stated to announce this in their November Ignite event.
Will this new move give Microsoft and OpenAI considerable edge over other AI products companies? Does Nvidia have reason to worry? Let’s find out.
Unraveling NVIDIA’s AI Dominance: It’s a Hardware and Software combination
It’s no secret that NVIDIA currently enjoys a near-monopolistic hold over the AI chip industry. Their H100 and A100 GPUs serve as the backbone of AI infrastructures across sectors and continents.
However, NVIDIA’s dominance isn’t solely rooted in hardware prowess. The company has meticulously constructed an ecosystem around its CUDA API and libraries, making it almost second nature for developers to leverage NVIDIA’s GPUs.
Nvidia has great developer advocacy program for its CUDA stack – starting from Universities itself where students have great access to Nvidia stack. This forms a strong affiliation of CUDA with AI stacks for students who in later go on to become AI researchers and developers, taking this CUDA advocacy with them
For years, CUDA has offered unmatched ease of deployment—think “pip install torch”—while alternatives like AMD’s ROCm languish due to lack of robust software support and developer affinity until recent times. Only now, AMD has started working on it.
More Than Just Silicon
As analysts and developers are keen to point out, hardware alone won’t dismantle NVIDIA’s hegemony. Microsoft needs to introduce a compelling software framework to accompany Athena.
Related: Applications of Graphics Cards Beyond Gaming
Why This Matters to the Industry
NVIDIA’s monopoly isn’t just a financial bottleneck. A monolithic supplier can often lead to stagnation in innovation, and there’s already a chorus of voices in academia and the private sector lamenting the lack of affordable, alternative AI acceleration options.
Microsoft’s endeavor could open the floodgates, offering more choices to developers and fostering competition, which can only expedite advancements in AI technologies.
What About OpenAI?
Interestingly, OpenAI, heavily backed by Microsoft and a user of NVIDIA’s chips, is also rumored to be exploring its own AI chip solutions. OpenAI CEO Sam Altman is actively looking to acquire startups in the field.
With OpenAI’s significant reliance on NVIDIA’s GPUs for its services like ChatGPT, an in-house alternative could alleviate some of the cost pressures and offer more scalable and reliable options for consumers.
Now, since OpenAI is already backed by Microsoft, it would make sense both will be using same AI custom chip taking advantages of shared resources and economies of scale.
There can be other case as well with same underlying architecture for their AI chips but customizing high level layer for their specific use case 0 like Microsoft of their AI cloud services and OpenAI for their GPT stack.
Alternatively, both could go separate ways, each developing their own custom AI chips. It would be interesting to see which direction they take.
The Verdict
Even though, foraying into custom AI chips, is a Herculean challenge. There are only a few companies with resources, scale and experience to accomplish this. Microsoft is one of them. Google is already having its own custom AI chip solution in form of Tensor TPU, which is somewhat based on Nvidia architecture.
Combating NVIDIA’s monopoly isn’t just about releasing a rival chip; it’s about disrupting an entire ecosystem. And if there’s one company with the resources and mettle to do so, it’s Microsoft. Keep an eye out for Ignite this November.
Leave a Reply
You must be logged in to post a comment.