Microsoft’s compact Phi-2 rivals larger AI models

Microsoft (NASDAQ: MSFT) has unveiled a new small language model (SML), dubbed Phi-2, designed to match the capabilities of larger artificial intelligence (AI) models.

Microsoft Research described Phi-2 as achieving remarkable performance in several benchmarks. Trained on only 2.7 billion parameters, Phi-2 held its own against base language models with less than 13 billion parameters despite a limited training data set.

The SML was unveiled at Microsoft’s Ignite 2023 event, with company CEO Satya Nadella
reeling out the model’s capabilities. In his speech, Nadella disclosed that Phi-2 demonstrated competence in critical reasoning and language understanding in academic benchmarks comparable to large language models (LLMs).

Today, we share our teams’ latest contributions, Phi-2 and promptbase.

Phi-2 outperforms other existing small language models, yet it’s small enough to run on a laptop or mobile device. https://t.co/wLhUeRsByL

— Microsoft Research (@MSFTResearch) December 12, 2023

Phi-2 showed proficiency in math, coding, and common sense reasoning tests, surpassing
Mistral and Llama 2 models with 7 billion and 13 billion parameters, respectively. Furthermore, the latest SML performed better than Llama 2’s 70B model on multi-step reasoning tasks, even outperforming Google’s (NASDAQ: GOOGL) newly minted Gemini Nano 2.

The company reached its conclusions after testing Phi-2 using internal proprietary datasets and tasks, adding testing with commonly used prompts in research circles. Microsoft achieved the feat with its SMLs thanks to advances in data curation techniques and textbook-quality data.

Microsoft Research says Phi-2’s size could see it play a leading role in AI and machine learning research and is in line with its mantra of “breaking the conventional language model scaling laws.”

“With its compact size, Phi-2 is an ideal playground for researchers, including for exploration around mechanistic interpretability, safety improvements, or fine-tuning experimentation on a variety of tasks,” read the report.

Before the release of Phi-2, Microsoft Research tested the waters with Phi-1, a 1.3 billion parameter SML, following up with a second iteration dubbed Phi-1.5 with advanced common sense reasoning.

Pushing the limits of AI

Microsoft’s foray into AI extends beyond model development, delving into the uncertain terrain of custom chip development. The company has previously hinted toward integrating custom chips—Maia and Cobalt—in the battle for AI supremacy against Google and other rivals.

In November, the company announced multi-year partnerships with Australia and the U.K. governments to improve the infrastructure for AI and other emerging technologies. Long-term collaboration with ChatGPT maker OpenAI has piqued the interest of antitrust regulators around the globe, opening a new can of worms for Microsoft.

In order for artificial intelligence (AI) to work right within the law and thrive in the face of growing challenges, it needs to integrate an enterprise blockchain system that ensures data input quality and ownership—allowing it to keep data safe while also guaranteeing the immutability of data. Check out CoinGeek’s coverage on this emerging tech to learn more why Enterprise blockchain will be the backbone of AI.

Watch: AI takes center stage at London Chatbot Summit with Craig Massey and Elizabeth Stokoe