MBZUAI Collaborates To Launch Groundbreaking Open-Source AI Model K2-65B

The Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), in collaboration with Petuum and LLM360, has introduced "K2-65B," an open-source 65-billion parameter large language model (LLM). This model sets new benchmarks in transparency and performance for open-source artificial intelligence (AI), offering a comprehensive blueprint for documenting and studying the full lifecycle of LLMs, including all reproduction details.

Utilising LLM360’s release framework, K2-65B supports the community-led pursuit of artificial general intelligence (AGI) through peer-reviewed, transparent, reproducible, and collaborative open-source research. The model is available globally under the Apache 2.0 licence. Notably, K2-65B is the only third-party reproducible LLM to outperform major private sector models, such as Llama 2 70B.

LLMs have become crucial tools in natural language processing (NLP), enabling computers to understand and generate text similarly to human communication. UAE companies have made significant strides in this field, including the development of Jais, the world’s most advanced Arabic LLM, in partnership with Core42, MBZUAI, and Cerebras Systems.

Expert Opinions

MBZUAI President and University Professor Eric Xing, who supported K2’s development, stated, "The launch of K2-65B demonstrates the UAE’s growing prowess in superior LLM development. The model epitomises the importance of embracing an open and collaborative approach to create LLMs with unmatched performance and efficiency."

Comprehensive Evaluation

K2-65B was trained in two stages and underwent rigorous evaluation through 22 multidisciplinary assessments. These assessments ensured a comprehensive performance review across various domains including math, coding, and medicine. The new model surpassed Llama 2 70B across each of these areas.

Competitive Performance

In competitive arenas like the Open LLM Leaderboard, K2-65B demonstrated high-quality performance metrics. The chat model, K2-Chat, outperformed Llama 2 70B Chat in every aspect of evaluation, highlighting its capabilities in understanding and generating human-like responses across diverse scenarios.

Community Impact

Petuum’s Head of Engineering and lead developer Hector Liu commented, "Releasing a model of this size and quality, along with reproduction steps, will have reverberating positive effects on the open-source ecosystem as the community engages with the model and benefits from our learnings."

Unique Features

K2-65B stands out due to its full transparency enabled through the LLM360 Pretraining and Developer Suites. Equipped with detailed training guides, intermediate checkpoints, and evaluation results, K2-65B ensures reproducibility and auditability throughout its development process.

Sustainability

Using less computing power than comparable LLMs, K2-65B supports greater operational efficiency and reduced energy consumption. This enables users worldwide to adhere to sustainable computing practices.

Future Developments

The developers plan to incorporate image understanding capabilities into K2-65B. Ongoing development and evaluation initiatives aim at continually enhancing its performance and versatility.

With inputs from WAM