MBZUAI Unveils Nanda: Open-Source Hindi LLM Enhancing Access For Over Half A Billion Speakers

The Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) has introduced Nanda, the most sophisticated open-source Hindi large language model (LLM) globally. Developed by MBZUAI’s Institute of Foundation Models in collaboration with Inception and Cerebras Systems, Nanda represents a significant step in India's AI development. This model empowers over half-a-billion Hindi speakers to utilise generative AI in their native language.

Nanda, officially known as Llama-3-Nanda-10B-Chat, is a 10-billion parameter model. It outperforms existing open Hindi and multilingual models of similar size in knowledge and reasoning capabilities. The model was trained on the Condor Galaxy supercomputer, created by G42 and Cerebras Systems. Named after one of India’s highest peaks, Nanda is accessible at https://huggingface.co/MBZUAI/Llama-3-Nanda-10B-Chat.

Preslav Nakov, the project lead and Chair of the Natural Language Processing Department at MBZUAI, highlighted the significance of Nanda for generative AI in Hindi. "Nanda is an important advancement for generative AI for Hindi, which is one of the most widely spoken languages in the world," Nakov stated. The model can be downloaded from HuggingFace and run locally due to its reasonable size and modest hardware requirements.

Eric Xing, President of MBZUAI, emphasised the importance of an accurate Hindi LLM for India's inclusive AI ambitions. "An accurate and efficient LLM for the Hindi language is vital for India’s ambitions for inclusive and accessible AI," he said. The release aligns with MBZUAI's mission to advance generative AI for public benefit and support the UAE's knowledge-driven economy.

Monojit Choudhury, co-lead of the project and Professor of Natural Language Processing at MBZUAI, noted that current Hindi LLMs are lacking compared to English or European languages. He stated that developing robust LLMs for a widely spoken language like Hindi is crucial. "India is one of the world’s largest economies; any LLM that can serve Hindi will benefit communities as it opens new commercial opportunities," Choudhury added.

The introduction of Nanda follows Jais's success, an Arabic LLM that revolutionised Arabic Natural Language Processing (NLP). Jais provided native-language generative AI capabilities to over 400 million Arabic speakers worldwide. Nanda joins this collection of advanced foundation models at MBZUAI.

This development underscores MBZUAI's dedication to creating open-source LLMs that are affordable, safe, ethical, and standardisable. By doing so, they aim to lead advancements in generative AI while contributing significantly to global knowledge economies.

With inputs from WAM