Introducing Jais: Advanced Arabic Language AI model

Jais is an advanced Arabic language AI model developed by G42, a company based in Abu Dhabi, United Arab Emirates. Jais is designed to be a powerful tool for natural language processing tasks in Arabic. The company behind the AI models claims it can generate high-quality text in Arabic, including regional dialects, and can also perform tasks such as translation, summarization, question answering, and more. Jais will be made available under an open-source license, enabling broader access and potential contributions from the developer community.

The main characteristics:

• Parameters: Jais contains 13 billion parameters, making it a large language model.
• Training: Jais was trained on the Condor Galaxy 1 AI supercomputer using a large amount of data. It was trained on 116 billion Arabic tokens and 279 billion English tokens.
• Features: The model is equipped with cutting-edge features like ALiBi position embeddings, which enhance its ability to handle context.

What is ALiBi? AliBi, or Attention with Linear Biases, is a positioning method that allows Transformer language models to consume, at inference time, sequences that are longer than the ones they were trained on.

Why does it matter?

Its open-source nature allows researchers, developers, and the Arabic-speaking community to access and contribute to its development. These models could also contribute to the progress of Arabic language processing and generative AI, igniting further innovation and advancement in the field. These models developed in other languages (besides English) are important for digital multilingualism as they facilitate translation, enable effective communication, expand global reach, and enhance communications and learning. They contribute to a more inclusive and connected digital world.