In a groundbreaking achievement for the technology landscape in the Middle East, Amazon Web Services, Inc. (AWS), an entity under Amazon.com, Inc., has unveiled a remarkable development in artificial intelligence. The Technology Innovation Institute (TII) of Abu Dhabi, the applied research arm of Abu Dhabi’s Advanced Technology Research Council (ATRC), has introduced its Falcon 180B large language model, making it available for deployment on Amazon’s SageMaker JumpStart platform. This momentous release marks an industry milestone, with Falcon 180B boasting an astounding 180 billion parameters and the support of an extensive 3.5-trillion-token dataset. This robust model represents an expansion of TII’s previous Falcon 40B model, incorporating advanced technologies like multi-query attention for heightened scalability.
Falcon 180B underwent training on a cluster of nearly 4,000 A100 GPUs, ensuring seamless compatibility with the Amazon SageMaker environment. Amazon Simple Storage Service (Amazon S3) handles data storage and checkpoint management, guaranteeing both workload reliability and operational ease. Additionally, TII has introduced a specialized chat version, Falcon 180B-Chat, optimized for conversational applications. These models operate within a rigorously secure AWS environment adhering to Virtual Private Cloud (VPC) protocols, ensuring enhanced data protection for users.
For those eager to explore Falcon 180B, the model is readily accessible within SageMaker Studio via Amazon SageMaker JumpStart. In alignment with TII’s responsible innovation philosophy, the model is released under the Falcon-180B TII License and Acceptable Use Policy, promoting responsible AI utilization. Benchmark tests confirm the model’s exceptional performance, positioning it between GPT-3.5 and GPT-4 across various language understanding and generation metrics.
Dr. Ebtesam Almazrouei, Executive Director and Acting Chief AI Researcher of AI Cross Center Unit and Project Lead for LLM Projects at TII, stated, “By offering Falcon 180B as an open-access tool, we’re establishing the foundation for organizations to harness its unparalleled capabilities and pave the way for future advancements and transformative AI solutions. Our pursuit of scientific knowledge should always serve the greater good, and open-sourcing this technology allows us to democratize its benefits, foster collaboration, and ensure transparency. We are thrilled to provide the Falcon 180B platform, which we believe will catalyze groundbreaking advancements in both academic research and industry solutions.”
Experts have responded positively to Falcon 180B, with Huggingface scientist Clémentine Fourrier highlighting the model’s memory efficiency, allowing users to reduce memory requirements by 75% without compromising inference quality.
Pushing the Limits of Large Language Models
TII’s AI Cross Center Unit selected Amazon SageMaker to create the Falcon 180B model, positioning itself as a leading technology enabler dedicated to innovative solutions. Falcon 180B is propelling the boundaries of large language models and advancing the field of AI in the UAE. Its exceptional capabilities and open accessibility have the potential to reshape multiple industries and ignite global- scale innovation.
Furthermore, Falcon 180B’s training has provided TII with valuable experience in high-performance infrastructure while bolstering their dataset resources for future research and development endeavors.
TII introduced its Falcon 180B model in September 2023 under the Falcon-180B TII License. Falcon 180B has surpassed contemporary models in various benchmarks with minimal fine-tuning. Amazon SageMaker JumpStart streamlines access to Falcon 180B, allowing customers of all sizes and industries to deploy and customize their own Falcon 180B model for tasks such as translation, question answering, summarization, or image identification.
Wojciech Bajda, Managing Director for Public Sector Middle East and Africa at AWS, remarked, “Collaborating with the Technology Innovation Institute to bring Falcon 180B to life using Amazon SageMaker has been a journey marked by numerous milestones – technological innovation, academic excellence, efficient training processes, and operational ease of use. Today’s milestone of Falcon 180B’s integration into Amazon SageMaker JumpStart is not only a triumph for TII but also a proud moment for technological innovation in the Middle East.”
With Falcon LLM 180B, organizations worldwide can harness its advanced technological capabilities, empowering researchers to drive AI innovation. For comprehensive information and a step-by-step guide on deploying Falcon 180B, refer to AWS’ detailed blog post: Falcon 180B Foundation Model from TII is Now Accessible via Amazon SageMaker JumpStart.
