hii

| On
July 16, 2023

Huawei Cloud develops AI weather system 10,000x faster than traditional

Share

HUAWEI CLOUD has achieved a significant breakthrough in weather forecasting with its latest AI technology, as detailed in a paper published in the prestigious scientific journal Nature. The paper outlines the development of the Pangu Weather AI model, which leverages deep learning and 43 years of data to create a highly accurate global weather forecast system.

What sets the Pangu-Weather model apart is its ability to deliver more precise predictions compared to traditional numerical weather forecast methods, while also significantly improving prediction speed. In fact, the AI model achieves a remarkable 10,000-fold increase in prediction speed, reducing the time required for global weather forecasts to mere seconds.

This groundbreaking development challenges the conventional belief that AI-based weather forecasts are less accurate than numerical forecasts. The HUAWEI CLOUD research team has successfully created an AI prediction model that surpasses the precision of traditional numerical methods.

The research paper, titled “Accurate medium-range global weather forecasting with 3D neural networks,” provides independent verification of the model’s capabilities. Notably, it marks a significant milestone, as it is the first time that employees of a Chinese technology company have served as sole authors of a Nature paper, according to Nature Index.

Over the past three decades, numerical weather forecasts have greatly improved in accuracy due to advancements in computing power. These forecasts play a crucial role in extreme weather event warnings and climate change predictions. However, the traditional numerical approach remains time-consuming. To address this, researchers have been exploring the use of deep learning methods to enhance prediction speeds. Yet, AI-based forecasting has generally lagged behind numerical forecasts in terms of precision for medium and long-term predictions, particularly for extreme and uncommon weather events like typhoons.

On a yearly basis, approximately 80 typhoons occur worldwide. In China alone, the direct economic losses caused by typhoons in 2022 amounted to 5.42 billion yuan, according to the China Ministry of Emergency Management. The earlier accurate warnings can be issued, the better preparedness and response measures can be implemented.

AI weather forecast models have been appealing due to their speed but have lacked precision for two key reasons. First, existing AI meteorological forecast models rely on 2D neural networks, which struggle to process non-uniform 3D meteorological data effectively. Second, repeated calls to the model for medium-range weather forecasts can result in cumulative forecast errors.

Pangu-Weather effectively addresses these challenges. During scientific trials, the model demonstrated higher precision than traditional numerical prediction methods for forecasts ranging from 1 hour to 7 days, while achieving a 10,000-fold increase in prediction speed. The model can accurately predict fine-grained meteorological features such as humidity, wind speed, temperature, and sea level pressure within seconds.

To process complex non-uniform 3D meteorological data, the model utilizes a 3D Earth-Specific Transformer (3DEST) architecture. By employing a hierarchical, temporal aggregation strategy, the model was trained for different forecast intervals (1 hour, 3 hours, 6 hours, and 24 hours), resulting in fewer iterations required for accurate predictions and a reduction in erroneous forecasts.

To train the model for specific time intervals, researchers utilized 100 epochs and hourly weather data samples spanning from 1979 to 2021. Each sub-model resulting from the training process required 16 days of training on 192 V100 graphics cards. Currently, the Pangu-Weather Model can complete 24-hour global weather forecasts in just 1.4 seconds on a V100 graphics card, representing a 10,000-fold improvement compared to traditional numerical prediction methods.

Dr. Tian Qi, Chief Scientist of HUAWEI CLOUD AI Field, an IEEE Fellow, and Academician of the International Eurasian Academy of Sciences, explained the significance of focusing on weather predictions, stating, “Weather forecasting is one of the most important scenarios in the field of scientific computing because meteorological prediction is a very complex system, yet it is difficult to cover all aspects of mathematical and physical knowledge. We are therefore delighted that our research has been recognized by Nature magazine. AI models can extract statistical laws of atmospheric evolution from extensive data. Currently, Pangu-Weather primarily focuses on the forecast system’s work and excels in predicting atmospheric state evolution. Our ultimate goal is to build a next-generation weather forecasting framework using AI technologies to enhance existing forecasting systems.”

Experts from Nature who reviewed the research by HUAWEI CLOUD praised the significance and quality of the work. They highlighted that the Pangu-Weather model is not only easy to download and run but also executes quickly on desktop computers. This accessibility allows the meteorological community to extensively test and evaluate the model’s prediction performance, fostering progress in the field. Reviewers noted that the results presented a significant leap forward compared to previous research, indicating a reevaluation of future forecasting models.

In May 2023, Typhoon Mawar gained global attention as the most powerful tropical cyclone of the year up to that point. Remarkably, Pangu-Weather accurately predicted the typhoon’s trajectory five days before it changed course in the eastern waters near Taiwan, according to the China Meteorological Administration.

To further advance leading AI weather forecast models, stable cloud environments, work suites, and corresponding operations and maintenance processes are essential.