Artifical Intelligence News

Extreme weather events are becoming increasingly severe and frequent. From record-breaking heat waves to widespread flooding during hurricanes, the impact of such events on communities and economies is profound. The extreme weather phenomena cause $150 billion in damage annually in the U.S. alone. 

Hurricane Beryl recently swept through the U.S., causing an estimated $2.5 to $4.5 billion in insured damages and triggering prolonged power outages across Texas. These figures only scratch the surface, as the total economic impact is likely much higher. 

Without precise forecasting, communities face increased risks of loss of life and extensive property damage. It has become more important than ever to improve and accelerate climate prediction using the latest technologies. 

NVIDIA, the powerhouse driving the future of graphics and AI technology, has unveiled a new AI model called StormCast that could help predict weather events more accurately. It can play a crucial role in disaster planning and mitigation.

Developed in collaboration with Lawrence Berkeley National Laboratory and the University of Washington, StormCast is an advanced iteration of an earlier atmospheric forecasting model called CorrDiff. 

Designed to work as a zoom-in tool, CorrDiff allows researchers to input a dataset of weather events at a resolution of 25 kilometers. CorrDiff then enhances this data, increasing the resolution to more detailed 3 kilometers, allowing for precise analysis of smaller-scale atmospheric features. 

With the more advanced StormCast model, NVIDIA has added autoregressive capabilities that enable AI to study past weather events to predict future developments. The model’s training dataset included two and a half years\’ worth of climate data from the central U.S. 

Using StormCast researchers can predict mesoscale weather events, such as flash floods and long-lasting storms capable of inflicting extensive damage. Traditional methods for weather predictions, such as convection-allowing models (CAMs), often require thousands of atmosphere parameters to generate predictions. 

The autoregression capabilities allow StormCast to deliver hourly weather predictions up to six hours into the future. NVIDIA claims the StormCast is 10% more accurate than the U.S. National Oceanic and Atmospheric Administration (NOAA)’s state-of-the-art 3-kilometer operational CAM. NVIDIA also claims that StormCast is the first AI model that can predict moisture concentration and atmospheric buoyancy variables. 

At its core, StormCast relies on NVIDIA’s accelerated computing hardware to significantly boost both efficiency and speed in calculations. The AI-chip giant has also included the Earth-2 software suite with StormCast to provide meteorologists with weather forecasting algorithms and various tools for managing atmospheric data. 

NVIDIA is collaborating with The Weather Company and Colorado State University to test the new model and may expand its availability.

“Given both the outsized impacts of organized thunderstorms and winter precipitation, and the major challenges in forecasting them with confidence, the production of computationally tractable storm-scale ensemble weather forecasts represents one of the grand challenges of numerical weather prediction,” said Tom Hamill, head of innovation at The Weather Company. 

“StormCast is a notable model that addresses these challenges, and The Weather Company is excited to collaborate with NVIDIA on developing, evaluating, and potentially using these deep learning forecast models.”

Several other companies are exploring ways to augment weather forecast models. Google is working on a neural network model, called GraphCast, that can predict atmosphere events faster than traditional models. It claims GraphCast can deliver accurate predictions up to 10 days in advance.  

Microsoft has also introduced Aurora Atmosphere, a powerful weather prediction platform that uses.3 billion parameters and is trained on extensive datasets, providing highly accurate and detailed weather forecasts. 

While newer AI models offer significant computational advantages over traditional methods, researchers, including the NVIDIA team, caution against completely discarding older forecasting techniques. Instead, AI should be used to enhance and complement traditional approaches.

Related Items 

IBM’s New GPU-Driven Global Weather Forecasting System 

Cloud-Based Weather Network Launched 

Cloud for Clouds: ClimaCell Leverages Cloud HPC to Deliver Weather Micro-Forecasting 

 

Most people don’t think about just how miraculous the human brain is. This organ contains about 80 billion neurons, each of which is connected to as many as 10,000 other neurons. Mapping the neurons themselves is a challenging endeavor, but trying to understand the connections between them is nothing short of a herculean task.

While fully mapping the human brain will take many more years of hard work, scientists at Argonne National Laboratory are laying the foundation for future explorations. The project is led by Argonne\’s Nicola Ferrier, Senior Computer Scientist in the Mathematics and Computer Science Division.

To learn more about this amazing work, we spoke with Thomas Uram, a Computer Scientist in the Argonne Leadership Computing Facility, who is also working on the project.

“(The brain) is one of the most complex things on the planet,” Uram said. “It’s certainly the most complex thing in our bodies, and we don’t totally understand how it works. What we’re trying to do is reconstruct its structure and connectivity.”

While Uram’s curiosity in this work stems from a desire to uncover the unknown, there are also some important incentives to understanding the brain’s connections. Learning more could help researchers understand more about human behavior, as well as provide insight into neurologically degenerative diseases.

A Cubic Centimeter of Brain

Research that maps the connections within an organism’s nervous system falls under the umbrella of connectomics. Considering the complexity of the brain’s structure, the connectomics study Uram and his colleagues are pursuing focuses on samples of brain tissue that are a cubic millimeter in size.

These samples are prepared by taking thousands of 30-nanometer-thick slices of tissue that are residual human brain tissue removed during surgery. Then, the scientists mount them on a tape that goes off to be imaged by an electron microscope. Each section is imaged individually as a collection of tiles and then reassembled as a larger section.

Once these sections are fully reconstructed, they are then aligned with the neighboring ones so that features within them match up. Then, the researchers use a neural network to trace objects within that stack of images. Specifically, Uram stated that the team uses a neural network developed by Google called Flood-Filling Network (FNN) to do the reconstruction part.

FFNs are machine learning neural networks specifically designed for neuron segmentation in connectomics. FFNs are a specialized type of Convolutional Neural Network (CNN) designed to distinguish neurons from other objects in electron microscopy images. CNNs are often used in tasks related to images, such as separating an object from background (a cow from a field, for example), generating captions that describe the objects in an image, or even generating new images.

This same CNN approach is used in FFN, to separate neurons from each other and from other objects found in brain tissue. A main part of the challenge in this case is identifying the many neurons in a small tissue volume.

Even with such a relatively small sample, studying every connection is a major computational challenge.

That cubic millimeter of tissues imaged at a lateral resolution of four nanometers generates about two petabytes of data. As Uram explained, that’s a huge problem – even for the most powerful machines we currently have.

With the current neural networks the lab is using to segment objects, Uram and his colleagues could segment a cubic millimeter of tissue in a few days using the entirety of Aurora’s computing potential. What’s more, this issue becomes exponentially worse as the scientists look to scale up this research.

“Looking into the future, if we wanted to reconstruct a whole mouse brain – that’s a cubic centimeter of data,” Uram said. “It\’s a thousand times as much data. That would take about 3,000 days on all of Aurora. That’s pushing like 9 or 10 years, on all of Aurora. We won\’t have access to all of Aurora for ten years straight. So clearly, we need much more computing than we have available now.”

What’s more interesting here is just how much computing power we’ll need to map an entire human brain. Uram stated that a human brain is about 1,000x larger than the cubic centimeter of a mouse. That causes a 1,000x increase in compute need and would require all of Aurora’s resources for 3 million days straight.

What the Future Holds

Clearly, using all the resources of one of the most powerful computers in the world for 3 million days straight isn’t possible. Uram admitted that we’ll need to create more powerful machines before we can even begin to seriously consider mapping an entire human brain.

However, he also points that the solution here isn’t to simply build machines that are 3 million times larger than what we currently have.

“More likely is that we will see significant advances in the tech that we\’re using,” Uram said. “If we can significantly speed up the neural network in terms of segmentation, then I think we could do much better on the machines that we currently have and that we expect to have in the next generation or two of machines.”

Uram mentioned that at this point, most people are familiar with the types of errors that can be seen in models like ChatGPT. Those same sorts of errors exist when scientists are trying to segment fine-structured neurons. This creates a large amount of data that must be proofread by humans.

He specifically mentioned a different project that worked to map a fly\’s brain. These researchers estimated that the human time involved in correcting the fly segmentation is on the order of thousands of hours.

On top of cutting back on the time spent on human proofreading, scientists also have a storage problem that they’ll need to solve. Right now, the researchers are working with petabytes of data from the cubic millimeter brain samples they have. For the larger work they want to do, the storage requirements would go way beyond exabytes of data. Exactly how we store and move that data around will demand new innovations.

This is clearly a difficult task, and progressing toward the mapping of a full human brain will only present more obstacles. However, Uram seems up for the challenge.

“I have always been interested in the big questions of life,” Uram said. “How the brain works is a complex and vexing question. It’s a great unknown.”

In a recent paper,  the authors suggest that AI models collapse when trained on recursively generated data points from previous LLM models. Basically, the snake is eating its tail. Generative-AI has been successful in mining the Internet to create large LLM models that provide human-like answers to many problems. In particular, creating new text has been a forte of many LLMs. LLMs create a large amount of text from document summation to report generation. Inevitably, the LLM text will show up on the Internet and be scooped up for the next generation of LLM. The researchers in the paper suggest that this recursive training will lead to degraded models and, eventually, model collapse.

One suggested solution is to use synthetic data to train models. Another is to use LLMs to create synthetic data when access to large, diverse labeled datasets is limited. Synthetic data can mimic real-world data characteristics, which can improve data quality and increase the performance of custom LLMs (not the foundation models).

Plenty of HPC Data

While mainstream GenAI development has begun to scrounge for original data, HPC has no such issue. The practice of HPC has been creating numeric models that predict or mirror physical systems. From galaxies to proteins to F100, HPC has created clean synthetic solutions (data) since the beginning. As computing speeds have increased, so has model fidelity.

The ability of LLMs to find hidden relationships (or a “feature-ness”) in data was described in a previous HPCwire article.

A good example of using traditional HPC data for a foundational model is the Microsoft Aurora weather project. (Note: the Microsoft Aurora model is different than the Argonne Aurora Exascale system) Compared to the traditional numerical forecasting Integrated Forecasting Systems, the Aurora model provided a 5,000-fold increase in computational speed. Microsoft explains the Aurora approach and states (emphasis HPCwire):

“Aurora’s effectiveness lies in its training on more than a million hours of diverse weather and climate simulations, which enables it to develop a comprehensive understanding of atmospheric dynamics.“

Basically, they teach the LLM by using the outcomes of “weather physics” and then using what it learned to predict the weather. The success of foundational models like Aurora, point the way toward a sea change in HPC.

The traditional HPC approach is shown in Figure 2, where a model is often developed using the physics that describe the system of interest. Once developed, the model is run for a specific set of initial conditions. These models can take days, weeks, and even months to produce a solution. If a different set of initial conditions is used, the model must be re-run from the beginning.

With AI-augmented HPC, an LLM-like model is trained using existing data (e.g., simulated weather data or real data). The creation or collection of the training data can take weeks or months. In many cases, the “synthetic” HPC training data may already exist. Once trained, however, the model can quickly supply a solution to a set of initial conditions (“a query”) through inference instead of numeric computation. This process is shown in Figure 3.

The advantage of an AI-augmented approach is the flexibility of the model. When the initial conditions change, the model can infer the result instead of calculating from scratch. In the case of the Aurora foundational weather model, a new weather prediction can be found quickly without the need to grind through the mathematics of the traditional model. In addition, the foundational models can be fine-tuned to more prefigured solutions.

For example, the Microsoft Aurora Foundation weather model provides both accuracy and computational speed but offers remarkable versatility that is not available in traditional weather models. Aurora can forecast a broad range of atmospheric variables, from temperature and wind speed to air pollution levels and concentrations of greenhouse gases. In addition, Aurora’s architecture is designed to handle heterogeneous inputs and generate predictions at different resolutions and levels of fidelity.

No Free HPC Lunch

The capabilities of AI-augmented HPC models represent a new vista for simulating our physical world. There is no free lunch, however. With LLM models, the computation is up-front; it takes huge resources to train a model, but the resulting generalized inference can work for many problems variations. As mentioned, once you have your LLM-based weather model, you don’t need to compute tomorrow’s weather from scratch using traditional HPC models; you can ask the model to infer the weather, which can be just as accurate as “computing out” the traditional model. The more the model is used, the more the upfront computation is amortized. AI-augmented models may become more computationally efficient in the long run than the traditional approach. The HPC lunch should get cheaper and perhaps a bit more tasty. (And we need to make sure our tail stays off the menu.)

Importance of Good HPC Data

Training LLMs requires large amounts of data. In Data Science, clean data is essential for model generation and use. The often stated rule is, “Data scientists and engineers spend 80% of their time finding, cleaning, and organizing the data. The rest of the time is actual developing and running modes.”

The same is true for collecting data from the Internet for LLM models. Garbage-In Garbage-Out (GIGO) holds at all levels. In almost all cases, however, the synthetic HPC model data used to train LLMs does not suffer from such issues. By design traditional HPC model output data is well managed and often designed for further use, e.g. visualization.

Looking forward, the HPC community knows how to create good, clean synthetic data to train foundational LLMs for science and engineering. In fact, HPC has been creating such data for decades. It is somewhat fortuitous that many of the GPU-based systems that create the training data need AI-augmented HPC are the same systems that can create the foundational model.

Finally, there has been some concern that relying too much on foundational models may produce biased results. There is the issue of hallucinations and improper training. As AI methods and technology develop, this can be addressed. Essentially, we are showing the best models of reality we can create to an LLM and asking, “What do you see?” As our models and questions get better, we can expect the LLM to find relationships and insights that are invisible to us.

Folding Proteins the Smart Way

The recent Microsoft Aurora model is not the only AI-augmented success story. Google has essentially solved the thorny protein folding problem using AI in an application called AlphaFold. The protein folding problem involves computing the conformation of proteins to form biologically active structures. Solving the protein folding problem using traditional molecular modeling is a compute-intensive operation.

AlphaFold does not use traditional molecular dynamics to determine a solution. Instead, AlphaFold employs a deep learning algorithm that uses machine learning to predict the 3D structure of a protein from its amino acid sequence. AlphaFold used over 170,000 proteins from a public repository of protein sequences and structures to train AlphaFold. The program uses an approach similar to the GenerativeAI transformer but is not strictly an LLM. The AlphaFold AI model combines the desired folding sequence with multiple sequence alignment (MSA) statistics as inputs and outputs of a structure prediction. The initial training was reportedly conducted using between 100 and 200GPUs.

Work on AlphaFold began in 2018. By then, the structure of 17% of human presence was known. Thanks to AlphaFold, the 3-D structures for virtually all (98.5%) of the human proteins are currently known.

Chasing the Wrong Dog

With AI taking a bigger and possibly leading role in HPC, chasing a double-precision Top500 benchmark may not be the best way to gauge future HPC performance.

According to Rick Stevens, who leads the Aurora exascale efforts at Argonne National Laboratory, “It was a deliberate design decision to not use silicon for a matrix unit for double precision. We put that extra silicon into accelerating lower precision. In bfloat16, for example, we have a lot more performance. So that’s the technical reason”

Measuring AI performance has fallen to the MLCommons, an Artificial Intelligence engineering consortium, and their MLPerf suite of AI benchmarks.

For HPC, both traditional floating point (Top500) and low precision AI (MLPerf) will be important. At this time, the aims of these two benchmarks are somewhat orthogonal, and a new way of defining HPC performance is probably needed.

The path to scientific and engineering insight is about to change in a very big way.

Impacting over 55 million people worldwide, dementia presents a significant global healthcare challenge. It costs $820 billion every year, and the number of people affected by dementia is anticipated to triple over the next 50 years. 

Alzheimer\’s disease is the most common cause of dementia, accounting for 60–80% of cases. The complexity of Alzheimer\’s disease along with its slow and subtle progress makes early diagnosis difficult, resulting in delayed treatment and missed opportunities for patients and families to plan for necessary support.

Researchers from the University of Cambridge may have uncovered a breakthrough that can revolutionize Alzheimer\’s diagnosis. The researchers have developed a new tool that outperforms clinical tests in predicting the progress of Alzheimer’s disease. 

The AI tool can predict whether people with early signs of Alzheimer’s disease will remain stable or develop the condition in four out of five cases. This makes the tool three times more accurate than standard clinical markers. 

 “We’ve created a tool which, despite using only data from cognitive tests and MRI scans, is much more sensitive than current approaches at predicting whether someone will progress from mild symptoms to Alzheimer’s – and if so, whether this progress will be fast or slow,” said Professor Zoe Kourtz, Senior author of the research paper and Professor at the Department of Psychology at the University of Cambridge. 

Koutz also expressed that he believes the new tool has the potential to improve patient well-being by identifying individuals who need the most intensive care. It will also help reduce anxiety for patients predicted to remain stable. 

The study was funded by the National Institute for Health Research Cambridge Biomedical Research Centre, the Alzheimer’s Drug Discovery Foundation Diagnostics Accelerator, Wellcome, the Royal Society, the Alan Turing Institute, and Alzheimer’s Research UK. The Cambridge researchers also collaborated with a cross-disciplinary team from the University of Birmingham and the National University of Singapore.

Early diagnosis of Alzheimer’s is crucial as that is when treatment is most effective. However, early detection through traditional methods may not be accurate without the use of expensive and invasive tests such as positron emission tomography (PET) scans or lumbar puncture, which are not commonly available at most memory clinics. 

The AI model was developed using low-cost and non-invasive patient data, including structural MRI scans and cognitive tests, to analyze gray matter atrophy of over 400 individuals. 

The gray matter in the brain is composed of neuronal cell bodies crucial for performing various cognitive functions. Reduced density or volume of gray matter is often associated with neurodegenerative conditions such as Alzheimer\’s disease. 

The researchers trained and built a predictive prognostic model (PPM) that analyzes gray matter atrophy and other clinically relevant predictors such as cognitive tests. The predictions were compared and validated with independent real-world data from different memory clinics across countries. The findings revealed that the tool was successful in identifying individuals who went on to develop Alzheimer’s in 82% of the cases and correctly identifying those who didn\’t in 81% of cases. 

The model categorized Alchemier’s patients into three groups: those whose symptoms would remain stable (around 50% of participants), those who would progress to Alzheimer\’s slowly (around 35%), and those who would progress more rapidly (the remaining 15%).

The AI algorithm’s robustness was confirmed by further testing the model using patient data from 600 participants from the US and longitudinal data collected from 900 individuals from memory clinics in the UK and Singapore. 

The research team is confident that their AI model is applicable in real-world patient and clinic settings. To ensure the AI model is generalizable to a real-world setting, the researchers used routinely collected data from actual memory clinics and research cohorts. 

Looking ahead, the research team plans to extend the model’s application to other forms of dementia, such as vascular dementia and frontotemporal dementia, by incorporating other types of data such as blood test markers.  

AI has unlocked new possibilities in disease research, revealing insights that were previously inaccessible. It has been instrumental in enabling researchers to identify Alzheimer’s drug targets. Insilico Medicine and the University of Cambridge jointly published a paper outlining a novel AI-based technique to pinpoint specific proteins that a drug can interact with to treat the disease. These advancements highlight AI\’s potential to transform healthcare and lead to more reliable diagnoses and treatments for Alzheimer\’s.

Related Items 

GPU-Accelerated Deep Neural Nets Look for Cures that Already Exist 

Deep Learning Being Used to Detect Earliest Stages of Alzheimer’s Disease 

Oracle Unveils Clinical Digital Assistant to Reshape Interactions Between Practitioners and Patients

Stadiums Are Embracing Face Recognition. Privacy Advocates Say They Should Stick to Sports

While the industry acknowledges the need for robust security measures, research from PSA Certified suggests that investment and best practices are struggling to keep pace with AI\’s rapid growth.

The survey of 1,260 global technology decision-makers revealed that two-thirds (68%) are…

Google has unveiled its new range of Pixel 9 smartphones, emphasising their enhanced AI capabilities.

The company released the devices much earlier than usual, as Google typically presents new Pixel models in the autumn. However, compared to previous versions, the changes in the new…