Deep learning is everywhere in astronomy, but most models live in silos—built for one data type, one survey, and one bespoke task. As a result, they must be retrained whenever these settings change. The AstronomIcal Omnimodal Network (AION-1) changes this: a single foundation model that spans multiple surveys and data types, producing a unified representation that transfers across domains and performs strongly even in low-data regimes. By learning shared astrophysical structure across instruments and modalities, AION-1 not only removes the need for retraining—it enables better performance, faster adaptation, and a deeper physical understanding from fewer examples.

What is AION-1?

AION-1 is a large-scale multimodal foundation model for astronomy that a wide variety of astronomical measurements into a single, unified model spanning everything from nearby stars to distant quasars and galaxies.

This approach addresses a longstanding challenge: integrating multiple heterogeneous datasets—spanning various instruments, measurement protocols, noise sources, and physical phenomena—into one coherent framework.

As a result, using AION-1 is straightforward:

1. One model for everything: no need to juggle separate tools for varying modalities.
2. Robust to missing data: useful predictions even when your dataset is incomplete.
3. Minimal extra training: plug in your data and retrieve high-quality, physically meaningful embeddings of uni- or multi-modal data for downstream tasks.

AION-1 integrates 39 different data modalities—multiband images, optical spectra, and various properties and measurements—into a single model usable for a wide range of downstream applications. It implements a two-step process: bespoke tokenization strategies that homogenize diverse scientific data, followed by multimodal masked modeling that learns how different observations relate, encouraging the model to learn the underlying physical relationships in the data. Astronomers can then leverage AION-1’s rich astrophysical understanding for a variety of downstream tasks.

What is a Multimodal Foundation Model?

In data science, a modality is a type of data, which can come in many forms. A multimodal model processes multiple forms at once (e.g., images, spectra, tabular measurements), as opposed to single-modal models that only handle one. Once trained, multimodal models streamline task performance by eliminating the need for separate models per modality, and they enable data fusion (using many modalities for a given task) and cross-modal generation (predicting one modality from another).

Think of how using multiple senses together—rather than one at a time—gives you a fuller understanding of an experience. Over time, your brain learns associations between how things look, taste, and smell, so if one sense is unavailable you can often infer the missing information from the others. Multimodal models work the same way: they learn joint structure across modalities, allowing them to fuse evidence or predict one modality from another.

How was AION-1 developed?

AION-1 was developed using a two-step process: tokenization and masked modeling.

Tokenization

Tokenization translates disparate data types into the same “language” so they can be compared and reasoned over. Data are transformed into tokens—small standardized units (an “alphabet”) that form the building blocks for modeling.

AION-1 uses modality-specific tokenizers: one for images, one for spectra, and several for scalar/catalog values. This standardizes and discretizes the data so that all modalities become consistent and mutually compatible. The tokenizers are modular, enabling independent development for different surveys or instruments while still producing tokens in the same unified AION-1 token space. In practice, each survey can contribute its own tokenizer tuned to its data idiosyncrasies; once tokenized, all streams become mutually compatible. This lets a single AION-style model reason jointly across heterogeneous datasets.

Masked Modeling

After tokenization, the next step is masked modeling: during training, the model randomly hides (“masks”) a subset of tokens and is trained to predict the missing content from the remaining context. This forces the model to learn statistical structure and dependencies rather than memorizing surface patterns.

If tokenization defines the vocabulary, masked modeling teaches the grammar. Given a partially observed sequence (e.g., “twinkle, twinkle, ___ star”), a well-trained model infers the missing token (“little”) from context. AION-1 extends this idea within and across modalities: it learns to reconstruct masked pixels, spectral bins, or photometric features from whatever complementary signals are available. Crucially, what is shared across modalities—and what the model ultimately learns to recover—is the underlying astrophysics that manifests differently in each data type but arises from the same physical processes.

Scale, Scope, and Size

• 39 distinct data modalities, including images, spectra, and scalar/catalog values.
• Trained on 200M+ observations drawn from The Multimodal Universe Collaboration et al. (2024) and five major surveys, Legacy Survey, Hyper Suprime-Cam (HSC), SDSS, DESI, and Gaia.
• Available in three sizes—300M, 800M, and 3.1B parameters—among the largest machine-learning models ever trained for astronomy.

AION-1 was trained as part of the Grand Challenge program on the new Jean Zay supercomputer in France, made possible through computational resources provided by GENCI at IDRIS (CNRS). This large-scale training effort leveraged the newly expanded H100 partition of Jean Zay to enable billion-parameter, multimodal modeling.

What can AION-1 do?

In practice, the primary way we envision AION-1 being used is as an encoder—it produces high-quality embeddings that capture deep astrophysical structure. These embeddings can then be used with minimal downstream adaptation (e.g., a small MLP or linear head) to perform a broad range of scientific tasks, often matching or exceeding specialized supervised models trained end-to-end. We demonstrate performance on a wide range of tasks:

• Photometric/Imaging/Spec-z estimation: measure distances to galaxies by estimating redshift from images, photometry, or spectra.
• Spectral super-resolution: enhance low-resolution spectra into high-resolution reconstructions.

• Galaxy property prediction: estimate stellar mass, age, metallicity, and star-formation rate.
• Morphology classification: classify galaxy shapes (spiral, elliptical, irregular).
• Structure segmentation: map galaxy substructures (spiral arms, bars, etc.).

• Stellar property prediction: predict stellar temperature, gravity, and chemical composition.
• Anomaly/rare-object discovery: shown here for three astronomical classes of decreasing prevalence—spirals, mergers, and strong lenses. Example candidates for a given query image are shown, along with rank by cosine similarity.

Open Source Release & Getting Started

AION-1 is free and easy to use. Everything you need to run the model:

• API & code: AION-1 API and Code
• Model weights: Hugging Face
• Tutorial: AION-1 Tutorial

– Sophie Barstein, Liam Parker

P.S. Thank you to Yann LeCun for beautiful astrophotography for our splash image!