MOJO: Decoding the Tumor with Multimodal AI

TL;DR: Integrating multiple biological data types gives oncologists a much richer picture of a patient’s cancer. However, effectively training AI on massive amounts of highly complex genomic data, without hitting computational bottlenecks, remains a major challenge. Enter MOJO: A multimodal AI model presented at ICML 2026, which jointly learns from bulk RNA-seq and DNA methylation data to outperform previously published models on cancer-type classification and patient survival prediction. Its hybrid U-Net transformer architecture delivers a 300x speedup over its purely transformer-based counterpart, and a mutual-information-based fine-tuning loss ensures it maintains diagnostic accuracy even when a modality is missing. Clinically resilient, biologically interpretable, and built for the real world.

Clinical settings are increasingly recognising the potential of AI to tackle some of the world’s most pressing and devastating diseases. High-throughput technologies have revolutionised molecular research, generating extensive genomic, transcriptomic, and epigenomic data with significant potential for personalised medicine and cancer diagnosis.

Omics, the large-scale analysis of biological systems and genetic profiles, sits at the heart of modern cancer research. To model the complexity of a patient’s disease, oncologists increasingly rely on two distinct modalities:

• Bulk RNA-seq: a quantitative estimate of mean expression of all genes across a tumor sample, capturing a high-dimensional readout of which genes are active.
• DNA methylation: the epigenetic modification of DNA sequences. These markers serve as critical regulatory mechanisms, functioning “on/off” switches that dictate transcriptional control and gene silencing.

Integrating multiple biological modalities into a single AI model has remained one of computational biology’s most persistent challenges. InstaDeep’s Multi-Omics JOint representation learning model, MOJO, presented at ICML 2026, was built to close this gap.

A bimodal foundation model that jointly learns from bulk RNA-seq and DNA methylation, MOJO outperforms previously published models on cancer-type classification and patient survival prediction, remaining robust even in the absence of a data modality.

Hybrid Architecture: How MOJO Solves the Memory Problem

At the heart of MOJO’s performance is a hybrid architecture designed to overcome one of the field’s most fundamental computational barriers: the memory problem. Pure transformer architectures have a fundamental limitation when applied to biological data: their attention mechanism scales quadratically with input length, meaning double the input, quadruple the memory. With over 17,000 genes per modality, and two modalities to process simultaneously, standard foundation models exhaust available memory before they can process the full biological picture, and computational costs rapidly become prohibitive.

MOJO preserves clinical utility by delivering the analytical power of a transformer without the memory overhead, through a U-Net style architecture that wraps the transformer in convolutional layers. Rather than feeding the full gene sequence directly into a transformer, MOJO progressively compresses high-dimensional genomic data into structured, manageable representations, without relying on any spatial ordering of genes. This is important because, unlike natural language or image data, gene expression and DNA methylation have no inherent positional structure; the model must learn biological meaning from the data itself, not from where features happen to sit in a sequence.

This was validated through a sub-variant called MOJO-ORDERED, in which genes were sorted by genomic coordinates. The absence of any statistically significant performance difference confirmed that MOJO learns the relevant biological structure directly from the data, proving that prior biological assumptions like gene ordering aren’t required to achieve high performance.The results speak for themselves: a 300x speedup over its purely transformer-based counterpart, with support for batch sizes up to 1,024 compared to just 4 to 16 for competing models.

How MOJO Works

MOJO’s training pipeline is fundamental to its performance, and follows three sequential stages, each building on the last. 

1. Pre-processing and tokenisation

Before the model can effectively learn, both modalities need to speak a common language. RNA-seq values are log-transformed, and DNA methylation values are averaged across all the sites associated with each gene, so instead of hundreds of thousands of individual measurements, the model works with one value per gene. This is a particularly notable finding: MOJO performs just as well using simple averages, implying that it learns the regulatory importance of different genomic regions directly from the data itself.

Once both data types are in the same numerical format, they are converted into tokens, the discrete units that language models process. MOJO does this by dividing the range of possible values into 64 bins and assigning each measurement to a bin, discretising continuous biological values into IDs the model can work with. Since gene expression and DNA methylation lack a fixed spatial sequence, gene embeddings act as positional encodings, helping the model keep track of individual genes.

2. Bimodal pre-training via masked language modeling

MOJO is pre-trained through masked language modeling, using data from the Cancer Genome Atlas (TCGA), spanning 33 cancer cohorts and 9,252 patient samples. This self-supervision technique trains the model to reconstruct randomly hidden inputs without requiring labelled examples. By masking 15% of tokens across both modalities, MOJO learns to reconstruct them, building an understanding of the underlying biological relationships between gene expression and regulatory markers across 192 billion tokens.

Critically, MOJO also addresses missing data through a dedicated variant, MOJO-EXTENDED-PRETRAINING, trained on an expanded dataset that explicitly includes samples with one missing modality. This prepares the model for real-world clinical settings where complete data cannot be guaranteed. In ovarian cancer subtyping, where only RNA-seq is available, MOJO-EXTENDED-PRETRAINING raises the weighted F1 score from 0.87 to 0.92, outperforming dedicated unimodal models and handily surpassing BulkRNABert, which achieves just 0.68 when given the same fine-tuning budget.

3. Fine-tuning for downstream clinical tasks

Once the model has learned the underlying biology through pre-training, it is fine-tuned for specific high-stakes clinical tasks spanning supervised classification, survival analysis, and zero-shot clustering:

• Cancer-type classification: pan-cancer classification across 33 TCGA cohorts, distinguishing between cancer types based on their joint RNA-seq and DNA methylation profiles
• Patient survival prediction: time-to-event analysis predicting the time from diagnosis to patient death, a critical tool for informing treatment decisions
• Cancer subtype clustering: in a fully unsupervised setting, MOJO’s embeddings cluster cancer subtypes without labelled examples, demonstrating that meaningful biological structure has been learned during pre-training
• Breast cancer sub-typing: PAM50 clustering distinguishing between Luminal A, Luminal B, Basal, and HER2 subtypes

To preserve the rich biological representations learned during pre-training, parameter-efficient fine-tuning is applied via IA3 for classification tasks. Rather than updating the full weight set, IA3 introduces small learnable vectors that make targeted, lightweight adjustments to activations in both the attention and convolutional layers, drastically reducing the number of trainable parameters needed while maintaining the integrity of the pre-trained model. For survival analysis, the model’s representations are used directly via layer probing, yielding results on par with parameter efficient fine-tuning, with only the task-specific head trained while the base model weights remain frozen, confirming that the pre-trained embeddings are already highly predictive.

Beyond task-specific fine-tuning, a mutual information auxiliary loss is applied during fine-tuning, penalising the model if its predictions change when a modality is removed and forcing it to maintain accurate outputs, regardless of whether it has access to one or two biological inputs.

Applying the mutual information loss allows MOJO to maintain a weighted F1 of 0.916 even when 100% of RNA-seq data is removed at test time, approaching the performance of dedicated unimodal models trained exclusively on methylation data (MethFormer: 0.931). This robustness is not a secondary feature. It is a core design principle, and one that makes MOJO uniquely suited to the realities of clinical deployment.

MOJO Encodes Known Cancer Pathways
MOJO not only performs well on benchmarks, it encodes known biology.Using SHAP values and gene set enrichment analysis (GSEA), a computational method used to determine whether a defined set of genes shows statistically significant differences between two biological states, the study shows that the embedding dimensions most predictive of the Luminal B breast cancer subtype are correctly enriched with known pathways: protein secretion, estrogen response, and heme metabolism. The model has learned the biological landscape, not simply pattern-matched to labels.

This interpretability is not incidental. It is evidence that MOJO’s representations reflect the underlying biology of disease, a pattern that emerges consistently through its architecture, pre-training, and fine-tuning, and a critical requirement for any model intended for clinical application.

MOJO aims to set a new benchmark across cancer classification, survival analysis, and zero-shot pan-cancer clustering. Its embeddings separate cancer types with significantly greater clarity than late integration approaches, demonstrating that joint modelling captures biological structure that sequential fusion cannot. This generalisation extends beyond the training data entirely: external validation on both the ICGC and TARGET datasets, a cohort held out from pre-training, confirms that MOJO performs consistently across platforms and patient populations, a critical requirement for real-world clinical deployment.

The clinical implications of MOJO’s performance hold the potential for significant impact. With its ability to distinguish disease subtypes and categorise high and low-risk patients, MOJO has the potential to enable more reliable patient stratification and better informed clinical decision-making.

MOJO’s results may mark a significant step towards a future in which multimodal AI models are a standard tool in clinical settings, enabling more reliable, data-driven personalised medicine. Looking ahead, the MOJO framework is designed to extend beyond two modalities, with future work targeting broader data types, relaxed modality alignment requirements, and further refinement of the mutual information approach. The ambition is a model that matches unimodal performance under any conditions, bringing multimodal AI one step closer to routine clinical deployment.

🚀Ready to discover MOJO? Explore the model checkpoints on HuggingFace, access the code on GitHub, and read the paper today!

Disclaimer: All claims made are supported by our research paper: Bimodal masked language modeling for bulk RNA-seq and DNA methylation representation learning unless explicitly cited otherwise.

FAQs