human-mpra-gosai-2024-model

Model Description

This model is a multi-task regression model trained to predict MPRA log2 fold change values for genomic DNA sequences across 3 cell lines. It was trained by fine-tuning the Enformer model using the grelu library on the MPRA dataset from Gosai et al. 2024 (https://www.nature.com/articles/s41586-024-08070-z).

• Architecture: Fine-tuned Enformer (EnformerPretrainedModel)
• Input: Genomic DNA sequences (hg38)
• Output: Predicted log2 fold change for 3 cell lines
• Parameters: 230M

Output Tasks

• K562_log2FC
• HepG2_log2FC
• SKNSH_log2FC

Performance

Split	MSE	Pearson
Validation	0.3999	0.8911
Test	0.3392	0.8550

Metrics are averaged across the 3 cell line tasks

Training Details

Parameter	Value
Task	Regression
Loss	MSE
Optimizer	Adam
Learning rate	0.0001
Batch size	512
Max epochs	50
grelu version	1.0.4.post1.dev39

Repository Content

1. model.ckpt: The trained model weights and hyperparameters (PyTorch Lightning checkpoint).
2. 1_model.ipynb: Jupyter notebook containing the training logic, architecture definition, and evaluation loops.
3. output.log: Training logs.

How to use

To load this model for inference or fine-tuning, use the grelu interface:

from grelu.lightning import LightningModel
from huggingface_hub import hf_hub_download

ckpt_path = hf_hub_download(
    repo_id="Genentech/human-mpra-gosai-2024-model",
    filename="model.ckpt"
)

model = LightningModel.load_from_checkpoint(ckpt_path, weights_only=False)
model.eval()