Queimo/rem3di-cosmo10-combisolv-multitask

PCQMep2 → COSMOep10 → CombiSolv-QM multi-task continuation

All encoder, learned atomic/geometric preprocessor, and COSMO-head weights were initialized from the exact PCQM epoch 2 → COSMO epoch 10 checkpoint used in the preceding BigSolDB 2.0 evaluation. Byte-level hashes were checked against that evaluation protocol. The omitted embedding-denoising decoder alone was restored from PCQM epoch 2.

Completed 5 full joint epochs. Every retained COSMO training molecule and every retained CombiSolv-QM training pair occurs in every epoch; the shorter stream cycles to match the longer one. Per-epoch coverage and repeats are saved with each checkpoint. Frozen MACE inputs are cached in float32; both learned preprocessors, the REM3DI encoder and all three task heads train. The new solvation head is 1281 → 256 → 1 (SiLU), over the ordered solute640/solvent640/T÷298 concatenation.

CombiSolv-QM predicts signed solvation free energy at 298 K, in kcal/mol; it is not a direct solubility target. Its labels use training-only mean/std standardization and no logarithm. The existing COSMO log1p/ln transforms and training-set standardization are preserved exactly so its pretrained output head keeps the same meaning. The COSMO task averages sigma-bin MSE, area MSE and volume MSE with equal task-component weight.

REM3DI denoising uses Gaussian noise σ=0.3 on preprocessed atomic embeddings, the original decoder, original atom mask, original geometric/pair detach paths and the original non-detached clean target. The historical atom-denoising loss is divided by its 1088 channels to express it per feature; the unnormalized original value is also logged. No coordinate-noise objective is substituted.

The three task log variances start at −0.5 and are learned with Monroe’s exact objective sum(L_t / (2*exp(s_t)) + s_t/2). They are not softmax-normalized or constrained to sum to one. The separate decorrelation penalty is 8 * mean(CovOffDiag(COSMO embeddings), CovOffDiag(unique CombiSolv embeddings)), where each penalty is the squared off-diagonal sample covariance sum divided by D*(D-1). This uses Monroe’s released checkpoint coefficient of 8. Value and gradient comparisons against the pinned author source pass exactly.

Data and exclusions

The unchanged raw CombiSolv table contains 1,000,000 rows. Prepared memberships: {"auxiliary_holdout": 31497, "benchmark_overlap": 8430, "train": 940497, "validation": 19576}. COSMO memberships: {"new_holdout": 251, "train": 352926, "validation": 7082}. Raw rows remain available in the dataset; exclusions affect new training membership only.

The published BigSolDB held-out solute connectivity groups are excluded from all new training objectives and from both CombiSolv input roles. The original COSMO ESOL/FreeSolv quarantine is preserved; CombiSolv is protected for the requested BigSolDB holdout and is not claimed to preserve ESOL/FreeSolv quarantine. CombiSolv validation holds out 2% of eligible solute connectivity groups across objectives; the original COSMO validation groups are also excluded from CombiSolv training. Supplied source InChI defines CombiSolv molecular identity; the original SMILES/InChI mapping is retained. These exclusions cannot undo earlier exposure already present in the starting COSMO10/PCQM2 checkpoint, so this is a continuation comparison, not a claim of fully decontaminated foundation pretraining.

BigSolDB 2.0 frozen probes

The exact previously audited author test partition is reused: 9,686 measurements from 126 held-out solutes, with all 75,731 non-test measurements as Kumo context. Solute/solvent conformers are byte-identical to those used for the starting checkpoint. Kumo Large uses four estimators, seed 42, float16 inference, query chunks 128, and all 1281 concatenated features including temperature in Kelvin. Its transforms fit on training rows only. Each checkpoint is reported; test scores do not select a pretraining checkpoint or alter the protocol.

Model	MAE	MSE	RMSE	R²	Pearson	Spearman
PCQMep2 COSMOep10 + Kumo L (starting checkpoint)	0.5949	0.6141	0.7836	0.5269	0.7404	0.7326
Joint epoch 1 + Kumo L	0.5697	0.5547	0.7448	0.5726	0.7667	0.7550
Joint epoch 2 + Kumo L	0.5713	0.5599	0.7483	0.5686	0.7642	0.7536
Joint epoch 3 + Kumo L	0.5766	0.5622	0.7498	0.5669	0.7646	0.7544
Joint epoch 4 + Kumo L	0.5685	0.5444	0.7378	0.5806	0.7708	0.7599
Joint epoch 5 + Kumo L	0.5774	0.5629	0.7503	0.5663	0.7639	0.7545
Monroe + Kumo L	0.5774	0.5555	0.7453	0.5720	0.7697	0.7594
XGBoost-DFT	0.6210	0.6500	0.8060	0.4990	0.7220	0.7220
XGBoost-CFP	0.7490	0.8890	0.9430	0.3150	0.5920	0.5610
XGBoost-MMB	0.7100	0.8310	0.9120	0.3600	0.6160	0.5910
SolvBERT	0.8710	1.2310	1.1100	0.0520	0.2470	0.2220
MPNN	0.6680	0.7460	0.8640	0.4250	0.7240	0.6930
EquiformerV2	0.6910	0.8580	0.9260	0.3450	0.6260	0.6060
Solvaformer	0.6430	0.7000	0.8370	0.4600	0.6940	0.6770
MPNN+Charges	0.6290	0.6670	0.8170	0.4860	0.7210	0.7100

Lower MAE/MSE/RMSE and higher R²/correlations are better. Error units are log10 mol/L (squared for MSE). Published models use different pretraining and model classes. The original paper prose says 9,250 test rows; its committed prediction file contains 9,686 and reproduces all six published MPNN+Charges metrics. Its private neural train/validation subdivision is unavailable; Kumo uses the audited non-test complement. No CV standard deviation or significance claim is attached to this single released holdout.

CombiSolv source · Prepared dataset · Checkpoints · Probe W&B · Training W&B

Monroe UW source · Monroe covariance source · Solvaformer Table 1

Component ablations and reference probes

Three-epoch ablations, epoch-zero initialization, published Rem3Di, Monroe and all eight Solvaformer Table 1 baselines. All measured probe labels, test row IDs, six metrics and checkpoint identities were independently verified.