📄sciexplor.com/cbm/articles/c… PIONEER combines ESM Cambrian embeddings with structure-aware graph learning to accurately identify PE/PPE proteins in Mycobacterium tuberculosis, achieving state-of-the-art performance. #AI #Bioinformatics #ProteinLanguageModels #Tuberculosis

Strict OOD Antigen-to-Antibody Retrieval with CDR-Aware Slot Late Interaction 1. The paper reframes antibody discovery as antigen-to-antibody retrieval: given an antigen sequence, rank a fixed candidate antibody library to enrich known binders in the top-K, matching early virtual screening better than pairwise bind/non-bind classification. 2. A key contribution is a strict antigen-cluster out-of-distribution (OOD) benchmark to reduce antigen homology leakage: antigens are clustered by MMseqs2 at min_seq_id = 0.8, and entire clusters are held out from training and checkpoint selection. 3. Benchmark scale and protocol: test has 849 antigen queries, a controlled corpus of 869 candidate antibodies, and 872 observed positive pairs. Unpaired antigen–antibody combinations are treated as unlabeled (not negatives), so evaluation focuses on Hits@K and enrichment vs an exact random baseline. 4. The proposed model Ab-CASLR is an asymmetric dual-tower retriever: antigens encoded by ESM-2 (150M), antibodies encoded by IgBert, both projected into a shared 128-d retrieval space and fine-tuned with small learning rates. 5. Core modeling idea: preserve antibody locality with CDR-aware slots. Instead of a single pooled antibody embedding, Ab-CASLR builds M=8 antibody “document slots” using slot attention constrained by CDR masks (Chothia H1–H3, L1–L3), injecting an inductive bias that specificity concentrates in CDR loops. 6. Scoring uses late interaction rather than global similarity: the antigen tower produces L=8 latent query summaries; a low-rank bilinear matcher computes compatibility between query summaries and antibody CDR slots; final score aggregates by taking, for each query summary, the max-compatible antibody slot (then averages across summaries). 7. Training uses a multi-positive InfoNCE retrieval objective (all in-batch observed binders are positives), plus auxiliary regularizers: contact-derived epitope/paratope supervision when available and a document-side slot diversity penalty to reduce redundant CDR slots. 8. Strict OOD results show early enrichment: Hits@10 = 7.42% with EF@10 = 6.28x over exact random screening (Random Hits@10 = 1.182% for a 869-antibody corpus). Hits@1 = 1.767% with EF@1 = 14.97x, indicating strongest gains at very small K. 9. Comparisons and diagnostics clarify what helps: Ab-CASLR beats k-mer homology transfer (Hits@10 5.53%) and global ESM2-ESM2 embedding similarity (Hits@10 3.29%). Ablations show global pooled dual-tower collapses (Hits@10 1.413%), removing the CDR mask hurts (6.360%), and replacing late interaction with pooled CDR similarity drops strongly (2.120%). Slot diagnostics show antigen-side summaries collapse to near-identical representations, while antibody CDR slots remain diverse—suggesting antibody-side CDR-local representation is the main effective mechanism, and antigen epitope grounding remains an open bottleneck. 📜Paper: biorxiv.org/content/10.64898… #Antibodies #ProteinLanguageModels #Retrieval #OutOfDistribution #ComputationalBiology #Bioinformatics #MachineLearning #VirtualScreening #ESM #Immunology

SA-MTP: A structure-aware framework for multifunctional therapeutic peptide annotation 1. SA-MTP targets a practical pain point in peptide discovery: short therapeutic peptides often have multiple bioactivities and strong conformational heterogeneity, making sequence-only multi-label predictors miss functions—especially in long-tail categories. 2. The core idea is to model peptide “structure as uncertainty” rather than a single fixed fold: SA-MTP uses probabilistic secondary-structure profiles (PSIPRED SS2, per-residue H/E/C probabilities) to represent flexible conformational tendencies. 3. It then builds an input-dependent, structure-aware residue graph by fusing (i) confidence-weighted secondary-structure similarity (local, uncertainty-aware) with (ii) PLM-derived residue–residue contact priors from ESM-2 (global, long-range). A fusion coefficient alpha balances the two signals (default alpha=0.5) and remains robust across ranges. 4. On top of this dynamic graph, SA-MTP applies a multi-head Graph Attention Network (GAT) to integrate sequence semantics (ESM-2 residue embeddings) with structure-guided neighborhood aggregation, capturing both local motifs and long-range dependencies in a peptide-specific way. 5. For multi-label output (15 therapeutic function categories), SA-MTP adds explicit label embeddings and label-to-sequence cross-attention so each function can attend to different residue regions, enabling label-conditioned evidence extraction rather than a one-size-fits-all pooling. 6. A lightweight FiLM-based classification head performs label-specific feature modulation (channel-wise scaling) to better separate heterogeneous functions while sharing a common backbone—improving discrimination without a heavy per-label model. 7. SA-MTP also addresses class imbalance at decision time via adaptive, per-label threshold optimization on the validation set (F1-optimized or MCC-optimized), improving recall and balanced metrics for sparse labels compared with a fixed 0.5 threshold. 8. Benchmarks (TPpred-LE dataset protocol; 90% identity reduction; 8:1:1 split; plus a stricter low-homology setting at 40% identity) show SA-MTP outperforming prior methods across label-level AUC/MCC/F1 and sample-level precision/recall; the gains are most visible for imbalanced or structurally complex categories. 9. Ablations quantify where improvements come from: adding structure-aware graph encoding improves macro MCC/F1/precision over a PLM-only baseline (e.g., macro F1 0.428 -> 0.434), and adding FiLM further improves the graph model (macro F1 0.434 -> 0.441), supporting complementary benefits from dynamic structure modeling label-aware modulation. 10. Interpretability: visualizations of SS2 similarity, ESM contact priors, and fused adjacency show graphs adapt to peptide types (coherent for helical peptides, modular for mixed motifs, sparse/local for high-entropy sequences). Integrated Gradients attention rollout highlight residue clusters consistent with known AMP characteristics (e.g., N-terminal hydrophobic anchoring and amphipathic helix-related regions), providing mechanistic clues beyond global similarity. 💻Code: github.com/LZW-TECH/SA-MTP 📜Paper: doi.org/10.1093/bib/bbag361 #ComputationalBiology #Bioinformatics #Peptides #TherapeuticPeptides #ProteinLanguageModels #GraphNeuralNetworks #MultiLabelLearning #Interpretability #DrugDiscovery #MachineLearning

AI-guided discovery for low-resource peptide engineering using evolutionary scale modeling 1. The study argues that a simple metric—cross-validation R²—can act as a practical early proxy for how well an active-learning peptide engineering campaign will perform, helping teams decide whether to invest in iterative wet-lab cycles. 2. It introduces SCARSE (Small-sample Classification And Regression Solution for low-resource peptide Engineering), a low-data framework designed for 20–500 labeled peptides, combining ESM-2 (650M) sequence embeddings with Gaussian process regression (for regression tasks) and extremely randomized trees (for classification). 3. A key design choice is to avoid PLM fine-tuning and avoid MSAs, reducing compute and making the approach more suitable for short peptides where alignments can be unreliable and evolutionary signal is limited. 4. Benchmarking spans 23 datasets: 10 substitution (DMS-style) datasets (39–87 aa), 10 indel datasets (39–71 aa), plus short antimicrobial peptides (5–20 aa, pMIC regression), cell-penetrating peptides (5–25 aa, classification), and toxic/non-toxic peptides (5–14 aa, classification). 5. Against a hand-engineered descriptor baseline (modlAMP global descriptors composition and helix-inspired features), SCARSE is consistently stronger on substitution and especially indel fitness prediction; the baseline often fails on indels (mean R² < 0), while SCARSE reaches mean R² ~0.55 with 20 samples and ~0.76 with 80. 6. On very short, compositionally diverse peptide datasets (AMPs and CPPs), SCARSE is often comparable to the descriptor baseline, supporting the interpretation that simple composition-level features can capture much of the signal when sequences are short and diverse rather than near-mutant neighborhoods. 7. Beyond overall test-set R², the paper emphasizes “extreme-value” utility for engineering: SCARSE improves top-ranked selection quality (e.g., Top-20 accuracy) on substitution datasets, aligning evaluation with how practitioners actually pick candidates to synthesize. 8. Active-learning workflow simulations (greedy top-k exploitation): start with 20 random peptides, then iteratively add the top 20 predicted peptides for 10 rounds (200 total). Across substitution datasets and short AMPs, SCARSE consistently beats random sampling at enriching true top-10% performers, in some cases achieving up to ~7x more high-value peptides at the endpoint than random. 9. The central practical takeaway: CV R² computed from small random subsets correlates strongly with active-learning endpoint enrichment (Pearson ~0.83–0.89). While 20 labeled peptides can already enable useful selection, around 50 labeled peptides are suggested as a more reliable minimum to estimate whether an active-learning campaign with SCARSE is likely to pay off. 💻Code: github.com/LeoAnd00/SCARSE ; github.com/LeoAnd00/SCARSE-r… 📜Paper: biorxiv.org/content/10.64898… #PeptideEngineering #ActiveLearning #ProteinLanguageModels #ESM2 #GaussianProcesses #AntimicrobialPeptides #ComputationalBiology #MachineLearning #Bioinformatics

Pepti-drift: Toxicity-repulsive drifting for antigen-conditioned discrete peptide generation 1. Pepti-drift is a one-step, antigen-conditioned peptide generator that explicitly tackles a core therapeutic-design tension: sequence features that improve binding often overlap with features linked to cytotoxicity/hemolysis, so “optimize binding only” can land candidates in a risky overlap zone. 2. The method reframes peptide design as controlled movement in a peptide embedding space: generated peptide latents are pulled toward antigen-matched binders (attraction) while being pushed away from toxicity-associated regions (repulsion), aiming for a “safe & active” region rather than a generic binder-rich area. 3. Architecture in one pass: frozen ESM-2 (3B) encodes antigen and peptide sequences; learned projection heads reshape embeddings into a compact normalized latent space; an antigen-conditioned latent generator samples an initial latent from Gaussian noise; a drifting block applies a single learned refinement step; a non-autoregressive Transformer decodes the final latent into a peptide sequence in parallel. 4. The drift field is decomposed into two forces: (i) attraction toward the matched positive binder latent for the given antigen, and (ii) repulsion away from a local neighborhood of “hard” toxic negatives (nearest toxicity-associated peptide latents), weighted by a kernel so closer negatives repel more strongly. 5. A key training issue is gradient competition because binders and toxic peptides can be close in representation space. Pepti-drift introduces a warm-up schedule: learn binding-oriented attraction first, then gradually ramp the repulsion coefficient, stabilizing learning under overlapping positive/negative distributions. 6. Dataset design supports antigen generalization: 20,547 antigen–peptide binding pairs (8,712 antigens; 13,910 peptides) plus 12,709 toxicity/hemolysis-associated peptides curated from DRAMP 4.0, ToxinPred 3.0, and Hemolytik 2.0; evaluation uses an antigen-level CD-HIT split at 90% identity to reduce train-test antigen similarity. 7. Warm-up is not cosmetic: without warm-up, drifted latents stay far from matched binders and sequence metrics degrade; with warm-up, the model achieves much better binder-proximal drift and improves uniqueness/diversity/novelty in test-time generation. 8. Efficiency is central: generating 64 peptides for each of 1,095 test antigens, Pepti-drift runs at ~0.302 ms/peptide (3,312 peptides/s), ~16.2x faster than PepMLM and ~1,092x faster than PepTune in end-to-end timing (including antigen embedding and decoding). 9. Sequence-level behavior: all methods yield 100% valid sequences, but Pepti-drift shows the strongest diversity profile (98.1% uniqueness; highest Shannon entropy) and near-zero cross-antigen reuse (0.27), indicating high antigen specificity rather than recycling the same peptides across targets. 10. Property predictions suggest a practical trade-off: Pepti-drift preserves target-related binding signal (PeptiVerse binding-affinity score below PepMLM but above PepTune) while consistently lowering predicted toxicity across length bins and reducing hemolysis risk in most ranges; an independent hemolysis predictor (HemoPI2) also reports lower mean hemolysis scores for Pepti-drift across all length bins. 📜Paper: arxiv.org/abs/2606.27824 #ComputationalBiology #ProteinDesign #PeptideDesign #GenerativeAI #MachineLearning #DrugDiscovery #Bioinformatics #ProteinLanguageModels #SafetyByDesign #Therapeutics

Prot2Prop: Structure-informed multitask protein property prediction 1 Prot2Prop presents a single multitask model that jointly predicts six protein developability properties (material production, solubility, temperature stability, aggregation propensity, expression yield, folding stability), aiming to replace fragmented one-model-per-property workflows. 2 The core design is parameter-efficient multitask adaptation: a frozen ProstT5 encoder is augmented with a shared residual adapter plus task-specific residual adapters, enabling shared transfer while still letting each endpoint specialize at the residue (token) level. 3 A key architectural choice is where specialization happens: task-specific adapters are applied before pooling, then each task uses attention-based sequence pooling to learn which residues matter most for that property (rather than mean pooling). 4 The framework supports heterogeneous outputs in one model: three binary classification heads (material production, solubility, temperature stability) and three regression heads (aggregation propensity, expression yield, folding stability), trained with masked losses so partially labeled proteins still contribute signal. 5 On held-out test data (Seed 1 reference checkpoint), classification performance reached AUROC 0.866 (material production), 0.870 (solubility), and 0.984 (temperature stability), with corresponding F1 scores 0.853, 0.749, and 0.931. 6 Regression performance on the same test set showed strong rank fidelity (Spearman): 0.861 (aggregation propensity), 0.732 (expression yield), 0.838 (folding stability). The paper notes that some endpoints preserve ranking well even when absolute scaling is imperfect. 7 Post-hoc calibration is treated as a first-class step: per-task affine calibration for regression reduced folding stability MAE from 0.677 to 0.486 (RMSE 0.879 to 0.641) without changing Spearman, suggesting a sizable fraction of remaining error can be calibration-related. 8 The authors document an iterative development trajectory and identify the biggest win: introducing the shared adapter plus task-specific residual adapters before pooling (version 2026-04-29) produced the strongest regression gains while keeping classification stable; more complex options (ranking-aware losses, learned uncertainty weighting, ensembles) were mixed or not clearly better. 9 Practical efficiency is emphasized: only ~3.19M parameters are trainable, while ProstT5 stays frozen. In an inference benchmark (100 sequences, length ≤72) Prot2Prop ran faster than TemStaPro and SaProt (8.95s vs 15.56s vs 18.97s) on the reported hardware setup. 💻Code: github.com/NeurosnapInc/Prot… 📜Paper: biorxiv.org/content/10.64898… #ProteinEngineering #ProteinLanguageModels #MultitaskLearning #Developability #DeepLearning #ComputationalBiology #Bioinformatics

Two-Stage Fine-Tuning for Protein Sequence Generation with Targeted Amino-Acid Composition 1 Protein language models are strong priors for sequence plausibility, but matching an explicit distributional target (a 20D amino-acid composition profile) is a different problem than optimizing a single scalar property. This paper frames AA-composition control as a constrained generation task: hit a target frequency vector while keeping sequences plausible, within length bounds, and diverse. 2 The motivating use case is synthetic feed-protein design, where nutritional value depends directly on amino-acid composition. The goal is not only “average composition near target”, but per-sequence compliance with practical constraints (e.g., essential AA coverage, handling near-zero target residues) so that individual candidates are usable for synthesis. 3 The core method is a two-stage alignment pipeline on top of ProtGPT2. Stage 1 is domain-adaptive fine-tuning on a curated in-domain set of natural proteins selected to be composition-close to the target (cosine similarity ≥ 0.95), length-filtered (100–500 AA), and de-redundified (<70% identity). This produces a target-specific FT prior that becomes a frozen reference policy for RL. 4 Stage 2 is iterative reward-weighted fine-tuning via RL with a KL anchor to the frozen FT reference. Each iteration: sample a candidate pool, sample an additional “diversity-pulse” pool with hotter decoding, score sequences, filter by length and identity (<0.85 with MMseqs2), re-inject an increasing fraction of diversity-pulse samples to counter late mode collapse, then update with a reward-weighted log-ratio objective plus KL regularization. 5 A key technical contribution is the composition reward design. The proposed “differentiated” composition term includes: asymmetric penalties that punish deficits in essential amino acids more than excesses, interchangeable residue pools (Met/Cys sulfur pool; Phe/Tyr aromatic-precursor pool), and a zero-target amplifier to strongly penalize residues whose target frequency is zero. The paper compares this to a symmetric ablation and to cosine-similarity and global L1-deviation baselines. 6 Results show a clear division of labor between stages. Domain-adaptive FT moves the mean composition close to target (e.g., on qA JSD 0.247 → 0.059; tolerance count N±30 3.70 → 9.17), but it does not reliably satisfy sharp per-sequence constraints. Adding RL closes that gap (qA mean JSD 0.032 ± 0.020; best run JSD 0.0044 with N±30 17.4 and all constraint indicators saturated at 1.00). 7 The paper evaluates two targets: qA (in-domain, includes two low-frequency residues with one exactly zero; residue identities anonymized) and qB (a published poultry-feed reference profile with all non-zero frequencies). Best runs reach near-perfect alignment on qB (best JSD 0.0008; N±30 20.0), consistent with qB being easier because it lacks zero-target residues. 8 Ablations support the need for the two-stage recipe. RL-only (starting from base ProtGPT2) underperforms FT→RL on all metrics in paired seeds and can exhibit reward-hacking collapse: one RL-only seed matched average composition but used a narrow AA palette, destroying essential-AA coverage and diversity. The FT reference provides a stable, composition-near manifold that makes the KL trust region effective. 9 The RL gains are not explained by simple oversampling from the FT prior. Best-of-500 from FT-only reaches JSD 0.018 (qA) and 0.0111 (qB), while best FT→RL runs reach 0.0044 and 0.0008 respectively, indicating RL moves the policy into regions not accessible by rejection sampling at that budget. 10 Sequence quality is checked with NetSolP predicted solubility, base ProtGPT2 log-perplexity, and ESM-2 pPPL, plus intra-pool diversity metrics. Composition alignment imposes a measurable plausibility cost (ESM-2 pPPL rises vs base), but policies remain far from an iid “random residues sampled from q” floor, and the biologically weighted differentiated reward tends to incur the smallest plausibility penalty among smooth-max variants. 📜Paper: arxiv.org/abs/2606.27939 #ProteinDesign #ProteinLanguageModels #GenerativeAI #ReinforcementLearning #ComputationalBiology #Bioinformatics #SequenceGeneration #MachineLearning

ProLoc: Text-guided Localization of Protein Functional Regions 1. The paper defines a new grounding problem: given a fixed protein sequence and a natural-language functional description, the model must localize the residue span(s) that support that function (domains, motifs, active/binding/PTM sites), enabling mechanistic interpretation beyond protein-level labels. 2. A key contribution is an InterPro-derived benchmark with explicit protein–text–region triples, covering both domain-level regions and short functional-site annotations, plus a unified span-level evaluation protocol (IoU@1, boundary MAE, Best@10, and multi-site recovery metrics). 3. To reduce homology leakage, the dataset is split at the protein-cluster level using MMseqs2 at 50% sequence identity, so similar sequences do not appear across train/valid/test; this makes residue-level localization evaluation more realistic. 4. The benchmark construction also addresses annotation imbalance: it uses description-group-aware identifier sampling (instead of raw-frequency sampling), filters weak/ambiguous descriptions (e.g., “unknown”, “putative”, DUF-like), and merges near-duplicate regions within the same protein–InterPro pair. 5. ProLoc is proposed as a text-conditioned localization model built on frozen raw ESM2-650M (protein) and frozen PubMedBERT (text), with a fusion backbone that injects pooled-text semantics via FiLM, then uses residue-to-text cross-attention for token-level evidence, followed by 1D convolution to smooth boundary signals. 6. The model provides two complementary outputs: a direct dense residue map for precise top-1 span localization, and an anchor-free proposal head (start/end/log-length) to generate ranked top-K spans for cases where one query corresponds to multiple separated functional regions. 7. On the held-out test set, ProLoc-Direct achieves the strongest single-region localization: 0.7730 IoU@1 overall (0.7938 domain IoU@1; 0.7141 site IoU@1) and improved boundary accuracy (B-MAE 65.77), outperforming window-based adaptations of ProtT5, ProtCLIP, ProtST, and others. 8. ProLoc-AF is particularly effective for visible multi-site functional-site recovery (without being trained with explicit multi-site group supervision): VM R@10 IoU50 = 0.9671 and VM All-Hit@50 = 0.9489, indicating strong top-K coverage when multiple sites exist for the same protein–function query. 9. Ablations suggest the gains are not from pooled-text conditioning alone (a FiLM-only control is much weaker), and the length branch in the anchor-free head is important for stable multi-span proposal generation (removing it sharply reduces multi-site recovery). 💻Code: github.com/ShiDeng7rz/Proloc 📜Paper: biorxiv.org/content/10.64898… #ComputationalBiology #Bioinformatics #ProteinFunction #ProteinLanguageModels #NLP #Grounding #InterPro #ESM2 #PubMedBERT #MachineLearning

Residue-Level Attributions in Protein Language Models Do Not Recover Allergen Epitopes 1 Protein language models can be very accurate at protein-level allergenicity prediction, but this study finds their residue-level explanations (saliency/attributions) do not meaningfully recover experimentally annotated allergen epitopes—highlighting a gap between prediction and biological mechanism. 2 The authors introduce an epitope-grounded residue-level benchmark to quantify “immunological faithfulness” of explanations: given IEDB-derived epitope masks on allergenic proteins, do high-importance residues coincide with known epitope regions? They evaluate with AUROC, AUPRC, and Precision@k under strong class imbalance. 3 Three allergenicity classifiers are tested: (i) Frozen ESM-2 with attention pooling MLP head, (ii) a multi-task (MTL) ESM-2 variant adding an auxiliary residue-level epitope head, and (iii) DeepPlantAllergy retrained to avoid leakage. All achieve strong protein-level performance (AUROC ~0.96–0.97 on 1,377 held-out sequences). 4 Despite strong protein-level AUROC/F1/MCC, residue-level attribution signals from the classification heads (Integrated Gradients, occlusion, attention weights, Grad×Input, SmoothGrad-IG) are near-random or worse in epitope alignment across metrics on 61 held-out epitope-annotated allergens. Quantitatively, “looks plausible” heatmaps do not translate into epitope localization performance. 5 A key conceptual contribution is separating two notions: model faithfulness (does the highlighted residue set causally affect the model output?) versus immunological faithfulness (do highlighted residues correspond to immune-recognition sites?). The paper shows these can strongly diverge. 6 Integrated Gradients passes a functional masking test: masking the top-ranked IG residues reduces predicted allergenicity significantly more than masking random residues (e.g., at k=40% of residues, strong paired significance). So IG identifies residues the model relies on—just not epitope residues. 7 Multi-task learning is used as an intervention/diagnostic: adding explicit residue-level epitope supervision yields an auxiliary residue head that does learn epitope signals, but it does not improve protein-level classification and does not make classification-head attributions more epitope-aligned. Even unfreezing the top ESM-2 layer in an exploratory MTL variant does not change the qualitative outcome. 8 Saturation mutagenesis suggests what the classifiers may be using instead: sensitivity patterns are structured by physicochemical/compositional properties (charge/polarity/hydrophobicity/aromaticity) rather than epitope-specific residue identity. Substitutions that change residue class (e.g., charge-altering) tend to have larger effects than within-class changes, consistent with global biophysical cues. 9 Mutagenesis further indicates non-epitope residues can be at least as influential as epitope residues: across all 61 allergens, mean absolute mutational effect is marginally larger outside annotated epitope regions, arguing against epitope-localized decision rules. 10 Practical implication: residue-level importance maps from allergenicity classifiers may help audit models, but should not be treated as immunological explanations for safety screening or for designing hypoallergens without quantitative epitope-grounded validation; the work motivates benchmarks and models with stronger structural/immunological priors. 📜Paper: arxiv.org/abs/2606.22181 #ProteinLanguageModels #Interpretability #Allergy #Allergenicity #Epitopes #IntegratedGradients #Bioinformatics #ComputationalBiology #MachineLearning

Multi-Scale Machine Learning for Antibody-Antigen Binding Affinity Prediction Using Deep Mutational Scanning and Structural Features 1. The study tackles a key failure mode in antibody–antigen (Ab–Ag) affinity ML: models can look good under standard CV but generalize poorly to unseen complexes. The author evaluates with leave-one-complex-out deep mutational scanning (LOCO-DMS), a stricter setting designed to expose leakage and overfitting. 2. Core idea: a multi-scale feature fusion framework integrating multiple “views” of a mutation. Descriptors span physicochemical mutation deltas, 3D structural interface context, ESM-2 protein language model embeddings, and biophysical SASA/ΔΔGfold-style features (with optional GNN interface embeddings explored). 3. Benchmark focus: AbAgym LOCO-DMS on 36,541 interface mutations from 68 DMS experiments across 13 pathogens. Under this realistic generalization test, no single modality beats the majority baseline on its own; performance emerges only when modalities are fused. 4. Best single-model result under LOCO-DMS comes from an optimized 93-feature representation (38 physicochemical 34 structural 4 ESM-2 17 SASA/ΔΔG). Gradient boosting with 300 trees reaches MCC = 0.206 (accuracy = 0.696; balanced accuracy = 0.601; AUC = 0.657). 5. A notable negative result is that learned graph embeddings can hurt under LOCO-DMS: adding GNN/GAT representations introduced noise/overfitting to per-complex topology. Removing GNN features improved MCC from 0.172 (231 features) to 0.206 (93 features), highlighting that “more representation capacity” is not automatically more transferable. 6. The paper introduces an ensemble confidence protocol tailored to LOCO-DMS: train multiple classical models (HGBT, RF, GBM) and use ensemble disagreement (σens) as uncertainty. Filtering to strong-effect mutations plus high-confidence predictions yields MCC = 0.374 with 83.5% accuracy at 25.5% coverage (5,046 mutations). 7. A physics-grounded interpretability point: the author quantifies a “Boltzmann ceiling”. In AbAgym, 45.9% of mutations are near-neutral (within ~kBT noise floor), implying an upper bound on attainable MCC of ~0.473 even for an oracle that perfectly predicts strong effects but guesses neutral ones. The confidence-filtered protocol reaches 79.1% of this ceiling. 8. Deep learning is benchmarked under LOCO-DMS (five architectures). A SelfAttention model matches histogram gradient boosting (MCC = 0.200), but none surpasses GBM (0.206), suggesting the bottleneck is not model class but the limited transferable signal under stringent OOD evaluation. 9. Transfer across pathogens remains unresolved: cross-pathogen training/testing on AbAgym averages 46.7% accuracy (vs ~83.5% within-pathogen), indicating models often learn pathogen/experiment-specific patterns rather than universal binding principles. 10. Practical takeaway for antibody engineering: use multi-modal fusion plus calibrated confidence/coverage tradeoffs. The work argues that aggregate “low MCC” should be interpreted in the context of thermodynamic/experimental noise limits, and recommends confidence-stratified deployment rather than unconditional predictions. 📜Paper: biorxiv.org/content/10.64898… #ComputationalBiology #Bioinformatics #MachineLearning #ProteinEngineering #Antibodies #Immunology #StructuralBiology #DeepMutationalScanning #ProteinLanguageModels #UncertaintyQuantification

TCRBinder: Unified pre-trained language model with paired-chain synergy for predicting T-cell receptor binding specificity 1. TCRBinder models TCR specificity using full-length paired TCR alpha and beta chains together with both peptide and HLA context, aiming to better reflect the cooperative recognition unit that binds peptide-HLA complexes. 2. Architecture: a four-branch design routes each component through a dedicated encoder (Roformer for TCR alpha and beta; ESM2 for peptide and HLA pseudo-sequence), then fuses representations with a multi-fusion CNN (MFCNN) and an MLP head to predict binding. 3. Key idea: paired-chain synergy is explicitly learned rather than approximated from single-chain (often CDR3beta-only) inputs, improving robustness when generalizing across epitopes, donors, and datasets. 4. Training data: 12,827 experimentally validated peptide-HLA-TCR (PHT) binders aggregated from VDJdb, IEDB, OTS, and McPAS-TCR, plus ~5x in silico negatives built via peptide-fixed mismatching with safeguards against peptide similarity to reduce false negatives. 5. Pre-training: separate Roformer language models for TCR alpha and beta were trained with masked amino-acid modeling on ~5.3 million unlabeled TCR sequences, producing embeddings that cluster by epitope more clearly than randomly initialized models (UMAP analysis). 6. Benchmark performance (PHT task): TCRBinder reports AUC-ROC 0.911 and AUPR 0.791, outperforming STAPLER, NetTCR v2.2, MixTCRpred, and EPACT under matched training/evaluation settings; it also supports pTCR prediction (AUC-ROC 0.886, AUPR 0.745). 7. Generalization tests: on unseen-epitope evaluation, performance remains above chance (AUC-ROC ~0.599), and stricter controls that fix peptide-HLA while shuffling TCRs still retain signal (AUC-ROC ~0.572), suggesting predictions are not explained solely by HLA or pHLA identity shortcuts. 8. External datasets: on an independent VDJdb test set, TCRBinder achieves AUC-ROC 0.712 and AUPR 0.430; on a clinically motivated SARS-CoV-2 repertoire with unseen peptides, it reports AUC-ROC ~0.696 and AUPR ~0.302, with up to 16.3% AUC-ROC improvement over leading alternatives in that setting. 9. Biological relevance: predicted binding scores correlate positively with antigen-driven clonal expansion in 10x Genomics single-cell pHLA capture data across four donors (with those donors excluded from training), supporting that model scores track in vivo selection trends. 10. Interpretability and mechanism: saliency/perturbation analyses plus alanine scanning highlight central CDR3beta positions as key determinants; hotspot residues (e.g., R98/S99/S100 in a classic HLA-A*02:01 GILGFVFTL complex) align with Rosetta ddG and structural contact geometry, and predictions agree with deep mutational scanning affinity trends from TRAIT for JM22 and A6 TCR variants. 💻Code: github.com/dongweihe/TCRBind… 📜Paper: doi.org/10.1371/journal.pcbi… #ComputationalBiology #Immunoinformatics #TCR #DeepLearning #ProteinLanguageModels #Transformers #Neoantigen #CancerImmunotherapy #VaccineDesign #SARSCoV2

DeepCDS: Ab initio coding sequence prediction in prokaryotic short reads 1 DeepCDS is presented as an ab initio CDS predictor designed specifically for short, fragmented prokaryotic reads where assembly is difficult and reference mapping can fail (unknown origins, novel taxa, sequencing errors). 2 The central idea is to bring protein language model signal into a nucleotide-level task: each of the 3 reading frames is translated, embedded with ESM-2 (8M), and combined with nucleotide one-hot codon features to decide whether each codon position is coding and where CDS boundaries lie. 3 Architecturally, it encodes each reading frame independently (shared weights), then performs joint structured decoding across all three frames using a linear-chain CRF over a shared state space of biologically valid cross-frame label combinations, enforcing consistent transitions around starts/stops and frameshifts. 4 The model is trained end-to-end on simulated 300 bp reads from 1,125 complete prokaryotic genomes, partitioned by taxonomic family to reduce leakage: 813 genomes for training, 97 for validation, and 215 for testing (phylogenetically diverse, spanning GC-content ranges). 5 Three practical model variants are provided to match data quality: DeepCDS N (noise-free), DeepCDS S (substitutions), and DeepCDS S I (substitutions indels, also predicts indel positions). The paper recommends DeepCDS S as a default for modern short-read data. 6 In benchmarking against MetaProdigal and FragGeneScan (via FragGeneScanRs) across multiple read lengths and Illumina-like error regimes, DeepCDS consistently improves CDS detection and codon-level classification, with gains increasing under harsher error conditions and at challenging lengths. 7 Boundary accuracy is a major focus: DeepCDS improves start and stop codon localization, including on very short reads where context is minimal. Example reported: on 75 bp error-free reads, median start-codon F1 rises from 0.152 (MetaProdigal) to 0.438 (DeepCDS N); under 300 bp stress-error, start-codon F1 is 0.771 (DeepCDS S) vs 0.469 (MetaProdigal). 8 DeepCDS is evaluated for robustness beyond standard assumptions: it generalizes across phylogenetic diversity, shows reduced dependence on genomic GC-content when trained with noise (S and S I), and remains strong on organisms using an alternative genetic code (e.g., Mycoplasmataceae where TGA encodes Trp), despite translating inputs with the standard table during inference. 9 DeepCDS extends into “ultra-short fragment” territory below existing tool limits (MetaProdigal and FragGeneScan minimum CDS length thresholds). When retaining CDS fragments down to 30 bp, DeepCDS retains meaningful signal (e.g., 60 bp error-free CDS-level F1 reported as 0.813 for DeepCDS N). 10 Ablations indicate complementary contributions: codon-only and pLM-only models both work, but the combined model is best, especially at short lengths; pLM features particularly help start-codon identification, consistent with protein-language priors about N-termini and protein-like composition. 📜Paper: biorxiv.org/content/10.64898… #metagenomics #geneprediction #bioinformatics #deeplearning #proteinlanguagemodels #prokaryotes #shortreads #ESM2 #computationalbiology

Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization 1. The paper proposes an unsupervised post-training framework to steer protein language models (PLMs) for controllable generation without wet-lab feedback or curated preference datasets, targeting the supervision bottleneck in biomolecular design. 2. Core idea: use task-agnostic proxy rewards that correlate with “prompt following” (e.g., functional keyword recovery) across models and sampling temperatures. The rewards combine intrinsic uncertainty signals from the PLM with extrinsic semantic-consistency signals computed from an external protein representation model. 3. Intrinsic reward family: sequence-level predictive entropy (and length-normalized entropy) under the reference policy, used as a proxy for model uncertainty. Empirically, entropy’s correlation with true controllability flips with temperature: low temperatures can be over-confident/degenerate; high temperatures can be under-confident/incoherent. 4. Extrinsic reward family: semantic distance in embedding space using an off-the-shelf representation model (ESMC). For a generated protein y under prompt x, the reward is the negative expected distance between q(y) and embeddings of other samples from the same prompt. This is designed to penalize “statistical outliers / hallucinations” and scales as O(N) embedding calls rather than O(N^2) pairwise checks. 5. A key empirical observation is a “critical temperature” where base-model performance peaks (about T≈0.7 for ProGen2-small and T≈1.0 for ESM3 on their tested Func2Seq setups). Around this point, intrinsic uncertainty metrics become least predictive, while certain embedding-consistency metrics peak—motivating temperature-aware reward selection. 6. Data construction recipe: multi-temperature sampling (T in {0.3, 0.5, 0.7, 1.0, 1.5}) to build an offline dataset of prompt-response-reward tuples. To reduce noise, for each prompt they sample 64 sequences and keep only top-4 and bottom-4 by proxy reward, yielding natural positives/negatives for preference-style objectives. 7. Two offline optimization algorithms are introduced to maximize a classical KL-regularized RLHF objective induced by these proxy rewards: Soft Reward Optimization (SRO) for continuous rewards and Binarized Reward Optimization (BRO) for noisy settings where only binary “good/bad” labels are trusted. 8. SRO: directly optimizes an importance-weighted objective that matches the model’s implicit reward to the explicit proxy reward, avoiding batch-normalized “weighted DPO”-style approximations. The paper provides a closed-form optimum showing how the updated policy reweights the reference policy by exp(r/β) (up to a multiplier). 9. BRO: treats reward optimization as binary classification against the reference policy via a logistic loss on σ(log πθ/πref). It can be viewed as a β→0 limit of soft reward optimization and as a zero-sum game where πθ learns to upweight positive samples and downweight negative samples relative to πref. 10. Results on a new compositional out-of-distribution benchmark (7 Pfam families; prompts combine a family tag with the first 10 residues from another family): BRO and SRO outperform DPO and KTO across temperatures and model scales (151M and 764M ProGen2 variants), and in some regimes approach or match an oracle trained with ground-truth labels. They also improve pass@k coverage mainly by boosting success at small k, suggesting better prioritization of prompt-aligned generations. 📜Paper: arxiv.org/abs/2606.18961 #ProteinDesign #ProteinLanguageModels #ComputationalBiology #MachineLearning #RLHF #UnsupervisedLearning #GenerativeModels #Bioinformatics #FoundationModels #Arxiv

Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment 1. The paper introduces LOGICA, a multimodal training framework that aligns biological foundation models directly in output-logit space, so the model’s native per-token probability interface is preserved while becoming context-conditioned (e.g., protein conditioned on ligand; TCR conditioned on peptide; mutations conditioned on drug). 2. Core idea: instead of CLIP-style alignment of pooled embeddings or adding task-specific heads, LOGICA defines compatibility as site-averaged conditional token log-likelihood. Matching and ranking are optimized with contrastive objectives (InfoNCE / Bradley–Terry) where the “score” is literally the contextualized token likelihood produced by each backbone’s original token head. 3. Key technical mechanism: native-head-preserving cross-modal adapters. Two pretrained encoders are coupled with bidirectional cross-attention adapters, and the adapter update is added back through a gated residual (near-zero init) so early training stays close to the pretrained model, while still allowing context to reshape token distributions. 4. LOGICA is designed to work even when modalities have distinct vocabularies and tokenizers (e.g., amino acids vs SELFIES tokens). Because alignment happens in each model’s own logit space, it does not require a shared embedding space, shared decoder, or shared tokenizer across modalities. 5. A main biological motivation is mutation-local variant ranking. LOGICA uses a context-conditioned likelihood-ratio score over only mutated sites (mutant token likelihood vs wild-type token likelihood under the same context). For comparing variants that share the same mutated positions, the wild-type terms cancel exactly, reducing pairwise ranking to comparing contextualized mutant-token likelihoods at perturbed sites. 6. Bidirectional scoring is supported when both modalities have token heads: score x|y and y|x can be combined with a learned convex mixture to handle calibration differences (length, entropy scale, vocabulary size) across modalities, while still remaining interpretable as token-level likelihood contributions. 7. Protein–ligand experiments: ESM-2 (protein) is paired with SELFormer (SELFIES ligand LM). LOGICA is pretrained on ~20M BindingDB measurements with anchored negatives (partner swaps and perturbations), then evaluated on DTI benchmarks (DAVIS, BindingDB-test, BioSNAP). Under controlled ablations, LOGICA outperforms (i) latent contrastive alignment using pooled vectors and (ii) conditional MLM-style adaptation without explicit mismatched negatives. 8. Drug-conditioned resistance prediction (protein DMS under drugs): LOGICA fine-tunes to rank single mutations using only mutated-site logits. On held-out-gene single-mutation resistance panels, latent-space baselines stay near random (~0.55 AUC), while LOGICA reaches ~0.65 AUC; in multi-oncogene panels it is consistently strongest among contextualized sequence-model variants and improves Spearman/AUC versus matched latent fusion approaches. 9. TCR–peptide experiments: TCRLang (paired CDR3α/β LM) is coupled with ESM-2 (peptide). LOGICA is trained with synthetic negatives from point mutations (reflecting that most local perturbations disrupt binding) and evaluated zero-shot on TCR–peptide deep mutational scanning. It shows strong directionality: TCR-anchored scoring is best for peptide-variant ranking; peptide-anchored scoring is best for TCR-variant ranking; a dual-anchor mixture balances both. 10. Because logits are preserved, LOGICA enables token-level interpretation without extra attribution heads: cross-modality dependency maps are computed by perturbing a token in one modality and measuring how the predicted distribution changes at positions in the other modality. On crystallized TCR–pMHC structures, these dependency scores are enriched for true contacts (notably stronger for peptide–CDR3β), connecting the learned contextual likelihoods to structural interaction signals; the same interface can also support context-conditioned generation via Gibbs sampling. 💻Code: anonymous.4open.science/r/lo… 📜Paper: arxiv.org/abs/2606.18703 #ComputationalBiology #Bioinformatics #ProteinLanguageModels #ContrastiveLearning #MultimodalAI #DrugDiscovery #Immunology #TCR #ProteinEngineering #VariantEffectPrediction

Analysis of Phylogenetic Signal in Protein Language Model Embeddings 1. The study tests a common zero-shot assumption: whether distances between protein language model (PLM) embeddings can be treated as evolutionary distances for phylogenetic tree inference (topology branch lengths), without any task-specific training. 2. Main finding: phylogenetic signal is largely lost when each protein is compressed into one fixed-size vector (mean-pooled residue embeddings or a CLS token). Trees inferred from these sequence-level embeddings show poor topology recovery and poor branch-length fidelity on 500 curated PANTHER families. 3. The key methodological twist is to avoid “one-vector-per-sequence” and instead accumulate distances across aligned residue-level embeddings. When site information is preserved and distances are aggregated position-by-position (or via small windows), topological accuracy improves substantially. 4. Among tested models, MSA-aware representations matter most. MSA Pairformer consistently outperforms single-sequence PLMs (ProtT5, ProtBERT, E1, ESM-C) for topology recovery when using residue-level, alignment-aware distance accumulation. 5. Best-performing zero-shot configurations use site-wise aggregation (window size = 1) and also benefit from selecting informative alignment positions. A sorted-coverage strategy that prioritizes high-entropy (Shannon) sites can reach very low relative RF error versus an NJ-on-MSA baseline. 6. The paper explores three aggregation families for residue-level embeddings aligned to an MSA: (i) sliding-window aggregation (local context), (ii) sorted coverage (rank sites by informativeness and accumulate), and (iii) weighted subset consensus (sample site subsets, infer multiple trees, then form a consensus). Consensus can work but often introduces multifurcations that complicate comparison to bifurcating references. 7. Critical limitation: even when topology improves, embedding-space distances do not reliably reproduce evolutionary branch lengths. Branch-length distributions from PLM-derived distances are distorted (excess of short branches and non-smooth / multimodal patterns), suggesting embedding distances do not scale linearly with divergence. 8. Benchmarking includes a model explicitly trained for phylogenetic distances (Phyloformer). Residue-level accumulation with MSA-aware PLMs can approach Phyloformer on topology in some settings, but Phyloformer better preserves branch lengths—highlighting the value of task-specific training for evolutionary distance calibration. 9. The conclusions are supported on both empirical data (PANTHER gene families with reference trees) and controlled simulations (SimPhy gene trees with duplication/loss ILS; AliSim sequences under WAG G4 indels). Simulations reinforce the same message: topology can be recovered, branch lengths remain problematic. 10. Practical takeaway: naive “embedding cosine distance → neighbor joining” is not a reliable phylogenetic pipeline. The phylogenetic signal is present at the residue level but is destroyed by pooling; future work should focus on recovering residue-level evolutionary signal without relying on MSAs, and on calibrating embedding-derived distances to meaningful branch lengths. 💻Code: github.com/brendonsa/phyloem… 📜Paper: doi.org/10.1038/s41598-026-5… #ComputationalBiology #Phylogenetics #ProteinLanguageModels #Bioinformatics #RepresentationLearning #Evolution #MSA #MachineLearning

MoE-Bind: Guiding De Novo Protein Binder Generation with Sparse Experts 1. The paper introduces MoE-Bind, a sequence-only autoregressive protein binder generator that combines Multi-head Latent Attention (MLA) with a sparse Mixture-of-Experts (MoE) feed-forward stack, aiming to keep binder generation fast and structure-free at inference while improving quality per unit compute. 2. Key architectural idea: sparsify where most parameters live. Since transformer FFNs hold a large fraction of parameters, MoE-Bind replaces dense FFNs with top-2 routing over 8 SwiGLU experts (plus a shared always-on expert), so only ~2/8 of expert parameters activate per token while total capacity increases. 3. MLA targets the other bottleneck: KV-cache memory during autoregressive decoding with long receptor prompts. MoE-Bind compresses keys/values into a low-rank latent (rKV=64) and uses decoupled RoPE (separate positional subspace), yielding a large KV-cache reduction (reported 24× vs a GPT2-like MHA peer at the 100M tier). 4. Compute/parameter framing: the 100M-parameter MoE-Bind model has ~102.7M total params but ~38.8M active params per token, positioning it as “compute-matched” against ~38M dense baselines while often matching or exceeding ~100M dense baselines in structure-level metrics. 5. Training pipeline: pre-train on UniRef50 (character-level tokenization; 31-token vocab including delimiters/control tokens) with next-token prediction, then instruction fine-tune on high-confidence STRING v12 physical PPIs (score ≥900) after heavy redundancy reduction (MMseqs2 clustering at 40% identity, 80% coverage), ending with ~2.1M usable interaction pairs. 6. Leakage control is a major methodological emphasis. For DB5 evaluation, the authors build a strict 22-target benchmark by removing any DB5 proteins with ≥10% identity (≥80% coverage) to UniRef50 or STRING sequences, then also report a larger benchmark (78 unique targets) under a relaxed fine-tuning-only leakage filter and additional deduplication. 7. Structure-level evaluation uses structure predictors only for external assessment, not for inference-time filtering: AlphaFold2-Multimer (ColabFold) on the strict 22-target DB5 set, and Boltz-2 with MSA on the larger 78-target set. Hits are defined stringently as generated ipTM ≥ reference (native pair) ipTM for the same target. 8. Main structure-level results: on the 22-target AF2-Multimer evaluation, MoE-Bind achieves 6/22 hits (27.3%) vs MHA 3/22 and GQA 4/22; on the 78-target Boltz-2 MSA benchmark, MoE-Bind reaches 19/78 hits (24.36%), slightly higher than dense 100M baselines (GQA-100M 23.08%, MHA-100M 21.79%) and higher than compute-matched dense ~38M baselines (GQA-38M 20.51%, MHA-38M 16.67%) while activating ~38.8M params/token. 9. Sequence-level quality: MoE-Bind’s generated binders better match DB5 amino-acid composition, avoid long homopolymer runs (no 6–7 or ≥8 runs reported), show “controlled novelty” vs STRING (less mass at ~0% identity than dense baselines), and have improved predicted stability by instability index (median ~29–30, with ~2/3 below 40). 10. Interpretability contribution: routing analysis reports expert specialization at individual amino-acid and biochemical-group levels, arguing that proteins’ small, biochemically structured alphabet makes MoE routing more interpretable than typical natural-language MoE behavior, and suggesting future expert pruning/specialization guided by biochemical priors. 📜Paper: biorxiv.org/content/10.64898… #ComputationalBiology #ProteinDesign #ProteinEngineering #ProteinLanguageModels #MixtureOfExperts #Transformers #DeepLearning #Bioinformatics #PPI #DeNovoDesign

Protein language models are accidental taxonomists 1. The paper argues that many high reported scores for multi-species protein–protein interaction (PPI) prediction can be explained by a confounder: models learn taxonomy/phylogeny signals rather than interaction biology, a behavior they call the “accidental taxonomist” hypothesis. 2. Core dataset issue: in BioGRID, 96.5% of positive PPIs are intra-species (same organism), but “randomly paired” negatives are intra-species only ~31.8% of the time. This mismatch lets a model treat “different species” as a proxy for “negative.” 3. A simple simulation shows the shortcut is strong: a perfect classifier for “same species vs different species” could reach ~82% accuracy and AUROC ~0.82 on a balanced PPI dataset, without learning any PPI mechanism. 4. The authors then test whether protein language model (pLM) embeddings encode taxonomy strongly enough to enable this shortcut. Using MLP probes on multiple pLMs, they show embeddings support taxonomic classification from domain down to species, with performance decreasing as labels get more fine-grained. 5. Key observation for pair-input tasks: even when exact species prediction is weak (best weighted F1 ~0.18 across 618 species), a paired binary task (“are these two proteins from the same species?”) is easy from embeddings. Most pLMs exceed 0.8 F1; ESMC600M reaches ~0.87 F1. This mirrors PPI’s two-sequence input format. 6. They propose a practical control: Strategic Sampling (SS) for negatives, where negative pairs are sampled only within the same species, preventing “inter-species = negative” cheating. This is contrasted with Normal Sampling (NS), where negatives are sampled across the whole multi-species pool. 7. Results on rigorously split multi-species BioGRID (C3 splits, 40% identity clustering): NS looks strong on its own validation (mean MCC ~0.71), but collapses when evaluated on an SS test set (mean MCC ~0.23). SS models are lower on validation (mean MCC ~0.39) but generalize better to the SS test set (mean MCC ~0.34). This gap supports the confounder hypothesis. 8. Training dynamics suggest “reward hacking”: NS drives predicted probabilities for positives toward ~0.94 (vs ~0.75 for SS), consistent with exploiting a near-deterministic shortcut; both methods find negatives difficult (negative probabilities ~0.5), indicating the real PPI signal remains challenging. 9. Inter-species PPIs remain a blind spot: on BioGRID-MV (enriched for multi-validated interactions including host–virus), both NS and SS show near-zero recall for inter-species positives (SS: 0/1091; NS: 2/1091), with predicted probabilities near coin-flip. This suggests host–pathogen PPI needs dedicated data and evaluation, not just multi-species scaling. 10. Broader implication: “accidental taxonomist” is not PPI-specific. Any supervised protein task where labels correlate with phylogenetic regions (e.g., EC numbers concentrated in certain taxa) may inflate performance unless datasets control for taxonomy/phylogeny structure. 💻Code: github.com/Gleghorn-Lab/PLMC… 📜Paper: doi.org/10.1186/s12859-026-0… #Bioinformatics #ComputationalBiology #ProteinLanguageModels #MachineLearning #PPI #DatasetBias #Benchmarking #Phylogeny #Taxonomy #RepresentationLearning

Interpretable enzyme function prediction via sparse autoencoder features of ESMC across the microbial protein universe 1. The paper proposes a “features-first” route to enzyme function prediction: instead of training a new deep model, it uses ESMC-6B’s sparse autoencoder (SAE) features as an interpretable semantic signature to predict EC numbers. 2. Key enabler: the ESMC SAE expands layer-60 hidden states into a 16,384-feature codebook with Top-K=64 sparsity, and each feature has an independently interpretable biological concept label (generated/annotated via multi-agent GPT-5), enabling mechanistic explanations alongside predictions. 3. Benchmark setup: 4,868 reviewed microbial SwissProt enzymes (Bacteria Archaea), balanced across 7 EC1 superclasses and spanning 161 EC3 subclasses; proteins are 80–700 aa and have ≥3 EC levels. Protein representations are mean-pooled SAE activations, evaluated with simple linear probes. 4. Main EC3 result (161-way classification, 80/20 split): SAE binary features (just the top-64 active concepts per protein) reach 78.9% top-1 and 88.5% top-5 accuracy, outperforming a 3-mer logistic regression baseline (57.3% top-1) and approaching BLASTp (80.5% top-1). 5. Using richer SAE information improves further: the combined binary activation weights representation reaches 85.6% top-1 and 90.5% top-5, exceeding BLASTp top-1 by 6.4% in this benchmark. 6. Practical advantage over homology transfer: BLASTp returns no hit for 12.6% of test proteins, while SAE features yield predictions for 100% of queries; for the BLASTp no-hit subset, SAE binary still achieves 62.1% top-5 accuracy. 7. “Dark-matter regime” analysis: test proteins are binned by maximum 3-mer Jaccard similarity to training data; in the lowest-similarity bin (<0.20, containing 61% of the test set), SAE top-5 accuracy is 0.656 vs 0.438 for the 3-mer baseline, showing robustness when sequence similarity is minimal. 8. Generalization to unseen enzyme classes: leave-one-EC3-class-out evaluation (60 populous EC3 subclasses held out) trains only an EC1 classifier and tests whether it recovers the correct EC1 superclass for a completely unseen EC3 class; SAE binary achieves 47.7% EC1 recovery (3.3× random; vs 26.6% for sequence features). 9. Interpretability check: the most discriminative SAE features align with known enzymology—e.g., hydrolases linked to α/β-hydrolase catalytic triad/nucleophilic elbow geometry; oxidoreductases to Rossmann NAD(P)H-binding motifs; transferases to P-loop/Walker A phosphate-binding patterns; translocases to multi-helix transmembrane bundle concepts—supporting mechanistic plausibility of the learned “concept” features. 10. Atlas-scale survey: scanning 7.7M ESM Atlas cluster representatives, the authors identify 169,859 “dark enzyme-like” candidates (uncharacterized clusters with enzyme-suggestive Pfam keywords) spanning major microbial phyla, positioning SAE-based signatures as a scalable prioritization tool for experimental enzyme discovery. 💻Code: github.com/YueHuLab/esmc-sae… 📜Paper: arxiv.org/abs/2606.12209 #ProteinLanguageModels #EnzymeDiscovery #ECNumber #Interpretability #SparseAutoencoders #Microbiome #Metagenomics #ComputationalBiology #ESMC #ESMAtlas

Viral Proteins Reveal Geometry of Protein Language Models 1. The paper shows that protein language model (pLM) embedding spaces are dominated by a single “nativeness axis” (PC1) that strongly aligns with masked-reconstruction perplexity (a model-relative measure of how in-distribution a sequence is). This axis orders sequences from well-modeled cellular proteins to viral proteins to shuffled/random controls. 2. In ESMC-600M, PC1 explains 73.1% of embedding variance and correlates with perplexity at Spearman ρ=0.961, indicating that reconstruction difficulty is not just a score but a major geometric direction organizing embeddings across the tree of life. 3. Viral proteins are not treated as extreme outliers: they sit in an intermediate region—less “native” than cellular proteins but more structured than biologically meaningless sequences (position-shuffled or i.i.d. random), suggesting pLMs encode a continuum from in-distribution to out-of-distribution. 4. The same nativeness-axis geometry generalizes across ESM families (ESM2, ESMC, ESM3), including ESM3-OPEN (trained without viral sequences), and also appears in non-ESM architectures (ProGen2 autoregressive; EvoDiff discrete diffusion). This supports the idea that a dominant “model-fit” direction may be a broader property of sequence models trained on imbalanced biological data. 5. The work quantifies the underlying imbalance: UniRef50 pretraining coverage is heavily cellular-dominated (about 46.3M cellular clusters vs 390.3k viral; ~119× ratio), motivating the question of how underrepresented viral sequences are represented. 6. A key control argues nativeness is not simply “seen vs unseen”: cellular Swiss-Prot proteins released after an ESMC checkpoint (thus absent from its pretraining data) still look far more native-like (median PPL 5.3) than human viral proteins (median PPL 15.3), implying nativeness reflects compatibility with a cellular-dominated prior more than mere exposure. 7. Scaling changes viral nativeness, but unevenly across viral families: in ESMC (300M→6B), the fraction of human viral proteins below a native-like threshold (PPL<5) increases only modestly overall (~5%→~17%), while some families (e.g., Papillomaviridae, Retroviridae) shift strongly toward the native region and others (e.g., Orthomyxoviridae, Orthoherpesviridae, Sedoreoviridae) remain displaced. 8. Despite this dominant nativeness axis, embeddings retain viral-specific information beyond perplexity: linear probes on mean-pooled embeddings classify human viral vs cellular proteins with near-ceiling AUC (0.97–1.00 for larger models) under a homology-controlled split, and outperform both perplexity-only zero-shot classification and shallow sequence baselines (length, amino-acid composition, dipeptide composition). 9. The separation is especially relevant at low false-positive rates (screening-like settings): at 1% FPR, embedding probes achieve much higher TPR than perplexity-only classifiers (e.g., ESMC-6B 96.7% vs 39.2%), showing that “viral identity” is linearly accessible even when perplexity becomes a weaker separator at large scale. 10. Implications: nativeness (perplexity / PC1 position) can act as a diagnostic for where pLMs may be less reliable (notably for certain viral families), while embedding-based signals may complement homology methods for viral detection and biosecurity screening—though the authors emphasize evaluation and safety framing over deployment. 💻Code: github.com/MisteFr/viral-pro… 📜Paper: arxiv.org/abs/2606.12609 #ProteinLanguageModels #ESM #ViralProteins #RepresentationLearning #ComputationalBiology #Bioinformatics #MachineLearning #Biosecurity #Interpretability #ScalingLaws

SurfDesign: Effective Protein Design on Molecular Surfaces 1. SurfDesign reframes protein design around molecular surfaces (shape physicochemical complementarity), aiming to better control functional regions like binding interfaces and enzyme pockets compared with backbone-only conditioning. 2. Core technical idea: treat molecular surfaces as continuous geometric manifolds rather than unordered point clouds/meshes, so the model can leverage local tangent structure, curvature, and directional consistency that are important for interaction-centric design. 3. The method builds an oriented surface point cloud Q where each point includes coordinates, a unit normal vector, and physicochemical attributes (e.g., hydrophobicity, charge, H-bond features). Surfaces are generated via PyMOL/MSMS and denoised with Gaussian smoothing; no residue identities, MSAs, or functional labels are used in surface generation. 4. SurfDesign introduces a Surface-conditioned Equivariant Message Passing (SEMP) encoder: SE(3)-equivariant updates use invariant radial distances, curvature descriptors (from local covariance eigenvalues), and directional angular features derived from surface normals (two intersection angles one dihedral angle). 5. Directionality is encoded with spherical Fourier–Bessel bases over distances and angles; messages are attention-reweighted and used to update both node features and coordinates, with optional per-layer recomputation of normals/curvatures to stay consistent as coordinates evolve. 6. To address limited surface-structure paired data and improve sequence priors, SurfDesign integrates pretrained protein language models (PLMs) via parameter-efficient fine-tuning (hybrid PEFT: structural adapter LoRA), trained with conditional masked language modeling rather than autoregressive decoding. 7. Binder design benchmark (6 targets) uses AF2 pAE_interaction as a functional proxy. SurfDesign achieves the best average pAE_interaction (15.85) and the highest overall success rate (30.14%), outperforming SurfPro (surface-conditioned baseline) and backbone-only baselines like ProteinMPNN, PiFold, and LM-DESIGN. 8. Enzyme design benchmark (5 enzyme–substrate systems; leakage-controlled by excluding overlaps with CATH pretraining) uses ESP score as a proxy for enzyme–substrate compatibility. SurfDesign attains the best average success rate (47.30%) and the best average ESP under greedy decoding (0.9058), with gains persisting in a zero-shot substrate setting. 9. Inverse folding is positioned as a diagnostic for structural compatibility (not “recovering the native sequence”). SurfDesign reports strong results on CATH splits, including perplexity 2.41 and AAR 74.13% on CATH 4.2, plus improved surface recovery metrics (IoU/CD/NC) versus PLM-based baselines; scaling larger PLMs further improves recovery. 💻Code: github.com/smiles724/SurfDes… 📜Paper: arxiv.org/abs/2606.07567 #ProteinDesign #ComputationalBiology #GeometricDeepLearning #ProteinEngineering #EnzymeDesign #ProteinBinding #EquivariantNetworks #ProteinLanguageModels #KDD2026