.. _Results: =============================== Results =============================== .. contents:: :local: :depth: 2 Synthetic Regulatory Networks ----------------------------- CIPHER was first evaluated using synthetic gene regulatory networks with known ground-truth interactions. The method accurately recovered perturbation responses in linear and moderately nonlinear systems, although performance decreased for large perturbation magnitudes and simultaneous perturbations of multiple genes. Biologically Informed Networks ------------------------------ CIPHER was next evaluated on seven literature-curated regulatory networks spanning a range of sizes and topologies: epithelial--mesenchymal transition (EMT), autophagy, the JNK regulatory module, myeloid differentiation, B-cell activation, hematopoiesis, and the circadian clock. Each signed network was simulated with continuous ordinary differential equations in which activating and inhibiting interactions are represented by shifted Hill functions. Rather than assigning one fitted parameter set per network, the random circuit perturbation (RACIPE) framework was used to sample ensembles of biologically plausible kinetic parameters -- production rates, degradation rates, regulatory thresholds, Hill coefficients, and interaction strengths -- so that each realization can be interpreted as a distinct cellular or environmental context, and performance is assessed across a broad distribution of dynamical regimes. The EMT network was used as the primary test case, since it is a well-established benchmark for multistable cell-fate regulation, is minimally frustrated, and retains a low-dimensional dominant mode across heterogeneous parameter ensembles. Each gene was knocked down in turn, and predictions were scored as the median absolute Pearson correlation (``|r|``) between the nonlinear response and the covariance-based linear-response prediction. CIPHER was strongly predictive for the literature-informed network (mean ``|r|`` of approximately 0.60 across genes), and performance dropped sharply for a degree-preserving randomized version of the same network (approximately 0.21) and for a mean-field control in which the gene--gene covariance matrix was scrambled (approximately 0.11). Topology and covariance structure, not the simulation machinery, therefore carry the predictive signal. .. figure:: images/Figure2I.png :name: fig-emt-network :alt: EMT regulatory network and CIPHER prediction accuracy against randomized and mean-field controls. :width: 30 % :align: center *EMT regulatory network and controls.* The 26-node EMT network, with activating (green arrows) and inhibiting (red bars) interactions. Below, the median absolute Pearson correlation between predicted and observed responses under knockdown of each gene, for the literature-curated EMT network, a degree-preserving randomized EMT network, and a mean-field EMT covariance. Each point is one gene; black bars mark the mean across all genes in a network. Across the remaining biologically informed networks, CIPHER recovered the direction of the nonlinear perturbation response with high accuracy despite substantial kinetic-parameter heterogeneity, with per-network means ranging from roughly 0.6 to 0.9 and an average absolute Pearson correlation of 0.80. The corresponding normalized frustration was low for every network, consistent with the theoretical account: minimal frustration yields an approximately low-rank regulatory structure, which in turn makes a single covariance direction sufficient to capture the response and keeps performance robust to parameter heterogeneity. .. figure:: images/Figure2J.png :name: fig-bio-networks :alt: CIPHER prediction accuracy and normalized frustration across seven biologically informed regulatory networks. :width: 70 % :align: center *Biologically informed regulatory networks.* Top: network topologies for autophagy, EMT, JNK signaling, myeloid differentiation, B-cell activation, the circadian clock, and hematopoiesis, with edges coloured as above. Middle: median absolute Pearson correlation for knockdown of each gene; each point is one gene and black bars indicate the network mean. Bottom: normalized mean frustration from Boolean simulations of each network. Genetic Perturbation -------------------- Across large-scale CRISPRi and CRISPRa datasets, CIPHER used gene--gene co-fluctuations in unperturbed cells to reconstruct transcriptome-wide expression changes following perturbation. Covariance matrices were computed from unperturbed single cells in each experimental count matrix, using raw absolute counts with a minimum average expression cutoff. As a null model, a "mean-field" covariance was constructed by shuffling expression values across cells independently for each gene. This preserves each gene's marginal expression distribution and its gene-wise count statistics while destroying the off-diagonal covariance structure, collapsing the block-structured experimental correlation matrix into an unstructured one. .. figure:: images/Figure3B.png :name: fig-meanfield :alt: Construction of the mean-field null model by shuffling expression values across cells independently for each gene. :width: 40 % :align: center *The mean-field null model.* Experimental and mean-field count matrices, obtained by shuffling over cells rather than genes, together with their corresponding covariance matrices. Colour scale runs from -1 to 1. Removing off-diagonal gene--gene covariances in this way substantially reduced Pearson correlation and other response-prediction metrics, showing that baseline covariance contains important information about perturbation outcomes. CRISPRi and CRISPRa Performance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ CIPHER recapitulated responses from both CRISPR interference (CRISPRi) and CRISPR activation (CRISPRa) experiments. Performance was evaluated with the coefficient of determination, Pearson correlation, and Spearman correlation, comparing the real covariance matrix against the mean-field control on a per-dataset basis. Replacing the mean-field covariance with the measured one improved the coefficient of determination by a mean of 0.37 for CRISPRa datasets and 0.34 for CRISPRi datasets. The gain was consistent in direction across essentially every dataset, not driven by a handful of outliers. Averaged per dataset, Pearson correlation improved by a mean of 0.48 (median 0.52) and Spearman correlation by a mean of 0.35. Under the mean-field control both metrics sit near zero, so the real covariance supplies effectively all of the predictive signal. .. figure:: images/Figure3C.png :name: fig-crispri-crispra :alt: Per-dataset improvement in coefficient of determination, Pearson correlation, and Spearman correlation when using the real covariance instead of the mean-field covariance. :width: 30 % :align: center *CRISPRi and CRISPRa results.* Left: improvement in transcriptome-wide prediction accuracy from using the experimentally measured covariance rather than the mean-field covariance. Each line is one dataset, connecting mean-field performance to real-covariance performance; red points are CRISPRi datasets and blue points are CRISPRa datasets. Centre and right: the same comparison scored by average Pearson and average Spearman correlation per dataset. Large diamonds denote means across datasets, with mean and median improvements annotated. Computational Benchmark ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ CIPHER was benchmarked against GenePert, scLAMBDA, Scouter, scGPT, GEARS, and a linear mean baseline across 27 single-gene perturbation datasets. For each dataset, perturbations were split evenly into training and test halves, with every method fit on the training half and evaluated on held-out in-distribution test perturbations in a common expression space. CIPHER reached a median Pearson correlation of 0.55, competitive with the leading methods GenePert (0.58) and Scouter (0.57) and ahead of scLAMBDA and scGPT (both 0.48). On cosine similarity CIPHER ranked first at 0.57, marginally above GenePert (0.57) and Scouter (0.56). Both deep-learning-free baselines aside, GEARS (0.17) and the linear mean baseline (0.28) trailed the field on both metrics, with the linear mean baseline outperforming GEARS. .. figure:: images/Figure3J.png :name: fig-benchmark :alt: Pearson correlation and cosine similarity between predicted and observed perturbation responses for CIPHER and six competing methods. :width: 80 % :align: center *Prediction performance benchmark.* Distributions of Pearson correlation (left) and cosine similarity (right) between predicted and experimentally observed perturbation responses, for CIPHER and comparison methods. Violins summarize the distribution across perturbations; horizontal bars mark medians, labelled numerically. Computational Efficiency ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ CIPHER completed in approximately 22 seconds per dataset using 3.8 GB of memory. By comparison, GEARS required roughly 1.7 hours and 52.7 GB, and scGPT roughly 8.7 hours and 72.7 GB -- a difference of two to three orders of magnitude in runtime. Only the linear mean baseline used less memory (3.5 GB), and it was slower (46 seconds). The gap reflects the absence of a training phase. For the deep learning methods, training dominates total runtime and requires GPU memory on top of system RAM, whereas CIPHER fits a single scaling parameter, trains no neural network, and needs no GPU. .. figure:: images/Figure3MN.png :name: fig-efficiency :alt: Peak memory and total runtime for CIPHER and comparison methods, with runtime broken down by phase. :width: 60 % :align: center *Computational efficiency.* Left: peak memory during execution, split into system RAM (blue) and GPU memory (green). Right: total runtime, partitioned into data loading, setup, training, and evaluation. Totals are annotated above each bar. The Inverse Problem ------------------- Having established that covariance structure explains the response to a *known* perturbation, the framework was inverted: given a covariance matrix and a measured transcriptome-wide change in expression, which gene or genes most likely caused it? CIPHER solves for a full combinatorial perturbation vector and ranks genes by inferred effect size. The problem is framed as Bayesian linear regression, ``ΔX = Σu + ε``, with a shrinkage prior and a noise model informed by the central limit theorem, so every gene receives an effect size with error bars and a posterior inclusion probability rather than a bare point estimate. .. figure:: images/Figure5A.png :name: fig-inverse-schematic :alt: Schematic of the CIPHER inverse problem and the posterior quantities it returns for each gene. :width: 30 % :align: center *The inverse problem.* Left: CIPHER ranks candidate drivers of a measured response using the expression fluctuations of control cells; each point is a single-cell transcriptome coloured by perturbed gene. Right: posterior probability of a nonzero effect is high for the truly perturbed genes A and B, and the effect-size distributions separate knockdown from activation, with unperturbed genes at zero. Ranking by maximum posterior spread recovered the true target with high fidelity across every dataset, cell type, and perturbation modality: mean AUC was 0.953 for CRISPRi and 0.913 for CRISPRa, with mean average precision of 0.428 and 0.373. The median rank of the true target across the full gene set was 2, and it was ranked first in 47% of cases. The same likelihood extends from identifying the perturbed gene to classifying perturbation identity: scoring held-out responses against signatures inferred for each candidate perturbation beat a log-fold-change baseline on both AUROC (0.914 vs 0.846) and AUPRC (0.857 vs 0.688). Chemical Perturbation Screens ----------------------------- Applied to the 188 pharmaceutical perturbations in the sci-Plex screen, CIPHER fit a transcriptional signature per drug and scored held-out cells against every candidate compound by likelihood. It outperformed both a mean-field control and log-fold-change classification, reaching an AUROC of 0.939 against 0.838 and an AUPRC of 0.996 against 0.939. Ranking errors were chemically sensible: for the held-out example below, the true compound fulvestrant ranks first and its closest competitor, toremifene citrate, is another agent acting on the estrogen receptor. .. figure:: images/Figure5JM.png :name: fig-sciplex :alt: Likelihood ranking, top-k accuracy, ROC and precision-recall curves for identifying the true compound in the sci-Plex chemical screen. :width: 100 % :align: center *Identifying causal compounds in sci-Plex.* Log-likelihood ranking for one held-out drug, with the true compound in blue; top-k accuracy for CIPHER (blue), log-fold-change ranking (purple), and the mean-field linear response (grey); and ROC and precision-recall curves pooled across sci-Plex perturbations. The advantage survived a harder test on Tahoe-100M, where training and validation used separate experimental replicates. Hierarchically clustering the inferred drug profiles recovered coherent mechanism-of-action blocks -- 17 HDAC inhibitors, plus groups of JAK/STAT, RTK/multikinase, DNA-damaging, and cell-cycle compounds -- far above a label-shuffled null (z = 12.71, p < 0.0001).