Compare Neural Circuit Correlations Across States¶

This tool uses 1.0 compute credits per hour.

Overview¶

The Compare Neural Circuit Correlations Across States tool analyzes pairwise Pearson correlations between neural cell traces across different behavioral states, enabling comparison of neural synchrony patterns across experimental conditions.

Input Data¶

This tool requires an .isxd cell set file containing neural activity data. An optional synchronized annotation file can be provided to analyze correlations across specific behavioral states.

Valid Inputs¶

Source Parameter	File Type	File Format
Cell Set Files	cell_set	isxd
Annotations File	experiment_annotations	parquet

Generating synchronized annotation files

You can generate the synchronized annotation file by running one of these synchronization tools:

Annotation File Usage

An annotation file is only required when analyzing correlations across specific behavioral states. Without an annotation file or specified State Names, the tool will calculate correlations across the entire recording duration.

Parameters¶

Parameter	Required?	Default	Description
Cell Set Files	True	N/A	ISXD cell set file(s)
Annotations File	False	N/A	File containing annotations about states, synchronised to ISXD cell set file
Column Name	True	state	Column name in annotations file that identifies state
State Names	True	N/A	Specify which behavioral states to analyze (comma-separated list). For single state analysis, correlations from that state will be compared with all other time periods. For multiple states, correlations will be compared between the specified states.
State Colors	True	N/A	List of matplotlib compatible colors to represent each state
Correlation Colors	True	tab:red, tab:blue	List of colors to use for positive and negative correlations, or a colormap
Statistic	True	max	Statistic of correlation to compare (max, mean or min)
Correlation Threshold for Spatial Map	False	0.5	Minimum absolute correlation value to display in the spatial correlation map. Only cell pairs with correlations above this threshold will be shown as connections.

The tool computes correlations based on behavioral states specified in the State Names parameter:

State-specific analysis: When State Names are provided along with an annotation file, correlations are calculated for each specified state using corresponding time segments from the annotation file.
Error condition: If State Names are provided but no annotation file is given, the tool will raise an error.
Full recording analysis: When State Names are not provided (empty or unspecified), correlations are computed across the entire recording duration (labeled as "all times").
Minimum data requirement: States with fewer than 10 frames are excluded from analysis due to insufficient data for reliable correlation estimation.
Unlabeled data handling: Time segments not assigned to any specified state (containing ≥10 frames) are automatically grouped into a "not_defined" state for separate analysis.

Spatial Analysis: When cell position information is available in the cell set file, the tool generates additional spatial correlation outputs: - The Correlation Threshold parameter filters which correlations are visualized in the spatial correlation map

Algorithm Details¶

Core Analysis Workflow:

graph TD A[Load Data & Prepare Annotations] --> B[Compute Pairwise Correlation Matrices] B --> C[Compute Average +/- Correlations] B --> D[Compute Correlation Statistics max/mean/min] B --> E[Extract Cell Positions if Available]

Correlation Calculation Methodology¶

Correlation Computation¶

For each state (or across all times if no states specified):

Computes pairwise Pearson correlation between all neuron pairs
Creates an n×n correlation matrix (where n = number of neurons)
Sets diagonal values to zero (self-correlations)

Matrix Organization¶

Optionally sorts the correlation matrix using hierarchical clustering to group similar cells together for visualization purposes.

Additional Analysis¶

For each cell, the tool computes one of the following statistics across its correlations with other cells:

Maximum correlation: Highest correlation value with any other cell
Mean correlation: Average correlation across all other cells
Minimum correlation: Lowest correlation value with any other cell

Correlation Method¶

The Pearson correlation coefficient is calculated as:

\[ \begin{aligned} r = \frac{\sum_{i=1}^{n}(X_i - \bar{X})(Y_i - \bar{Y})}{\sqrt{\sum_{i=1}^{n}(X_i - \bar{X})^2 \sum_{i=1}^{n}(Y_i - \bar{Y})^2}} \end{aligned} \]

where \(X_i\) and \(Y_i\) are individual trace values, and \(\bar{X}\) and \(\bar{Y}\) are the means.

Cell Selection Criteria¶

The analysis uses cell status information from the input cell set to determine which cells to include:

Accepted cells: If present, only accepted cells are included in the analysis
Undecided cells: Used when no accepted cells are available
Rejected cells: If all cells are rejected, the tool will terminate with an error message

Outputs¶

The tool generates the following output files:

File	Format	Description
Statistic correlations data file	CSV	Contains the specified statistic ('max', 'mean', or 'min') of correlations for each cell across each state (including 'other' or 'all times' if applicable). Cells as rows, states as columns.
Average correlations data file	CSV	Contains the average positive and average negative correlation value for each state.
Raw correlations data file	HDF5	A HDF5 file containing the raw correlation matrix for each state. Each H5 file contains datasets where: • The name of each dataset corresponds to a state (e.g., "exploring_familiar", "exploring_novel", "all times"). • The value of each dataset is a 2D NumPy array representing the cell-cell Pearson correlation matrix for that state.
Raw correlations data file	ZIP	A ZIP archive containing detailed raw correlation data in CSV format. Includes: - `correlation_matrix_[state].csv`: Full raw correlation matrix for each state. - `correlation_matrix_[state]_triu.csv`: Upper triangle of the correlation matrix with cell name pairs. This file also contains cell centroid coordinates (x1, y1, x2, y2) and the Euclidean distance between cell pairs. - `README.txt`: Explains the contents of the zip file.

CSV Output Examples¶

Statistic correlations data file (showing max correlations):

name	exploring_familiar	exploring_novel
C00	0.85	0.62
C01	0.73	0.81
C02	0.91	0.58

Average correlations data file:

state	positive	negative
exploring_familiar	0.42	-0.15
exploring_novel	0.38	-0.18

Raw correlations triu data file (from ZIP archive):

cell_name_1	cell_name_2	correlation	centroid_x1	centroid_y1	centroid_x2	centroid_y2	distance
C00	C01	0.73	120	85	145	92	26.9
C00	C02	-0.15	120	85	98	110	31.4
C01	C02	0.42	145	92	98	110	51.8

Previews¶

The tool generates the following visualization previews to help interpret the correlation analysis results:

Correlation Heat Maps¶

Displays correlation matrices as heat maps for each analyzed state. Data is reorganized using hierarchical clustering to group highly correlated cell pairs together for easier pattern identification.

*Example preview of correlations heatmap*

Correlation Statistic Distributions¶

Compares the distribution of the selected correlation statistic (max, mean, or min) across different behavioral states through two complementary visualizations:

Cumulative Distribution Function (CDF): Displays the cumulative probability distribution of correlation statistics for each state, facilitating comparison of overall distribution shapes
Box Plot: Shows the central tendency, spread, and outliers of correlation statistics across states (displayed only when analyzing multiple states)

Average Correlations¶

Presents bar plots comparing the mean positive and negative correlation values across all analyzed states, providing a summary view of overall correlation patterns.

Spatial Correlation Analysis¶

Examines the relationship between spatial proximity and neural correlation strength through scatter plots with regression lines and density visualizations. This analysis reveals whether spatially adjacent cells exhibit stronger correlations than distant cells.

Spatial Correlation Map¶

Visualizes the spatial organization of neural correlations by displaying cell positions with connecting lines between highly correlated pairs. Line colors represent correlation polarity, while line presence indicates correlations above the specified threshold. This visualization reveals spatial clustering patterns in neural synchrony.