Compare Neural State Data Across Epochs¶

Overview¶

The Compare Neural State Data Across Epochs tool performs comprehensive analysis of neural activity patterns across behavioral states and time epochs, calculating activity measures, correlation metrics, and modulation indices relative to a user-specified baseline state-epoch combination. This tool integrates correlation analysis and population activity analysis into a unified workflow, enabling researchers to understand how neural dynamics change across experimental conditions.

Input Data¶

This tool requires synchronized neural activity data, behavioral annotations, and epoch definitions. It can analyze calcium trace data and optionally calcium event data.

Valid Inputs¶

Parameters¶

Algorithm Details¶

The following diagram demonstrates the steps of the tool algorithm.

The following sections describe the metrics and procedures performed by the algorithm.

Epoch Definition¶

Time epochs are defined using one of three methods specified by the Define Epochs By parameter:

• global file time: Epochs are defined relative to the start of the entire recording session. For concatenated files, time continues across file boundaries.
• files: Each input file is treated as a separate epoch automatically, ignoring the Epochs parameter.
• local file time: Epochs are defined relative to the start of each individual file. The same epoch time ranges are applied to each file independently.

The Epochs parameter specifies time ranges as comma-separated tuples: "(start1, end1), (start2, end2), ..." in seconds.

Activity Measures¶

• Trace Activity: The mean of the calcium trace signal during a given state-epoch combination. Traces can be optionally rescaled using several methods:

  • none: Original raw calcium trace values are preserved without any transformation
  • normalize: Scales all trace values to the range [0, 1] by applying min-max normalization across the entire recording period
  • standardize: Converts traces to z-scores by subtracting the mean and dividing by the standard deviation across the entire recording
  • fractional_change: Calculates the ratio of activity to baseline state mean after ensuring non-negative values
  • standardize_baseline: Computes z-scores relative to the baseline state

• Event Rate: If event sets are provided, the tool calculates the event rate (events per second) for each cell. These rates can be rescaled using the same methods as trace activity.

Correlation Analysis¶

For each state-epoch combination, the tool computes:

1. Pairwise Pearson correlation between all neuron pairs using the Pearson correlation coefficient:

where \(X_i\) and \(Y_i\) are individual trace values, and \(\bar{X}\) and \(\bar{Y}\) are the means.

1. Per-cell correlation statistics: For each cell, one of the following statistics is computed across its correlations with other cells:
2. Maximum correlation: Highest correlation value with any other cell
3. Mean correlation: Average correlation across all other cells

4. Minimum correlation: Lowest correlation value with any other cell

5. Population correlation metrics: Average positive and negative correlation values across all cell pairs

Modulation Score¶

The modulation score quantifies the change in activity between each state-epoch combination and the baseline state-epoch combination. This tool exclusively uses the "state vs baseline" comparison method to avoid combinatorial complexity (see note in Overview section):

Before calculation, a minimum value subtraction is applied to ensure activity values are non-negative, bounding the score between -1 and 1.

Significance Level (p-value)¶

Trace modulation significance is determined using a circular permutation test:

1. The time labels corresponding to the state-epoch combinations being compared are randomly shifted (rolled) for the specified number of shuffles.
2. For each shuffle, a surrogate modulation score is calculated.
3. Two one-tailed probabilities are computed:
4. \(p_\text{high}\): fraction of shuffled scores greater than the observed score
5. \(p_\text{low}\): fraction of shuffled scores less than the observed score
6. The reported trace_p_values column stores \(\min(p_\text{high}, p_\text{low})\), which lies in \([0, 0.5]\) and reflects the stronger tail.

Directional classifications (up- vs down-modulated) apply the original tail probabilities with a Bonferroni-style split:
- up-modulated if the observed modulation score is positive and \(p_\text{high} < \alpha / 2\)
- down-modulated if the observed modulation score is negative and \(p_\text{low} < \alpha / 2\)
- otherwise non-significant

This matches the permutation procedure used in the standard Python population-activity workflow and maintains an overall Type I error rate of α. The resolution of the probabilities depends on the number of shuffles (e.g., 1000 shuffles give a resolution of 0.001).

Event modulation uses the same permutation workflow whenever raw event traces are available (i.e., when Event Set File(s) are supplied). In that case, event_p_values follow the same interpretation as trace p-values and directional classifications use the identical α/2 thresholds. If only aggregated event rates are present (no time-resolved event traces), the tool falls back to an effect-size heuristic: event_p_values are set to 1.0 and modulation labels rely on |score| > 0.1.

Cell Selection Criteria¶

The tool uses cell status information from the input cell set(s) to determine which cells to include in the analysis:

• Accepted cells: When present, only accepted cells are analyzed
• Undecided cells: Used when no accepted cells are available
• Rejected cells: If all cells are rejected, the tool terminates with an error
• Automatic rejection: Cells whose traces contain only NaN values are automatically marked as rejected

Outputs¶

The tool generates four main CSV files containing detailed analysis results and comprehensive visualization previews.

Data Files¶

• activity_per_state_epoch_data.csv: Contains mean activity, standard deviation, median, and coefficient of variation for trace and event data (if available) for each cell in each state-epoch combination.
• correlations_per_state_epoch_data.csv: Contains all correlation statistics (max, mean, and min) for each cell in each state-epoch combination, plus population-level positive and negative correlation averages.
• average_correlations.csv: Contains the average positive and average negative correlation values for each state-epoch combination.
• modulation_vs_baseline_data.csv: Contains modulation scores, p-values, and modulation classifications for each cell across all state-epoch combinations relative to the baseline.

Activity Per State Epoch Data¶

Each row corresponds to a single cell in a specific state-epoch combination (long format). Columns include:

• name: Cell identifier
• cell_index: Numeric cell index
• state: Behavioral state name
• epoch: Time epoch name
• mean_trace_activity: Mean calcium trace activity for the cell during that state-epoch combination
• std_trace_activity: Standard deviation of trace activity
• median_trace_activity: Median trace activity
• trace_activity_cv: Coefficient of variation for trace activity
• mean_event_rate: Mean event rate for the cell (if event data provided)
• std_event_rate: Standard deviation of event rate (if event data provided)
• median_event_rate: Median event rate (if event data provided)
• event_rate_cv: Coefficient of variation for event rate (if event data provided)

Example activity_per_state_epoch_data.csv:

Correlations Per State Epoch Data¶

Each row corresponds to a single cell in a specific state-epoch combination (long format). Columns include:

• name: Cell identifier
• cell_index: Numeric cell index
• state: Behavioral state name
• epoch: Time epoch name
• max_trace_correlation: Maximum correlation with any other cell for traces
• min_trace_correlation: Minimum correlation with any other cell for traces
• mean_trace_correlation: Mean correlation across all other cells for traces
• max_event_correlation: Maximum correlation with any other cell for events (if event data provided)
• min_event_correlation: Minimum correlation with any other cell for events (if event data provided)
• mean_event_correlation: Mean correlation across all other cells for events (if event data provided)
• positive_trace_correlation: Population-level average of positive correlations for traces (same value for all cells in this state-epoch)
• negative_trace_correlation: Population-level average of negative correlations for traces (same value for all cells in this state-epoch)
• positive_event_correlation: Population-level average of positive correlations for events (if event data provided)
• negative_event_correlation: Population-level average of negative correlations for events (if event data provided)

Example correlations_per_state_epoch_data.csv:

Average Correlations¶

Each row corresponds to a state-epoch combination. This file provides a summary of population-level correlation patterns. Columns include:

• state: State-epoch combination identifier in format "{state}-{epoch}" (e.g., "rest-baseline")
• positive_trace_correlation: Average positive correlation value for traces
• negative_trace_correlation: Average negative correlation value for traces
• positive_event_correlation: Average positive correlation value for events (if event data provided)
• negative_event_correlation: Average negative correlation value for events (if event data provided)

Example average_correlations.csv:

Modulation vs Baseline Data¶

Each row corresponds to a single cell. Columns include:

• name: Cell identifier
• cell_index: Numeric cell index
• state: State name for this modulation comparison
• epoch: Epoch name for this modulation comparison
• baseline_state: Baseline state name
• baseline_epoch: Baseline epoch name
• trace_modulation_scores in {state}-{epoch}: Modulation score for trace activity relative to baseline
• trace_p_values in {state}-{epoch}: P-value from permutation test for trace modulation
• trace_modulation in {state}-{epoch}: Modulation classification (-1 for significantly down-modulated, 1 for significantly up-modulated, 0 for non-significant)
• event_modulation_scores in {state}-{epoch}: Modulation score for event rate relative to baseline (if event data provided)
• event_p_values in {state}-{epoch}: P-value from the event permutation test (or 1.0 when only aggregate event rates are available)
• event_modulation in {state}-{epoch}: Modulation classification for events (permutation-based when event traces exist, or |score| > 0.1 when falling back to rate-only data)

Example modulation_vs_baseline_data.csv:

Additional Output Files¶

The tool also generates binary files containing raw correlation data:

• pairwise_correlation_heatmaps.h5: HDF5 file containing raw correlation matrices for each state-epoch combination. Each dataset corresponds to a state-epoch combination and contains a 2D NumPy array of cell-cell Pearson correlations.
• spatial_analysis_pairwise_correlations.zip: ZIP archive containing detailed correlation data in CSV format, including full correlation matrices, upper triangle matrices with cell pairs and spatial information (centroid coordinates and distances between cell pairs), and a README file explaining the contents.

Previews¶

The tool generates comprehensive SVG visualizations for data interpretation:

State-Epoch Time Analysis¶

Displays the duration spent in each state-epoch combination, organized by state with epoch breakdowns.

Population Activity Summary¶

Bar plots displaying mean population activity (calcium traces) for each state-epoch combination with error bars showing standard error.

Neural Trace Preview¶

Shows calcium traces for a subset of neurons over time, with traces colored by behavioral state to visualize state-dependent activity patterns.

Event Rate Summary¶

Box plots showing mean population event rates for each state-epoch combination with individual cell data points. Generated only when event data is provided.

Event Raster Display¶

Raster plot visualizing detected calcium events across cells over time, with events colored by behavioral state. Generated only when event data is provided.

Spatial Modulation Map¶

Spatial footprint visualization showing significantly up-modulated (red outline) and down-modulated (blue outline) cells relative to baseline for each state-epoch combination.

Modulation Distribution¶

Histograms showing modulation score distributions for each state-epoch combination, with color-coded regions indicating significantly up-modulated (red) and down-modulated (blue) cells.

Event-Based Spatial Modulation¶

Spatial footprint maps showing cells with significant event rate modulation using color-coded outlines. Generated only when event data is provided.

Event Modulation Distribution¶

Histograms displaying event rate modulation score distributions with color-coded regions. Generated only when event data is provided.

Correlation Statistic Distributions¶

Compares the distribution of the selected correlation statistic across state-epoch combinations through cumulative distribution functions (CDFs) and box plots.

Average Correlations¶

Bar plots comparing the mean positive and negative correlation values across all state-epoch combinations.

Correlation Heat Maps¶

Displays correlation matrices as heat maps for each state-epoch combination. Matrices are reorganized using hierarchical clustering to group highly correlated cell pairs.

Spatial Correlation Analysis¶

Examines the relationship between spatial proximity and neural correlation strength through scatter plots with regression lines and density visualizations for each state-epoch combination.

Spatial Correlation Map¶

Visualizes the spatial organization of neural correlations by displaying cell positions with connecting lines between highly correlated pairs for each state-epoch combination. Line colors represent correlation polarity (positive/negative), while line presence indicates correlations above a predefined threshold.