Combine and Compare Correlation Data¶

Overview¶

This tool combines cell-cell correlation data generated by the Compare Neural Circuit Correlations Across States tool from multiple recordings. It focuses on specific states defined by the user. The tool calculates and compares several correlation metrics across recordings, states, and experimental groups:

Parameters¶

Input Files¶

The input files have the following requirements:

• File Type: Input files must be HDF5 files (.h5) generated by the Compare Neural Circuit Correlations Across States.
• Group Size: If data for a group is provided, that group must contain at least two H5 files.
• State Matching: The State Names parameter provided to this tool must be a comma-separated list of strings that exactly match (case-insensitive) the names of the datasets within the input H5 files that you wish to analyze.
• State Consistency: All input H5 files should contain datasets for the states specified in State Names.
• State Colors: The number of state_colors provided must be equal to the number of State Names provided.

Additionally, each input H5 file is expected to contain top-level datasets where:

• The name of each dataset corresponds to a state (e.g., "immobile", "mobile", "other").
• The value of each dataset is a 2D NumPy array representing the cell-cell Pearson correlation matrix for that state. The diagonal elements are expected to be zero.

Algorithm Description¶

The tool follows a simple three-step process:

1. Data Processing: Reads correlation data from input files, filters for specified states, and calculates summary statistics.
2. Statistical Analysis: Compares correlations across states and groups to identify significant differences.
3. Output Generation: Saves combined data, statistical results, and creates visualization plots.

Data Processing¶

1. Read Data: The tool reads correlation data from H5 files for each group.
2. Filter States: Only the user-specified states are included in the analysis.
3. Calculate Average Correlations: For each recording and state, the tool calculates average positive and negative correlations.
4. Calculate Cell Statistics: For each individual cell, the tool calculates the specified statistic (max, min, or mean) of that cell's correlations with other cells.

Statistical Comparisons¶

The tool uses functions from the pingouin package for statistical analysis.

1. Average Correlation Analysis:

   • Single Group: Compares average positive/negative correlations across states using a one-way Repeated Measures ANOVA (pingouin.rm_anova).
   • Two Groups (Paired): Compares average positive/negative correlations across states and groups using a two-way Repeated Measures ANOVA (pingouin.rm_anova).
   • Two Groups (Unpaired): Compares average positive/negative correlations across states (within-subject factor) and groups (between-subject factor) using a Mixed ANOVA (pingouin.mixed_anova).

2. Cell-level Statistic Analysis:

   • State Comparison: Uses Linear Mixed Models (LMM) (pingouin.linear_regression) to compare the cell-level statistic across states, accounting for within-subject variability and potentially group differences. This handles the nested structure (cells within subjects).
   • Group Comparison: First, averages the cell-level statistic per subject/state/group. Then, compares these subject-level averages between groups using ANOVA (RM-ANOVA for paired, Mixed ANOVA for unpaired).

3. Pairwise Comparisons: Following significant ANOVA or LMM results, pairwise tests (pingouin.pairwise_tests) are performed to pinpoint differences between specific states or groups. The user selects the multiple comparison correction method and effect size calculation.

Outputs¶

Combination Outputs¶

Combined Data (CSV)¶

Two types of combined data files are generated per group:

1. Average Correlations: A CSV file containing the calculated average positive and negative correlations for each recording and state.

2. Cell Statistic Correlations: A CSV file containing the calculated cell-level statistic (max, min, or mean) for each cell, state, and recording.

Example Average Correlation Data:

Example Cell Statistic Data (Maximum Correlation):

Combination Previews (SVG)¶

For each group, multiple visualization files are generated as previews for the combined data:

Average Correlation Previews: * Two boxplot visualizations for average positive and negative correlations.

Cell-level Statistic Previews: * One CDF plot showing the distribution of cell-level correlation statistics. * One boxplot with individual data points for cell-level correlation statistics.

These plots show the distribution of the respective correlation values across all recordings in the group, with separate lines or groupings colored by state.

Statistical Comparison Outputs¶

Statistical Results (CSV)¶

Two CSV files summarize the statistical tests:

1. ANOVA Results: Contains summary results from statistical tests comparing groups and states, including test statistics, p-values, and analysis parameters.

2. Pairwise Comparisons: Contains detailed results from follow-up tests that identify specific differences between conditions, including effect sizes and corrected p-values.

Note: The example tables below are illustrative and may not exactly match all columns generated by every possible test configuration.

Example ANOVA Results:

Example Pairwise Comparisons:

Statistical Comparison Previews (SVG)¶

Several plots visualize the statistical results:

1. Average Correlation Distribution: Shows how average positive and negative correlations differ across states and groups.

1. Correlation State Comparison: Displays cell-level statistic comparisons across different states.

1. Correlation Group Comparison: Shows how correlation statistics differ between experimental groups.