Skip to content

2. Decode

Jump to bottom
Diana Dima edited this page Mar 19, 2019 · 2 revisions

Functions in the Decode directory implement different types of decoding approaches in terms of feature selection (space/time) and cross-validation.

Functions in the main directory implement a linear SVM based on the LibLinear library. A LibSVM implementation is supported in the libsvm directory (which can run a non-linear kernel and also outputs AUC), and a slower Matlab implementation in the matlab directory. All functions take optional name-value pairs that are specified in the documentation for each function; these control both SVM and analysis parameters.

Functions

SVM decoding

Once the data has been reduced to a matrix of observations x features, we can train a SVM model to learn the relationship between features and labels, and evaluate its performance using cross-validation (CV). Three core functions perform these steps, and are also used when performing time and space-resolved decoding (see below).

  • svm_train Trains a SVM model with the specified parameters. The function runs the LibLinear train function and outputs a trained SVM model containing the bias and weights, which can then be tested on independent data.

    Some name-value pairs that can be adjusted are:

    • 'solver', default 1 (L2 regularized dual-problem)
    • 'boxconstraint', default 1 (C parameter controlling penalty on misclassified examples)
    • 'standardize', true (standardize the feature vectors based on mean and SD of the training set - recommended)
    • 'weights', false (calculate and output weights & activation patterns - see below)

  • svm_decode_holdout Trains SVM on a training dataset and tests it on a separate (held-out) dataset.

  • svm_decode_kfold Runs kfold cross-validation with specified number of folds and specified number of cross-validation iterations. Can create random stratified folds using Matlab's cvpartition or pre-specified folds.

    Some additional parameters than can be specified for cross-validation are:

    • 'iterate_cv', number of CV iterations to run - repeating this procedure can produce a more robust estimate, see overview here)
    • 'kfold', number of folds to use in k-fold cross-validation (default 5)
    • 'cv_indices', pre-supplied indices for cross-validation splits (indices x folds matrices); this is useful when we need to keep cross-validation folds constant for any reason

    The output results structure contains different fields with performance metrics and parameters (for each CV iteration):

    • Accuracy: proportion correctly classified observations out of total observations
    • AccuracyMSError: mean-squared error computed by LibLinear
    • AccuracyFold: accuracy for each CV fold
    • Confusion: confusion matrix showing performance for each class (explained here)
    • Sensitivity: classifier sensitivity, or true positive rate (definitions here)
    • Specificity: classifier specificity, or true negative rate
    • Fscore1: F-score (harmonic mean of precision and recall) for true positives
    • Fscore2: F-score for true negatives (this is less common, but important to obtain a balanced F-score)
    • WeightedFscore: Weighted F-score taking into account both class labels and their prevalence

    If requested ('weights', true), classifier weights are also output in 3 forms:

    • Weights: weight vector output after retraining the model on the entire dataset. (Note this is not true for svm_decode_holdout, since we are not using the whole dataset for training)
    • WeightPatterns: result of multiplying the weights by the data covariance matrix to obtain interpretable "activation" patterns (Haufe et al., 2014)
    • WeightPatternsNormalized: as above, normalized using min-max normalization

Time-resolved decoding

In MEG decoding, we are most often interested in the timing and localization of information that can be decoded with a classifier, so arguably the most important choices revolve around feature selection. A commonly used approach is to perform within-subject classification across time, by training and testing a classifier separately at each time-point or time-window and plotting the resulting performance as a timeseries.

For time- and space-resolved decoding, data will be in a channels/sources x time x trials format. Fieldtrip data structures can be brought into this format by running datamatrix = cat(3, data.trial{:}).

  • time_resolved_kfold Time-resolved decoding with k-fold cross-validation. Data is sensors/sources x time x trials. Can do feature selection, trial averaging, whitening. Decoding is done using independent windows of specified length.

  • time_resolved_holdout As above on separate training and test sets.

    Apart from the SVM parameters, you can specify name-value parameters including:

    • 'decoding_window' , epoch selected for decoding (in samples or timepoints)
    • 'window_length', number of samples used in classification at a time (default 1)
    • 'time', timepoints corresponding to samples (optional)
    • 'channels', channels or sources selected for decoding (numbers or MEG sensor labels)
    • 'sensor_idx', neighbours structure (output from e.g. get_sensor_info) for channel selection
    • 'pseudo', parameters for creating pseudotrials (subgroup size + number of permutations)
    • 'mnn', perform multivariate noise normalization (default true)

    The output results structure contains all the fields specified above, for each time window and CV iteration.

Temporal generalization

An alternative approach to time-resolved classification is cross-decoding across time points, by using each time point for training a separate model and testing it on held-out data from every other time point (temporal generalization). The results of this analysis can be visualized as a time x time matrix. Diagonal patterns of high accuracy are thought to suggest transient information, while square patterns point to information sustained across time.

The temporal_generalization function performs split-half validation by holding out half of the trials for testing. It returns an accuracy time x time matrix, as well as the training and testing trial indices. Note that apart from all other parameters described above, you can also specify 'train_idx' and 'test_idx' in advance to control the cross-validation scheme.

  • temporal_generalization Trains a model at each time point/window and tests it on all time points/windows. Implements a random split-half hold-out validation for speed.

Searchlight decoding

Searchlight analyses involve assigning each voxel/source/sensor to be the centroid of a cluster, and iteratively performing classification on each cluster. This method provides whole-brain coverage while overcoming the curse of dimensionality, but it is also computationally expensive and involves a multiple comparison problem (more details here). In MEG in particular, signal leakage and spatial resolution limitations can make searchlight results more difficult to interpret.

The searchlight decoding functions in the Decode directory implement both searchlight classification and an anatomical parcellation-based approach, where classification is performed separately for each of 90 AAL ROIs. (Other types of parcellation can be implemented by specifying the corresponding sensor/source indices). To perform spatial source selection, information about the layout of the sensors/sources needs to be obtained by running get_sensor_info or get_source_info.

Other parameters (preprocessing, temporal selection, SVM parameters) can be specified as described above. The outputs are a searchlight x time accuracy matrix and a corresponding F-score matrix.

  • searchlight_kfold Searchlight time-resolved decoding with k-fold cross-validation, based on spatial structure obtained from get_source_info/get_sensor_info. Can also do decoding at each sensor/source or using any specified sensor/source clusters as long as these are given in a cell array of the right size.

  • searchlight_holdout As above on separate training and test sets.

Clone this wiki locally