Goal: feature extraction for signals that correlate brain activity with typing words/sentences on keyboard
Approach:
Experiment
-
Start an lsl stream (device instructions)
- muse
pip install muselsl
muselsl stream
- neurosity (notion 1,2 & crown) I think the lsl stream is always active once devices are on (eventually want to use brainflow for this)
- muse
-
Prompt a word and have user type it
python.exe collect.py session00x
-
Record raw eeg epoch as user is typing
-
Python Keylogger with timestamps - https://github.com/tamaramueller/keylogger
-
-
choosing samples
- find timestamp for every valid word entry in keystroke data
- filter eeg data with keystroke data for every word sample
- pick each entry for every sample
-
filtering frequency
- use filtering methods in mne study template
- apply a band pass filter to remove harmoinic noise
-
selecting what channels to use
- visualize fft for a single sample and pick the channel with the highest (but not weirdest)
- x axis bins (0-55), y- axis voltage
- plot all channels together
- after picking a single channel, you're ready to send fft data for that to gpt
- visualize fft for a single sample and pick the channel with the highest (but not weirdest)
-
feeding data to gpt
-
get combined df for typing matchings (across different words)
- loop through word prompts
- apply current preprocessing and have a data set that contains "word prompt" & fft string pairs
- use 60:40 split for training and testing
-
send fft prompt for channel to gpt "wordprompt: easy; fft: [rrrbs,bsbssbsbs]"
- select frequencies between 2-50Hz
-
measure accuracy, precision & recall for examples
-
(misc)
- record more experiments and performance
- do the experiment a few different times on different days
- clean up scratch.ipynb into a shareable script
- consider transfer learning if gpt doesnt work well
Inference how do we do inteference realtime on notion
- https://github.com/neurosity/eeg-pipes
- https://github.com/neurosity/brainwaves-node/blob/master/lowAlpha.js
7.08.2022 - the goal for today is: - switch to brainflow for collecting data - run the collect.py and make sure - timestamp of prompts - csv data with keypress is included
8.08.2022 - get back to feature analyis - generate an input matrix that can be fed into a classifier - try multiple classifiers and measure