# Last edited on 2013-11-12 18:22:39 by stolfilocal

OBTAINING THE DATA FILES

  Copied the Ghislain EEG data from ~/Dropbox/eeg_sinal_ghislain/ 

    s{NS}_bl{NB}.txt     -- experimental block {NB} (1..2) of subject {NS} (1..8).
    s{NS}_bl{NB}_seq.log -- event log
    
  Subject 5 has a single data file called "s5_bl1-2.txt".

  Auxiliary files

    organizacao_arquivos.pdf -- the folder's "README".
    saunier2012.pdf -- article describing the experiment.
    OrdemDoCanal.txt -- Electrode names.
  
  Files "s{NS}_bl{NB}.txt" were compressed with gzip to save space.
  
  Counting the lines:
  
    for f in raw/s*_bl*.txt.gz ; do
      printf "%s " "${f}"
      cat ${f} | gunzip | wc -l
    done

     s1_bl1   282640
     s1_bl2   285220
     s2_bl1   295148
     s2_bl2   292860
     s3_bl1   281334
     s3_bl2   287132
     s4_bl1   290840
     s4_bl2   296645
     s5_bl1-2 555755
     s6_bl1   283420
     s6_bl2   290412
     s7_bl1   286880
     s7_bl2   287820
     s8_bl1   286700
     s8_bl2   286280

SPLITTING THE RUNS

  Each line of the files "s{NS}_bl{NB}.txt" has 21 numbers: measurements of 20 electrodes
  (microvolts?) and a "trigger" channel that is valued 0 or 1000.
  
  Each experimental block consists of 50 runs separated by 4-5 seconds.
  
  Each run lasts ~5.2 seconds and is marked by 4 short trigger pulses.
  
  Extracting the runs as separate files:
  
    extract_runs.sh > raw-runs/extract_runs.log 2>&1 
    echo 'done.'; egrep -e '[!*][!*]' raw-runs/extract_runs.log
    
  Checking statistics of trigger positions: 
  
    egrep -e 'trigger pulse  0' extract_runs.log \
      | gawk \
          ' BEGIN { mint = 999999; maxt = 0; } 
            //{ t = $(14); sumt += t; n++; 
                if (t < mint) { mint = t; }
                if (t > maxt) { maxt = t; }
              }
            END { printf "min = %10.4f max = %10.4f avg = %10.4f\n", mint, maxt, sumt/n; }
          '

  Pulses for most Subjects span ~5.060 sec (less than the 5.200 sec given in the paper).
  For Subject 2 they span ~5.400 sec.

IDENTIFYING THE RUNS

  Extracting the run types:
  
    grep type raw-runs/s???_r???.txt > raw-runs/run-types.txt

  Tabulating the range of electrodes Fp1 and Fp2 between trigger pulses 
  1 and 4, and flagging those that vary too much:
  
    vfile="raw-runs/Fp-variation.txt"
    rm -f ${vfile}
    for f in raw-runs/s_r???.txt ; do 
      printf "%s " "${f}" >> ${vfile}
      cat ${f} \
        | get_electrode_ranges.gawk -v ix1=4 -v ix2=12 -v ixt=21 -v dvmax=150 \
        >> ${vfile}
    done
  
FILTERING THE RUNS

  Running a bandpass filter over the extracted runs.
  
  First trial (Gislain's filter):
  
    filter_runs_A.sh

  The filtered runs are flt-runs-A/*.txt and the removed noise is in nse-runs-A/*.txt
  
  Again with somewhat tighter filter band:
  
    filter_runs_B.sh
    
  
  

SAMPLE RUNS FOR TESTING

  Identified two runs from each subject, one "Bio" and one "nonBio", both apparently
  free from blinks in the inter-pulse region.

    datadir=${HOME}/programs/c/NEUROMAT/neuromat_show_eeg/tests/data
    cp -av raw-runs/s001_r005.txt ${datadir} #  nonBio 
    cp -av raw-runs/s001_r039.txt ${datadir} #  Bio    
    cp -av raw-runs/s002_r016.txt ${datadir} #  Bio    
    cp -av raw-runs/s002_r043.txt ${datadir} #  nonBio 
    cp -av raw-runs/s003_r009.txt ${datadir} #  Bio    
    cp -av raw-runs/s003_r030.txt ${datadir} #  nonBio 
    cp -av raw-runs/s004_r030.txt ${datadir} #  nonBio 
    cp -av raw-runs/s004_r036.txt ${datadir} #  Bio    
    cp -av raw-runs/s005_r014.txt ${datadir} #  Bio    
    cp -av raw-runs/s005_r044.txt ${datadir} #  nonBio 
    cp -av raw-runs/s006_r024.txt ${datadir} #  Bio    
    cp -av raw-runs/s006_r042.txt ${datadir} #  nonBio 
    cp -av raw-runs/s007_r028.txt ${datadir} #  nonBio 
    cp -av raw-runs/s007_r046.txt ${datadir} #  Bio    
    cp -av raw-runs/s008_r015.txt ${datadir} #  Bio    
    cp -av raw-runs/s008_r049.txt ${datadir} #  nonBio