Multiple subject, single trial example¶
Conda environment¶
always activate a conda environment before starting. This notebook uses mkconda 0.0.11
Python Libraries¶
from pathlib import Path
from mkpy import mkh5
import pandas as pd
Build the file¶
There is now a loop to incorporate multiple subjects into one file. This makes the following changes:
A variable is added at the beginning with a list of the subjects you want to include in the loop/file (sids)
Creating the file names is under the loop for the data files (not the h5 files since there is one file)
mk5 commands to create the continuous file go under the loop
An ‘if’ statement is used for the append_mkdata command for an extra cal file
Commands after the continuous file is built (get_event_table, set_epochs, export_epochs) are done on the entire file and aren’t in the loop
If you need a refresher on any of the commands, see Binned, single subject example
A time function has been added at the top of the cell to demonstrate that the file now takes a while to build
Code Map:
For this example we will use the .xls format for the code map (see Code Maps xlsx example)
%%time
# set data directories
MKDIG_DIR = Path("../../mkdig")
MKPY_DIR = Path("../../mkpy")
# set the subject number(s) to loop across
sids = ['02', '03', '04', '05', '06', '07', '08', '09', '10',
'12', '13', '14', '15', '16', '17', '19', '20', '21',
'22', '23', '24', '25', '26', '27', '28', '30', '31',
'33', '34', '35']
# name files
h5_f = MKPY_DIR / ("stmath.h5")
epochtable = MKPY_DIR / ("stmath.epochs.h5")
# reset the .h5 file
myh5 = mkh5.mkh5(h5_f)
myh5.reset_all()
# loop over subjects to add to one h5 file
for sid in sids:
# subject number
sub = 'stm'+sid
# subject file names
yhdr = MKPY_DIR / (sub + ".yhdr")
eeg = MKDIG_DIR / (sub + ".crw")
log = MKDIG_DIR / (sub +".x.log")
# load in subject files
myh5.create_mkdata(sub, eeg, log, yhdr)
##stm03 has separate cal files, so append the data
if sub == 'stm03':
myh5.append_mkdata('stm03',
MKDIG_DIR / ("stm03cal.crw"),
MKDIG_DIR / ("stm03cal.x.log"),
MKPY_DIR / ("stm03.yhdr"))
# calibrate data
pts, pulse, lo, hi, ccode = 5, 10, -40, 40, 0
myh5.calibrate_mkdata(sub, # specific data group
n_points = pts, # pts to average
cal_size = pulse, # uV
lo_cursor = lo, # lo_cursor ms
hi_cursor = hi, # hi_cursor ms
cal_ccode= ccode) # condition code
# Find event codes in the data
event_table_raw = myh5.get_event_table(MKPY_DIR / ("stmath_code_map_new.xlsx"))
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:3706: UserWarning: negative event code(s) found for cal condition code 0 -16384 warnings.warn(msg)
Found cals in /stm02/dblock_5
Calibrating block /stm02/dblock_0 of 6: (148224,)
Calibrating block /stm02/dblock_1 of 6: (146944,)
Calibrating block /stm02/dblock_2 of 6: (148224,)
Calibrating block /stm02/dblock_3 of 6: (271104,)
Calibrating block /stm02/dblock_4 of 6: (122624,)
Calibrating block /stm02/dblock_5 of 6: (19200,)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:3406: LogRawEventCodeMismatch: ../../mkdig/stm03.crw eeg data underrun ../../mkdig/stm03.x.log, dropping trailing log events [[ 291 725055 0 0] [ 291 725180 0 0] [ 291 725305 0 0] [ 291 725430 0 0] [ 291 725555 0 0] [ 291 725685 0 0] [ 291 725810 0 0] [ 291 725935 0 0] [ 291 726060 0 0] [ 291 726185 0 0] [ 291 726311 0 0] [ 291 726436 0 0] [ 291 726561 0 0] [ 291 726686 0 0] [ 291 726811 0 0] [ 291 726936 0 0] [ 291 727061 0 0] [ 291 727187 0 0] [ 291 727312 0 0] [ 291 727437 0 0] [ 291 727562 0 0] [ 291 727687 0 0] [ 291 727812 0 0] [ 291 727938 0 0] [ 291 728063 0 0] [ 291 728188 0 0] [ 291 728313 0 0] [ 291 728438 0 0] [ 291 728563 0 0] [ 291 728689 0 0] [ 291 728814 0 0] [ 291 728939 0 0] [ 291 729064 0 0] [ 291 729189 0 0] [ 291 729314 0 0] [ 291 729440 0 0] [ 291 729565 0 0] [ 291 729690 0 0] [ 291 729815 0 0] [ 291 729940 0 0] [ 291 730065 0 0] [ 291 730191 0 0] [ 291 730316 0 0] [ 291 730441 0 0] [ 291 730566 0 0] [ 291 730691 0 0] [ 291 730816 0 0] [ 291 730942 0 0] [ 291 731067 0 0] [ 291 731192 0 0] [ 291 731317 0 0] [ 291 731442 0 0] [ 291 731567 0 0] [ 291 731693 0 0] [ 291 731818 0 0] [ 291 731943 0 0] [ 291 732068 0 0] [ 291 732193 0 0] [ 291 732318 0 0] [ 291 732443 0 0] [ 291 732569 0 0] [ 291 732694 0 0] [ 291 732819 0 0] [ 291 732944 0 0] [ 291 733069 0 0] [ 291 733194 0 0] [ 291 733320 0 0] [ 291 733445 0 0] [ 291 733570 0 0] [ 291 733695 0 0] [ 291 733820 0 0] [ 291 733945 0 0] [ 291 734071 0 0] [ 291 734196 0 0] [ 291 734321 0 0] [ 291 734446 0 0] [ 291 734571 0 0] [ 291 734696 0 0] [ 291 734822 0 0] [ 291 734947 0 0] [ 291 735072 0 0] [ 291 735197 0 0] [ 291 735322 0 0] [ 291 735447 0 0] [ 291 735573 0 0] [ 291 735698 0 0] [ 291 735823 0 0] [ 291 735948 0 0] [ 291 736073 0 0] [ 291 736198 0 0] [ 291 736324 0 0] [ 291 736449 0 0] [ 291 736574 0 0] [ 291 736699 0 0] [ 291 736824 0 0] [ 291 736949 0 0] [ 291 737074 0 0] [ 291 737200 0 0] [ 291 737325 0 0] [ 291 737450 0 0] [ 291 737575 0 0] [ 291 737700 0 0] [ 291 737825 0 0] [ 291 737951 0 0] [ 291 738076 0 0] [ 291 738201 0 0] [ 291 738326 0 0] [ 291 738451 0 0] [ 291 738576 0 0] [ 291 738702 0 0] [ 291 738827 0 0] [ 291 738952 0 0] [ 291 739077 0 0] [ 291 739202 0 0] [ 291 739327 0 0] [ 291 739453 0 0] [ 291 739578 0 0] [ 291 739703 0 0] [ 291 739828 0 0] [ 291 739953 0 0] [ 291 740078 0 0] [ 291 740204 0 0] [ 291 740329 0 0] [ 291 740454 0 0] [ 291 740579 0 0] [ 291 740704 0 0] [ 291 740829 0 0] [ 291 740955 0 0] [ 291 741080 0 0] [ 291 741205 0 0] [ 291 741330 0 0] [ 291 741455 0 0] [ 291 741580 0 0] [ 291 741705 0 0] [ 291 741831 0 0] [ 291 741956 0 0] [ 291 742081 0 0] [ 291 742206 0 0] [ 291 742331 0 0] [ 291 742456 0 0] [ 291 742582 0 0] [ 291 742707 0 0] [ 291 742832 0 0] [ 291 742957 0 0] [ 291 743082 0 0] [ 291 743207 0 0] [ 291 743333 0 0] [ 291 743458 0 0]] warnings.warn(msg, LogRawEventCodeMismatch)
Found cals in /stm03/dblock_8
Calibrating block /stm03/dblock_0 of 9: (31232,)
Calibrating block /stm03/dblock_1 of 9: (92160,)
Calibrating block /stm03/dblock_2 of 9: (121600,)
Calibrating block /stm03/dblock_3 of 9: (119040,)
Calibrating block /stm03/dblock_4 of 9: (84992,)
Calibrating block /stm03/dblock_5 of 9: (39680,)
Calibrating block /stm03/dblock_6 of 9: (116992,)
Calibrating block /stm03/dblock_7 of 9: (119296,)
Calibrating block /stm03/dblock_8 of 9: (19200,)
Found cals in /stm04/dblock_7
Calibrating block /stm04/dblock_0 of 8: (122368,)
Calibrating block /stm04/dblock_1 of 8: (118784,)
Calibrating block /stm04/dblock_2 of 8: (115968,)
Calibrating block /stm04/dblock_3 of 8: (116992,)
Calibrating block /stm04/dblock_4 of 8: (67328,)
Calibrating block /stm04/dblock_5 of 8: (47360,)
Calibrating block /stm04/dblock_6 of 8: (115200,)
Calibrating block /stm04/dblock_7 of 8: (17920,)
Found cals in /stm05/dblock_6
Calibrating block /stm05/dblock_0 of 7: (125952,)
Calibrating block /stm05/dblock_1 of 7: (122112,)
Calibrating block /stm05/dblock_2 of 7: (117504,)
Calibrating block /stm05/dblock_3 of 7: (118272,)
Calibrating block /stm05/dblock_4 of 7: (116224,)
Calibrating block /stm05/dblock_5 of 7: (117504,)
Calibrating block /stm05/dblock_6 of 7: (19968,)
Found cals in /stm06/dblock_10
Calibrating block /stm06/dblock_0 of 11: (63744,)
Calibrating block /stm06/dblock_1 of 11: (29440,)
Calibrating block /stm06/dblock_2 of 11: (12288,)
Calibrating block /stm06/dblock_3 of 11: (21760,)
Calibrating block /stm06/dblock_4 of 11: (123392,)
Calibrating block /stm06/dblock_5 of 11: (5632,)
Calibrating block /stm06/dblock_6 of 11: (129280,)
Calibrating block /stm06/dblock_7 of 11: (135680,)
Calibrating block /stm06/dblock_8 of 11: (136960,)
Calibrating block /stm06/dblock_9 of 11: (136704,)
Calibrating block /stm06/dblock_10 of 11: (18688,)
Found cals in /stm07/dblock_6
Calibrating block /stm07/dblock_0 of 7: (122112,)
Calibrating block /stm07/dblock_1 of 7: (117760,)
Calibrating block /stm07/dblock_2 of 7: (117760,)
Calibrating block /stm07/dblock_3 of 7: (116480,)
Calibrating block /stm07/dblock_4 of 7: (118016,)
Calibrating block /stm07/dblock_5 of 7: (120064,)
Calibrating block /stm07/dblock_6 of 7: (18944,)
Found cals in /stm08/dblock_10
Calibrating block /stm08/dblock_0 of 11: (82176,)
Calibrating block /stm08/dblock_1 of 11: (40704,)
Calibrating block /stm08/dblock_2 of 11: (122368,)
Calibrating block /stm08/dblock_3 of 11: (70144,)
Calibrating block /stm08/dblock_4 of 11: (58368,)
Calibrating block /stm08/dblock_5 of 11: (121088,)
Calibrating block /stm08/dblock_6 of 11: (5120,)
Calibrating block /stm08/dblock_7 of 11: (117760,)
Calibrating block /stm08/dblock_8 of 11: (81152,)
Calibrating block /stm08/dblock_9 of 11: (40448,)
Calibrating block /stm08/dblock_10 of 11: (20992,)
Found cals in /stm09/dblock_6
Calibrating block /stm09/dblock_0 of 7: (132352,)
Calibrating block /stm09/dblock_1 of 7: (122624,)
Calibrating block /stm09/dblock_2 of 7: (130816,)
Calibrating block /stm09/dblock_3 of 7: (140800,)
Calibrating block /stm09/dblock_4 of 7: (143872,)
Calibrating block /stm09/dblock_5 of 7: (157952,)
Calibrating block /stm09/dblock_6 of 7: (18688,)
Found cals in /stm10/dblock_6
Calibrating block /stm10/dblock_0 of 7: (117504,)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:3791: UserWarning: dropping 1 cal epochs out of data bounds len(fails)
Calibrating block /stm10/dblock_1 of 7: (116736,)
Calibrating block /stm10/dblock_2 of 7: (114688,)
Calibrating block /stm10/dblock_3 of 7: (119808,)
Calibrating block /stm10/dblock_4 of 7: (115200,)
Calibrating block /stm10/dblock_5 of 7: (114944,)
Calibrating block /stm10/dblock_6 of 7: (18944,)
Found cals in /stm12/dblock_8
Calibrating block /stm12/dblock_0 of 9: (125440,)
Calibrating block /stm12/dblock_1 of 9: (4608,)
Calibrating block /stm12/dblock_2 of 9: (123904,)
Calibrating block /stm12/dblock_3 of 9: (121088,)
Calibrating block /stm12/dblock_4 of 9: (123136,)
Calibrating block /stm12/dblock_5 of 9: (38144,)
Calibrating block /stm12/dblock_6 of 9: (83456,)
Calibrating block /stm12/dblock_7 of 9: (117760,)
Calibrating block /stm12/dblock_8 of 9: (18688,)
Found cals in /stm13/dblock_7
Calibrating block /stm13/dblock_0 of 8: (37632,)
Calibrating block /stm13/dblock_1 of 8: (67072,)
Calibrating block /stm13/dblock_2 of 8: (48640,)
Calibrating block /stm13/dblock_3 of 8: (279552,)
Calibrating block /stm13/dblock_4 of 8: (126720,)
Calibrating block /stm13/dblock_5 of 8: (130816,)
Calibrating block /stm13/dblock_6 of 8: (121856,)
Calibrating block /stm13/dblock_7 of 8: (20480,)
Found cals in /stm14/dblock_7
Calibrating block /stm14/dblock_0 of 8: (149504,)
Calibrating block /stm14/dblock_1 of 8: (155392,)
Calibrating block /stm14/dblock_2 of 8: (100096,)
Calibrating block /stm14/dblock_3 of 8: (43008,)
Calibrating block /stm14/dblock_4 of 8: (136192,)
Calibrating block /stm14/dblock_5 of 8: (136704,)
Calibrating block /stm14/dblock_6 of 8: (128768,)
Calibrating block /stm14/dblock_7 of 8: (18688,)
Found cals in /stm15/dblock_5
Calibrating block /stm15/dblock_0 of 6: (255744,)
Calibrating block /stm15/dblock_1 of 6: (123904,)
Calibrating block /stm15/dblock_2 of 6: (127232,)
Calibrating block /stm15/dblock_3 of 6: (118528,)
Calibrating block /stm15/dblock_4 of 6: (125184,)
Calibrating block /stm15/dblock_5 of 6: (20224,)
Found cals in /stm16/dblock_7
Calibrating block /stm16/dblock_0 of 8: (121088,)
Calibrating block /stm16/dblock_1 of 8: (72960,)
Calibrating block /stm16/dblock_2 of 8: (50688,)
Calibrating block /stm16/dblock_3 of 8: (127744,)
Calibrating block /stm16/dblock_4 of 8: (122368,)
Calibrating block /stm16/dblock_5 of 8: (121600,)
Calibrating block /stm16/dblock_6 of 8: (119296,)
Calibrating block /stm16/dblock_7 of 8: (20480,)
Found cals in /stm17/dblock_7
Calibrating block /stm17/dblock_0 of 8: (10240,)
Calibrating block /stm17/dblock_1 of 8: (127744,)
Calibrating block /stm17/dblock_2 of 8: (128768,)
Calibrating block /stm17/dblock_3 of 8: (124672,)
Calibrating block /stm17/dblock_4 of 8: (52224,)
Calibrating block /stm17/dblock_5 of 8: (1792,)
Calibrating block /stm17/dblock_6 of 8: (328448,)
Calibrating block /stm17/dblock_7 of 8: (21248,)
Found cals in /stm19/dblock_6
Calibrating block /stm19/dblock_0 of 7: (104448,)
Calibrating block /stm19/dblock_1 of 7: (128256,)
Calibrating block /stm19/dblock_2 of 7: (124672,)
Calibrating block /stm19/dblock_3 of 7: (121344,)
Calibrating block /stm19/dblock_4 of 7: (116480,)
Calibrating block /stm19/dblock_5 of 7: (116224,)
Calibrating block /stm19/dblock_6 of 7: (25344,)
Found cals in /stm20/dblock_5
Calibrating block /stm20/dblock_0 of 6: (105984,)
Calibrating block /stm20/dblock_1 of 6: (260864,)
Calibrating block /stm20/dblock_2 of 6: (121856,)
Calibrating block /stm20/dblock_3 of 6: (121600,)
Calibrating block /stm20/dblock_4 of 6: (124160,)
Calibrating block /stm20/dblock_5 of 6: (18944,)
Found cals in /stm21/dblock_8
Calibrating block /stm21/dblock_0 of 9: (18688,)
Calibrating block /stm21/dblock_1 of 9: (17408,)
Calibrating block /stm21/dblock_2 of 9: (140032,)
Calibrating block /stm21/dblock_3 of 9: (156928,)
Calibrating block /stm21/dblock_4 of 9: (161280,)
Calibrating block /stm21/dblock_5 of 9: (167936,)
Calibrating block /stm21/dblock_6 of 9: (159488,)
Calibrating block /stm21/dblock_7 of 9: (160768,)
Calibrating block /stm21/dblock_8 of 9: (20224,)
Found cals in /stm22/dblock_6
Calibrating block /stm22/dblock_0 of 7: (166656,)
Calibrating block /stm22/dblock_1 of 7: (173824,)
Calibrating block /stm22/dblock_2 of 7: (185856,)
Calibrating block /stm22/dblock_3 of 7: (170752,)
Calibrating block /stm22/dblock_4 of 7: (160768,)
Calibrating block /stm22/dblock_5 of 7: (166144,)
Calibrating block /stm22/dblock_6 of 7: (18688,)
Found cals in /stm23/dblock_7
Calibrating block /stm23/dblock_0 of 8: (108800,)
Calibrating block /stm23/dblock_1 of 8: (41984,)
Calibrating block /stm23/dblock_2 of 8: (176896,)
Calibrating block /stm23/dblock_3 of 8: (178176,)
Calibrating block /stm23/dblock_4 of 8: (178944,)
Calibrating block /stm23/dblock_5 of 8: (146432,)
Calibrating block /stm23/dblock_6 of 8: (134144,)
Calibrating block /stm23/dblock_7 of 8: (17920,)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:3437: LogRawEventCodeMismatch: These log event codes differ from the EEG codes, make sure you know why
{mismatch_events}
warnings.warn(mismatch_msg, LogRawEventCodeMismatch)
Found cals in /stm24/dblock_6
Calibrating block /stm24/dblock_0 of 7: (132864,)
Calibrating block /stm24/dblock_1 of 7: (132352,)
Calibrating block /stm24/dblock_2 of 7: (135680,)
Calibrating block /stm24/dblock_3 of 7: (134144,)
Calibrating block /stm24/dblock_4 of 7: (132864,)
Calibrating block /stm24/dblock_5 of 7: (132352,)
Calibrating block /stm24/dblock_6 of 7: (18688,)
Found cals in /stm25/dblock_6
Calibrating block /stm25/dblock_0 of 7: (120576,)
Calibrating block /stm25/dblock_1 of 7: (125440,)
Calibrating block /stm25/dblock_2 of 7: (121344,)
Calibrating block /stm25/dblock_3 of 7: (122624,)
Calibrating block /stm25/dblock_4 of 7: (121344,)
Calibrating block /stm25/dblock_5 of 7: (115968,)
Calibrating block /stm25/dblock_6 of 7: (18176,)
Found cals in /stm26/dblock_8
Calibrating block /stm26/dblock_0 of 9: (120320,)
Calibrating block /stm26/dblock_1 of 9: (11520,)
Calibrating block /stm26/dblock_2 of 9: (122880,)
Calibrating block /stm26/dblock_3 of 9: (146688,)
Calibrating block /stm26/dblock_4 of 9: (156160,)
Calibrating block /stm26/dblock_5 of 9: (51456,)
Calibrating block /stm26/dblock_6 of 9: (100096,)
Calibrating block /stm26/dblock_7 of 9: (150272,)
Calibrating block /stm26/dblock_8 of 9: (19200,)
Found cals in /stm27/dblock_7
Calibrating block /stm27/dblock_0 of 8: (50944,)
Calibrating block /stm27/dblock_1 of 8: (131328,)
Calibrating block /stm27/dblock_2 of 8: (158976,)
Calibrating block /stm27/dblock_3 of 8: (157440,)
Calibrating block /stm27/dblock_4 of 8: (172800,)
Calibrating block /stm27/dblock_5 of 8: (165120,)
Calibrating block /stm27/dblock_6 of 8: (140032,)
Calibrating block /stm27/dblock_7 of 8: (18432,)
Found cals in /stm28/dblock_9
Calibrating block /stm28/dblock_0 of 10: (38912,)
Calibrating block /stm28/dblock_1 of 10: (62720,)
Calibrating block /stm28/dblock_2 of 10: (30464,)
Calibrating block /stm28/dblock_3 of 10: (7424,)
Calibrating block /stm28/dblock_4 of 10: (135424,)
Calibrating block /stm28/dblock_5 of 10: (146432,)
Calibrating block /stm28/dblock_6 of 10: (135168,)
Calibrating block /stm28/dblock_7 of 10: (137216,)
Calibrating block /stm28/dblock_8 of 10: (136704,)
Calibrating block /stm28/dblock_9 of 10: (18688,)
Found cals in /stm30/dblock_6
Calibrating block /stm30/dblock_0 of 7: (200448,)
Calibrating block /stm30/dblock_1 of 7: (167936,)
Calibrating block /stm30/dblock_2 of 7: (153600,)
Calibrating block /stm30/dblock_3 of 7: (150528,)
Calibrating block /stm30/dblock_4 of 7: (142336,)
Calibrating block /stm30/dblock_5 of 7: (133376,)
Calibrating block /stm30/dblock_6 of 7: (18944,)
Found cals in /stm31/dblock_5
Calibrating block /stm31/dblock_0 of 6: (254464,)
Calibrating block /stm31/dblock_1 of 6: (119040,)
Calibrating block /stm31/dblock_2 of 6: (134656,)
Calibrating block /stm31/dblock_3 of 6: (128512,)
Calibrating block /stm31/dblock_4 of 6: (125696,)
Calibrating block /stm31/dblock_5 of 6: (18944,)
Found cals in /stm33/dblock_8
Calibrating block /stm33/dblock_0 of 9: (129792,)
Calibrating block /stm33/dblock_1 of 9: (112896,)
Calibrating block /stm33/dblock_2 of 9: (4352,)
Calibrating block /stm33/dblock_3 of 9: (5632,)
Calibrating block /stm33/dblock_4 of 9: (122880,)
Calibrating block /stm33/dblock_5 of 9: (122112,)
Calibrating block /stm33/dblock_6 of 9: (118272,)
Calibrating block /stm33/dblock_7 of 9: (116736,)
Calibrating block /stm33/dblock_8 of 9: (18944,)
Found cals in /stm34/dblock_7
Calibrating block /stm34/dblock_0 of 8: (129024,)
Calibrating block /stm34/dblock_1 of 8: (127232,)
Calibrating block /stm34/dblock_2 of 8: (133632,)
Calibrating block /stm34/dblock_3 of 8: (54016,)
Calibrating block /stm34/dblock_4 of 8: (79616,)
Calibrating block /stm34/dblock_5 of 8: (138752,)
Calibrating block /stm34/dblock_6 of 8: (126720,)
Calibrating block /stm34/dblock_7 of 8: (19968,)
Found cals in /stm35/dblock_6
Calibrating block /stm35/dblock_0 of 7: (143872,)
Calibrating block /stm35/dblock_1 of 7: (138496,)
Calibrating block /stm35/dblock_2 of 7: (142336,)
Calibrating block /stm35/dblock_3 of 7: (145920,)
Calibrating block /stm35/dblock_4 of 7: (146688,)
Calibrating block /stm35/dblock_5 of 7: (141568,)
Calibrating block /stm35/dblock_6 of 7: (18944,)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1060: UserWarning: As of mkpy 0.2.0 to match events with a codemap regexp pattern, the ccode column in stmath_code_map_new.xlsx must also match the log_ccode in the datablock. If this behavior is not desired, delete or rename the ccode column in the codemap. warnings.warn(msg)
searching codes in: stm02/dblock_0
searching codes in: stm02/dblock_1
searching codes in: stm02/dblock_2
searching codes in: stm02/dblock_3
searching codes in: stm02/dblock_4
searching codes in: stm02/dblock_5
searching codes in: stm03/dblock_0
searching codes in: stm03/dblock_1
searching codes in: stm03/dblock_2
searching codes in: stm03/dblock_3
searching codes in: stm03/dblock_4
searching codes in: stm03/dblock_5
searching codes in: stm03/dblock_6
searching codes in: stm03/dblock_7
searching codes in: stm03/dblock_8
searching codes in: stm04/dblock_0
searching codes in: stm04/dblock_1
searching codes in: stm04/dblock_2
searching codes in: stm04/dblock_3
searching codes in: stm04/dblock_4
searching codes in: stm04/dblock_5
searching codes in: stm04/dblock_6
searching codes in: stm04/dblock_7
searching codes in: stm05/dblock_0
searching codes in: stm05/dblock_1
searching codes in: stm05/dblock_2
searching codes in: stm05/dblock_3
searching codes in: stm05/dblock_4
searching codes in: stm05/dblock_5
searching codes in: stm05/dblock_6
searching codes in: stm06/dblock_0
searching codes in: stm06/dblock_1
searching codes in: stm06/dblock_2
searching codes in: stm06/dblock_3
searching codes in: stm06/dblock_4
searching codes in: stm06/dblock_5
searching codes in: stm06/dblock_6
searching codes in: stm06/dblock_7
searching codes in: stm06/dblock_8
searching codes in: stm06/dblock_9
searching codes in: stm06/dblock_10
searching codes in: stm07/dblock_0
searching codes in: stm07/dblock_1
searching codes in: stm07/dblock_2
searching codes in: stm07/dblock_3
searching codes in: stm07/dblock_4
searching codes in: stm07/dblock_5
searching codes in: stm07/dblock_6
searching codes in: stm08/dblock_0
searching codes in: stm08/dblock_1
searching codes in: stm08/dblock_2
searching codes in: stm08/dblock_3
searching codes in: stm08/dblock_4
searching codes in: stm08/dblock_5
searching codes in: stm08/dblock_6
searching codes in: stm08/dblock_7
searching codes in: stm08/dblock_8
searching codes in: stm08/dblock_9
searching codes in: stm08/dblock_10
searching codes in: stm09/dblock_0
searching codes in: stm09/dblock_1
searching codes in: stm09/dblock_2
searching codes in: stm09/dblock_3
searching codes in: stm09/dblock_4
searching codes in: stm09/dblock_5
searching codes in: stm09/dblock_6
searching codes in: stm10/dblock_0
searching codes in: stm10/dblock_1
searching codes in: stm10/dblock_2
searching codes in: stm10/dblock_3
searching codes in: stm10/dblock_4
searching codes in: stm10/dblock_5
searching codes in: stm10/dblock_6
searching codes in: stm12/dblock_0
searching codes in: stm12/dblock_1
searching codes in: stm12/dblock_2
searching codes in: stm12/dblock_3
searching codes in: stm12/dblock_4
searching codes in: stm12/dblock_5
searching codes in: stm12/dblock_6
searching codes in: stm12/dblock_7
searching codes in: stm12/dblock_8
searching codes in: stm13/dblock_0
searching codes in: stm13/dblock_1
searching codes in: stm13/dblock_2
searching codes in: stm13/dblock_3
searching codes in: stm13/dblock_4
searching codes in: stm13/dblock_5
searching codes in: stm13/dblock_6
searching codes in: stm13/dblock_7
searching codes in: stm14/dblock_0
searching codes in: stm14/dblock_1
searching codes in: stm14/dblock_2
searching codes in: stm14/dblock_3
searching codes in: stm14/dblock_4
searching codes in: stm14/dblock_5
searching codes in: stm14/dblock_6
searching codes in: stm14/dblock_7
searching codes in: stm15/dblock_0
searching codes in: stm15/dblock_1
searching codes in: stm15/dblock_2
searching codes in: stm15/dblock_3
searching codes in: stm15/dblock_4
searching codes in: stm15/dblock_5
searching codes in: stm16/dblock_0
searching codes in: stm16/dblock_1
searching codes in: stm16/dblock_2
searching codes in: stm16/dblock_3
searching codes in: stm16/dblock_4
searching codes in: stm16/dblock_5
searching codes in: stm16/dblock_6
searching codes in: stm16/dblock_7
searching codes in: stm17/dblock_0
searching codes in: stm17/dblock_1
searching codes in: stm17/dblock_2
searching codes in: stm17/dblock_3
searching codes in: stm17/dblock_4
searching codes in: stm17/dblock_5
searching codes in: stm17/dblock_6
searching codes in: stm17/dblock_7
searching codes in: stm19/dblock_0
searching codes in: stm19/dblock_1
searching codes in: stm19/dblock_2
searching codes in: stm19/dblock_3
searching codes in: stm19/dblock_4
searching codes in: stm19/dblock_5
searching codes in: stm19/dblock_6
searching codes in: stm20/dblock_0
searching codes in: stm20/dblock_1
searching codes in: stm20/dblock_2
searching codes in: stm20/dblock_3
searching codes in: stm20/dblock_4
searching codes in: stm20/dblock_5
searching codes in: stm21/dblock_0
searching codes in: stm21/dblock_1
searching codes in: stm21/dblock_2
searching codes in: stm21/dblock_3
searching codes in: stm21/dblock_4
searching codes in: stm21/dblock_5
searching codes in: stm21/dblock_6
searching codes in: stm21/dblock_7
searching codes in: stm21/dblock_8
searching codes in: stm22/dblock_0
searching codes in: stm22/dblock_1
searching codes in: stm22/dblock_2
searching codes in: stm22/dblock_3
searching codes in: stm22/dblock_4
searching codes in: stm22/dblock_5
searching codes in: stm22/dblock_6
searching codes in: stm23/dblock_0
searching codes in: stm23/dblock_1
searching codes in: stm23/dblock_2
searching codes in: stm23/dblock_3
searching codes in: stm23/dblock_4
searching codes in: stm23/dblock_5
searching codes in: stm23/dblock_6
searching codes in: stm23/dblock_7
searching codes in: stm24/dblock_0
searching codes in: stm24/dblock_1
searching codes in: stm24/dblock_2
searching codes in: stm24/dblock_3
searching codes in: stm24/dblock_4
searching codes in: stm24/dblock_5
searching codes in: stm24/dblock_6
searching codes in: stm25/dblock_0
searching codes in: stm25/dblock_1
searching codes in: stm25/dblock_2
searching codes in: stm25/dblock_3
searching codes in: stm25/dblock_4
searching codes in: stm25/dblock_5
searching codes in: stm25/dblock_6
searching codes in: stm26/dblock_0
searching codes in: stm26/dblock_1
searching codes in: stm26/dblock_2
searching codes in: stm26/dblock_3
searching codes in: stm26/dblock_4
searching codes in: stm26/dblock_5
searching codes in: stm26/dblock_6
searching codes in: stm26/dblock_7
searching codes in: stm26/dblock_8
searching codes in: stm27/dblock_0
searching codes in: stm27/dblock_1
searching codes in: stm27/dblock_2
searching codes in: stm27/dblock_3
searching codes in: stm27/dblock_4
searching codes in: stm27/dblock_5
searching codes in: stm27/dblock_6
searching codes in: stm27/dblock_7
searching codes in: stm28/dblock_0
searching codes in: stm28/dblock_1
searching codes in: stm28/dblock_2
searching codes in: stm28/dblock_3
searching codes in: stm28/dblock_4
searching codes in: stm28/dblock_5
searching codes in: stm28/dblock_6
searching codes in: stm28/dblock_7
searching codes in: stm28/dblock_8
searching codes in: stm28/dblock_9
searching codes in: stm30/dblock_0
searching codes in: stm30/dblock_1
searching codes in: stm30/dblock_2
searching codes in: stm30/dblock_3
searching codes in: stm30/dblock_4
searching codes in: stm30/dblock_5
searching codes in: stm30/dblock_6
searching codes in: stm31/dblock_0
searching codes in: stm31/dblock_1
searching codes in: stm31/dblock_2
searching codes in: stm31/dblock_3
searching codes in: stm31/dblock_4
searching codes in: stm31/dblock_5
searching codes in: stm33/dblock_0
searching codes in: stm33/dblock_1
searching codes in: stm33/dblock_2
searching codes in: stm33/dblock_3
searching codes in: stm33/dblock_4
searching codes in: stm33/dblock_5
searching codes in: stm33/dblock_6
searching codes in: stm33/dblock_7
searching codes in: stm33/dblock_8
searching codes in: stm34/dblock_0
searching codes in: stm34/dblock_1
searching codes in: stm34/dblock_2
searching codes in: stm34/dblock_3
searching codes in: stm34/dblock_4
searching codes in: stm34/dblock_5
searching codes in: stm34/dblock_6
searching codes in: stm34/dblock_7
searching codes in: stm35/dblock_0
searching codes in: stm35/dblock_1
searching codes in: stm35/dblock_2
searching codes in: stm35/dblock_3
searching codes in: stm35/dblock_4
searching codes in: stm35/dblock_5
searching codes in: stm35/dblock_6
CPU times: user 1h 57min 49s, sys: 1min 14s, total: 1h 59min 3s
Wall time: 1h 59min 8s
Note
The warnings above:
LogRawEventCodeMismatch is to check the data and make sure it is okay that the raw and log file event codes mismatch. This happens sometimes when you pause DIG and should be fine as long as the mismatch is trailing events (as in this case).
dropping out of bounds happens when the pause mark happened outside of the window you set for cals to be measured (i.e., -40ms to 40ms)
Data tranformations to the event table¶
You do not have to do this step, but it is useful to know that you can ‘break in’ to the pandas data frame to add more data or drop rows to make the epochs file not as large when you export it.
Add extra variables across all columns¶
These are new variables that you want columns for that are not imported with the header or code map (useful if you are combining multiple experiments but want one code map)
# show the column names before adding variables
display(event_table_raw.columns)
Index(['data_group', 'dblock_path', 'dblock_tick_idx', 'dblock_ticks',
'crw_ticks', 'raw_evcodes', 'log_evcodes', 'log_ccodes', 'log_flags',
'epoch_match_tick_delta', 'epoch_ticks', 'dblock_srate', 'match_group',
'idx', 'dlim', 'anchor_str', 'match_str', 'anchor_code', 'match_code',
'anchor_tick', 'match_tick', 'anchor_tick_delta', 'is_anchor', 'ccode',
'regexp', 'List', 'ThreatCondition', 'Item', 'Stimulus', 'Operand1',
'Operand2', 'Operand3', 'ShownProduct', 'CorrectProduct', 'Condition',
'ResponseAccuracy', 'Anticipation', 'Certainty'],
dtype='object')
# code for experiment
event_table_raw['expt'] = "expt_1"
# show the a few variables from the data frame with the new column added
event_table_raw[['data_group','crw_ticks','regexp','match_code','Stimulus','expt', 'anchor_tick_delta']].head(6)
| data_group | crw_ticks | regexp | match_code | Stimulus | expt | anchor_tick_delta | |
|---|---|---|---|---|---|---|---|
| 0 | stm02 | 115262 | 1 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O1 | expt_1 | 0 |
| 1 | stm02 | 115417 | 2 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O2 | expt_1 | 0 |
| 2 | stm02 | 115571 | 3 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O3 | expt_1 | 0 |
| 3 | stm02 | 115725 | 4 (12001) (#22001) (2064) 777 1040 | 12001 | P | expt_1 | -625 |
| 4 | stm02 | 116350 | 4 (12001) (#22001) (2064) 777 1040 | 22001 | P | expt_1 | 0 |
| 5 | stm02 | 116764 | 4 (12001) (#22001) (2064) 777 1040 | 2064 | P | expt_1 | 414 |
Get reaction times¶
Note: this example has an event in between the one we want to time lock too and the one we want to get a reaction time for. If you do not have this problem, then you just need to drop the extra capture group lines.
Use code similar to the first line below (.query)
Query for anchor_tick_delta > or < 0 depending on where you put the anchor (#)
# pull out the rows of the dataframe with positive response ticks(i.e, after the anchor) and copy to new dataframe
rt_df = event_table_raw.query('anchor_tick_delta > 0').copy()
# rename anchor tick delta to response ticks, so variables don't get overwritten on merging
rt_df["response_ticks"] = rt_df["anchor_tick_delta"]
# select on the variables needed for merging
rt_df = rt_df.loc[:, ['data_group','response_ticks','anchor_code']]
# set the index for merging later
rt_df.set_index(['data_group','anchor_code'],inplace=True)
# check the dataframe has the variables you want for merging later
rt_df.head()
| response_ticks | ||
|---|---|---|
| data_group | anchor_code | |
| stm02 | 22001 | 414 |
| 21003 | 899 | |
| 22007 | 417 | |
| 23014 | 374 | |
| 22015 | 402 |
#get clean rows of the dataframe for merging (i.e., drop the two capture groups that shouldn't have time-locked epochs)
clean_events = event_table_raw.query('log_evcodes < 20000 and anchor_tick_delta <= 0')
# set the index for merging later
clean_events.set_index(['data_group','anchor_code'],inplace=True)
# check the dataframe has the variables you want for merging later; display only a few variables for demonstration
# anchor_tick_delta is negative for the product event because the time-locked event is before the anchor code
clean_events[['crw_ticks','regexp','match_code','Stimulus','expt', 'anchor_tick_delta']].head(6)
| crw_ticks | regexp | match_code | Stimulus | expt | anchor_tick_delta | ||
|---|---|---|---|---|---|---|---|
| data_group | anchor_code | ||||||
| stm02 | 2001 | 115262 | 1 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O1 | expt_1 | 0 |
| 2001 | 115417 | 2 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O2 | expt_1 | 0 | |
| 2001 | 115571 | 3 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O3 | expt_1 | 0 | |
| 22001 | 115725 | 4 (12001) (#22001) (2064) 777 1040 | 12001 | P | expt_1 | -625 | |
| 1003 | 21491 | 1 (#1003) .* 11003 21003 1040 777 1040 | 1003 | O1 | expt_1 | 0 | |
| 1003 | 21649 | 2 (#1003) .* 11003 21003 1040 777 1040 | 1003 | O2 | expt_1 | 0 |
# merge response ticks and clean dataframes together
clean_rt = clean_events.join(rt_df,on=['data_group','anchor_code'])
# reset the index to the default (the rest of mkpy expects a certain index structure)
clean_rt.reset_index(['anchor_code'],inplace=True)
# check the dataframe; display only a few variables for demonstration
# response ticks only merge onto the product rows since that is what the response is to
clean_rt[['crw_ticks','regexp','match_code','Stimulus', 'expt', 'anchor_tick_delta','response_ticks']].head(8)
| crw_ticks | regexp | match_code | Stimulus | expt | anchor_tick_delta | response_ticks | |
|---|---|---|---|---|---|---|---|
| data_group | |||||||
| stm02 | 115262 | 1 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O1 | expt_1 | 0 | NaN |
| stm02 | 115417 | 2 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O2 | expt_1 | 0 | NaN |
| stm02 | 115571 | 3 (#2001) .* 12001 22001 2064 777 1040 | 2001 | O3 | expt_1 | 0 | NaN |
| stm02 | 115725 | 4 (12001) (#22001) (2064) 777 1040 | 12001 | P | expt_1 | -625 | 414.0 |
| stm02 | 21491 | 1 (#1003) .* 11003 21003 1040 777 1040 | 1003 | O1 | expt_1 | 0 | NaN |
| stm02 | 21649 | 2 (#1003) .* 11003 21003 1040 777 1040 | 1003 | O2 | expt_1 | 0 | NaN |
| stm02 | 21803 | 3 (#1003) .* 11003 21003 1040 777 1040 | 1003 | O3 | expt_1 | 0 | NaN |
| stm02 | 21958 | 4 (11003) (#21003) (1040) 777 1040 | 11003 | P | expt_1 | -625 | 899.0 |
Merge in data from Excel¶
# set columns of interest from the spreadsheet
npsych_coi = [
'Subj', 'Gndr', 'Age', 'NativeLang', 'Bilingual', 'Major', 'GoodMath',
'Rhand','Lhand', 'Famhand', 'WhoLH', 'ARTCorrect', 'ARTFoils',
'MRTCorrect','MRTFoils', 'MathAnxiety1', 'MathAnxiety2',
'BEM_M', 'BEM_F', 'BEM_N','DS_F', 'DS_B', 'DS_H'
]
# read the Excel spreadsheet into pandas and do some data transformation;
# look up pd.read_excel for arguments
stmath_neuro = (
pd.read_excel(
"/home/kadelong/Exps/STMath/Neuro/Neuro.xlsx",
header=2,
nrows=30)
.loc[:, npsych_coi]
.rename(columns={"Subj": "data_group"})
.set_index('data_group')
)
# check the data imported properly; display only a few variables for demo
stmath_neuro[['Rhand','Lhand','ARTCorrect','MRTCorrect', 'DS_F']].head()
| Rhand | Lhand | ARTCorrect | MRTCorrect | DS_F | |
|---|---|---|---|---|---|
| data_group | |||||
| stm02 | 24 | 6 | 7 | 5 | 8 |
| stm03 | 26 | 3 | 1 | 2 | 10 |
| stm04 | 21 | 5 | 6 | 5 | 13 |
| stm05 | 20 | 7 | 5 | 8 | 9 |
| stm06 | 29 | 9 | 7 | 12 | 8 |
Merge dataframes together¶
This gets a final event_table dataframe for exporting by merging all the other dataframes you already created.
# merge neuropsych dataframe with response ticks dataframe
event_table=clean_rt.join(stmath_neuro, on="data_group")
# reset the index to the default (the rest of mkpy expects a certain index structure)
event_table.reset_index(['data_group'],inplace=True)
# display columns and a few variables to check everything merged correctly
display(event_table.columns)
event_table[['data_group','crw_ticks','regexp','Stimulus','expt','response_ticks','Rhand','Lhand']].head(8)
Index(['data_group', 'anchor_code', 'dblock_path', 'dblock_tick_idx',
'dblock_ticks', 'crw_ticks', 'raw_evcodes', 'log_evcodes', 'log_ccodes',
'log_flags', 'epoch_match_tick_delta', 'epoch_ticks', 'dblock_srate',
'match_group', 'idx', 'dlim', 'anchor_str', 'match_str', 'match_code',
'anchor_tick', 'match_tick', 'anchor_tick_delta', 'is_anchor', 'ccode',
'regexp', 'List', 'ThreatCondition', 'Item', 'Stimulus', 'Operand1',
'Operand2', 'Operand3', 'ShownProduct', 'CorrectProduct', 'Condition',
'ResponseAccuracy', 'Anticipation', 'Certainty', 'expt',
'response_ticks', 'Gndr', 'Age', 'NativeLang', 'Bilingual', 'Major',
'GoodMath', 'Rhand', 'Lhand', 'Famhand', 'WhoLH', 'ARTCorrect',
'ARTFoils', 'MRTCorrect', 'MRTFoils', 'MathAnxiety1', 'MathAnxiety2',
'BEM_M', 'BEM_F', 'BEM_N', 'DS_F', 'DS_B', 'DS_H'],
dtype='object')
| data_group | crw_ticks | regexp | Stimulus | expt | response_ticks | Rhand | Lhand | |
|---|---|---|---|---|---|---|---|---|
| 0 | stm02 | 115262 | 1 (#2001) .* 12001 22001 2064 777 1040 | O1 | expt_1 | NaN | 24 | 6 |
| 1 | stm02 | 115417 | 2 (#2001) .* 12001 22001 2064 777 1040 | O2 | expt_1 | NaN | 24 | 6 |
| 2 | stm02 | 115571 | 3 (#2001) .* 12001 22001 2064 777 1040 | O3 | expt_1 | NaN | 24 | 6 |
| 3 | stm02 | 115725 | 4 (12001) (#22001) (2064) 777 1040 | P | expt_1 | 414.0 | 24 | 6 |
| 4 | stm02 | 21491 | 1 (#1003) .* 11003 21003 1040 777 1040 | O1 | expt_1 | NaN | 24 | 6 |
| 5 | stm02 | 21649 | 2 (#1003) .* 11003 21003 1040 777 1040 | O2 | expt_1 | NaN | 24 | 6 |
| 6 | stm02 | 21803 | 3 (#1003) .* 11003 21003 1040 777 1040 | O3 | expt_1 | NaN | 24 | 6 |
| 7 | stm02 | 21958 | 4 (11003) (#21003) (1040) 777 1040 | P | expt_1 | 899.0 | 24 | 6 |
Set epochs¶
Set epochs with all the new added columns. Same as the single subject example.
myh5.set_epochs('stmath', event_table, -100, 900) # tmin ms, tmax ms,
Sanitizing event table data types for mkh5 epochs table ...
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (8197, b'stm07', 291, b'stm07/dblock_6', 0, 15, 712207, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 15, 15, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'M', 20, b'English', b'Yes', b'Pharmalogical Chemistry', b'Yes', 30, 4, b'Yes', b'Cousin', 3, 0, 3, 0, 30, 42, 4.15, 4.65, 4.75, 10, 8, 7)
+ "skipping epoch {0}".format(e)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (12374, b'stm10', 291, b'stm10/dblock_6', 0, 10, 698890, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 10, 10, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'M', 19, b'Chinese', b'Yes', b'Economics', b'Yes', 22, 9, b'No', b'.NAN', 1, 0, 0, 0, 10, 10, 4.6, 4.5, 4.05, 10, 8, 6)
+ "skipping epoch {0}".format(e)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (23428, b'stm20', 291, b'stm20/dblock_5', 0, 4, 734468, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 4, 4, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'M', 20, b'Chinese', b'Yes', b'Math', b'Yes', 23, 3, b'No', b'.NAN', 2, 0, 0, 0, 16, 17, 3.95, 5.1, 5.05, 8, 5, 5)
+ "skipping epoch {0}".format(e)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (27605, b'stm23', 291, b'stm23/dblock_7', 0, 9, 965385, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 9, 9, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'M', 19, b'English', b'Yes', b'Economics', b'Yes', 23, 6, b'No', b'.NAN', 6, 0, 4, 0, 17, 13, 4.15, 4.45, 4.5, 14, 11, 9)
+ "skipping epoch {0}".format(e)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (31562, b'stm26', 291, b'stm26/dblock_8', 0, 5, 859397, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 5, 5, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'F', 23, b'Korean', b'Yes', b'Social Psychology', b'Yes', 38, 2, b'No', b'.NAN', 2, 0, 4, 2, 23, 26, 4.5, 5.4, 4.6, 11, 11, 7)
+ "skipping epoch {0}".format(e)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (37116, b'stm31', 291, b'stm31/dblock_5', 0, 15, 762383, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 15, 15, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'M', 21, b'English', b'Yes', b'Bioengineering, Biotechnology', b'Yes', 29, 4, b'No', b'.NAN', 5, 0, 15, 1, 12, 29, 3.8, 5.05, 5., 12, 10, 8)
+ "skipping epoch {0}".format(e)
/home/astoermann/.conda/envs/test_jupyter_books_11/lib/python3.6/site-packages/mkpy/mkh5.py:1514: UserWarning: data error: pre-stimulus interval is out of bounds left ... skipping epoch (38506, b'stm33', 291, b'stm33/dblock_8', 0, 22, 732694, 291, 291, 0, 0, 0, 1, 250., 1, 0, 1, b'291', b'291', 291, 22, 22, 0, True, 0, b'(#291)', b'.NAN', b'.NAN', 0, b'.NAN', nan, nan, nan, nan, nan, b'.NAN', b'.NAN', b'.NAN', b'.NAN', b'expt_1', nan, b'M', 20, b'English', b'No', b'Psychology', b'Yes', 26, 1, b'No', b'.NAN', 6, 0, 5, 0, 24, 32, 5.25, 4.84210526, 4.45, 9, 7, 6)
+ "skipping epoch {0}".format(e)
Note
The warnings above data error happens when the pause mark happened outside of the window you set to be epoched (i.e., -40ms to 40ms)
Export epochs¶
# set columns of interest for export to reduce file size, can also reorder thiem this way.
# some columns are required for other software in the lab, so be careful when dropping
COI = ['epoch_id','match_time','data_group', 'Item', 'List',
'ThreatCondition', 'Condition', 'Stimulus', 'ResponseAccuracy',
'Operand1', 'Operand2', 'Operand3', 'ShownProduct', 'CorrectProduct',
'Gndr', 'Age', 'NativeLang', 'Bilingual','Major', 'GoodMath',
'Rhand', 'Lhand', 'Famhand', 'WhoLH', 'ARTCorrect', 'ARTFoils',
'MRTCorrect', 'MRTFoils', 'MathAnxiety1', 'MathAnxiety2',
'BEM_M', 'BEM_F', 'BEM_N', 'DS_F', 'DS_B', 'DS_H',
'response_ticks', 'log_evcodes', 'log_flags',
'lle', 'lhz', 'MiPf', 'LLPf', 'RLPf', 'LMPf', 'RMPf', 'LDFr', 'RDFr',
'LLFr', 'RLFr', 'LMFr', 'RMFr', 'LMCe', 'RMCe', 'MiCe', 'MiPa', 'LDCe',
'RDCe', 'LDPa', 'RDPa', 'LMOc', 'RMOc', 'LLTe', 'RLTe', 'LLOc', 'RLOc',
'MiOc', 'A2', 'HEOG', 'rle', 'rhz']
# export epochs
myh5.export_epochs('stmath', epochtable, file_format='h5', columns=COI)
print('done stmath1')
done stmath1