CSVFolderDataset¶
- class torcheeg.datasets.CSVFolderDataset(csv_path: str = './data.csv', read_fn: None | ~typing.Callable = <function default_read_fn>, online_transform: None | ~typing.Callable = None, offline_transform: None | ~typing.Callable = None, label_transform: None | ~typing.Callable = None, io_path: None | str = None, io_size: int = 1048576, io_mode: str = 'lmdb', num_worker: int = 0, verbose: bool = True, **kwargs)[source][source]¶
Read meta information from CSV file and read EEG data from folder according to the meta information. The CSV file should contain the following columns:
subject_id
(Optional): The subject id of the EEG data. Commonly used in training and testing dataset split.label
(Optional): The label of the EEG data. Commonly used in training and testing dataset split.file_path
(Required): The path to the EEG data file.
# data.csv # | subject_id | trial_id | label | file_path | # | ---------- | ------- | ----- | ------------------------- | # | sub1 | 0 | 0 | './data/label1/sub1.fif' | # | sub1 | 1 | 1 | './data/label2/sub1.fif' | # | sub1 | 2 | 2 | './data/label3/sub1.fif' | # | sub2 | 0 | 0 | './data/label1/sub2.fif' | # | sub2 | 1 | 1 | './data/label2/sub2.fif' | # | sub2 | 2 | 2 | './data/label3/sub2.fif' | from torcheeg.datasets import CSVFolderDataset from torcheeg import transforms def default_read_fn(file_path, **kwargs): # Load EEG file raw = mne.io.read_raw(file_path) # Convert raw to epochs epochs = mne.make_fixed_length_epochs(raw, duration=1) # Return EEG data return epochs dataset = CSVFolderDataset(csv_path='./data.csv', read_fn=default_read_fn, online_transform=transforms.ToTensor(), label_transform=transforms.Select('label'), num_worker=4)
- Parameters:
csv_path (str) – The path to the CSV file.
read_fn (Callable) – Method for reading files in a folder. By default, this class provides methods for reading files using
mne.io.read_raw
. At the same time, we allow users to pass in custom file reading methods. The first input parameter of whose is file_path, and other parameters are additional parameters passed in when the class is initialized. For example, you can passchunk_size=32
toFolderDataset
, thenchunk_size
will be received here.online_transform (Callable, optional) – The transformation of the EEG signals and baseline EEG signals. The input is a
np.ndarray
, and the ouput is used as the first and second value of each element in the dataset. (default:None
)offline_transform (Callable, optional) – The usage is the same as
online_transform
, but executed before generating IO intermediate results. (default:None
)label_transform (Callable, optional) – The transformation of the label. The input is an information dictionary, and the ouput is used as the third value of each element in the dataset. (default:
None
)io_path (str) – The path to generated unified data IO, cached as an intermediate result. If set to None, a random path will be generated. (default:
None
)io_size (int) – Maximum size database may grow to; used to size the memory mapping. If database grows larger than
map_size
, an exception will be raised and the user must close and reopen. (default:1048576
)io_mode (str) – Storage mode of EEG signal. When io_mode is set to
lmdb
, TorchEEG provides an efficient database (LMDB) for storing EEG signals. LMDB may not perform well on limited operating systems, where a file system based EEG signal storage is also provided. When io_mode is set topickle
, pickle-based persistence files are used. When io_mode is set tomemory
, memory are used. (default:lmdb
)num_worker (int) – Number of subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default:
0
)verbose (bool) – Whether to display logs during processing, such as progress bars, etc. (default:
True
)