Note
Click here to download the full example code
IO Tutorial
This notebook is designed to demonstrate the pynapple IO. It is build around the specifications of the BIDS standard for sharing datasets. The key ideas are summarized as follow :
Navigating a structured dataset
The dataset in this example can be found here.
import numpy as np
import pynapple as nap
# mkdocs_gallery_thumbnail_path = '_static/treeview.png'
project_path = "../../your/path/to/MyProject"
project = nap.load_folder(project_path)
print(project)
Out:
The pynapple IO offers a convenient way of visualizing and navigating a folder based dataset. To visualize the whole hierarchy of Folders, you can call the view property or the expand function.
Out:
📂 MyProject
└── 📂 sub-A2929
└── 📂 ses-A2929-200711
├── 📂 derivatives
│ ├── spikes.npz | TsGroup
│ ├── sleep_ep.npz | IntervalSet
│ ├── position.npz | TsdFrame
│ └── wake_ep.npz | IntervalSet
├── 📂 pynapplenwb
│ ├── A2929-200711_nc | NWB file
│ └── A2929-200711 | NWB file
├── x_plus_1.npz | Tsd
├── x_plus_y.npz | Tsd
└── stimulus-fish.npz | IntervalSet
Here it shows all the subjects (in this case only A2929), all the sessions and all of the derivatives folders. It shows as well all the NPZ files that contains a pynapple object and the NWB files.
The object project behaves like a nested dictionnary. It is then easy to loop and navigate through a hierarchy of folders when doing analyses. In this case, we are gonna take only the session A2929-200711.
Out:
📂 ses-A2929-200711
├── 📂 derivatives
├── 📂 pynapplenwb
├── x_plus_1.npz | Tsd
├── x_plus_y.npz | Tsd
└── stimulus-fish.npz | IntervalSet
I can expand to see what the folders contains.
Out:
📂 ses-A2929-200711
├── 📂 derivatives
│ ├── spikes.npz | TsGroup
│ ├── sleep_ep.npz | IntervalSet
│ ├── position.npz | TsdFrame
│ └── wake_ep.npz | IntervalSet
├── 📂 pynapplenwb
│ ├── A2929-200711_nc | NWB file
│ └── A2929-200711 | NWB file
├── x_plus_1.npz | Tsd
├── x_plus_y.npz | Tsd
└── stimulus-fish.npz | IntervalSet
None
Loading files
By default, pynapple save objects as NPZ. It is a convenient way to save all the properties of an object such as the time support. The pynapple IO offers an easy way to load any NPZ files that matches the structures defined for a pynapple object.
spikes = session["derivatives"]["spikes"]
position = session["derivatives"]["position"]
wake_ep = session["derivatives"]["wake_ep"]
sleep_ep = session["derivatives"]["sleep_ep"]
Objects are only loaded when they are called.
Out:
Index rate location group
------- -------- ---------- -------
0 7.30333 adn 0
1 5.7325 adn 0
2 8.11917 adn 0
3 6.67833 adn 0
4 10.7708 adn 0
5 11.0042 adn 0
6 16.5158 adn 0
7 2.19667 ca1 1
8 2.01583 ca1 1
9 1.06833 ca1 1
10 3.91833 ca1 1
11 3.30833 ca1 1
12 1.09417 ca1 1
13 1.27917 ca1 1
14 1.32333 ca1 1
Metadata
A good practice for sharing datasets is to write as many metainformation as possible. Following BIDS specifications, any data files should be accompagned by a JSON sidecar file.
Out:
wake_ep.json
position.json
sleep_ep.json
spikes.npz
sleep_ep.npz
spikes.json
position.npz
wake_ep.npz
To read the metainformation associated with a file, you can use the functions doc, info or metadata :
Out:
╭─ ../../your/path/to/MyProject/sub-A2929/ses-A2929-200711/derivati─╮
│ time : 2023-07-11 12:40:10.338066 │
│ info : Neurons recorded simultaneously in ADN and CA1 │
╰───────────────────────────────────────────────────────────────────╯
╭─ ../../your/path/to/MyProject/sub-A2929/ses-A2929-200711/derivati─╮
│ time : 2023-07-11 12:40:10.364061 │
│ info : Position and head-direction of the mouse recorded with │
│ Optitrack │
╰───────────────────────────────────────────────────────────────────╯
Saving a pynapple object
In this case, we define a new Tsd and a new IntervalSet that we would like to save in the session folder.
tsd = position["x"] + 1
epoch = nap.IntervalSet(start=np.array([0, 3]), end=np.array([1, 6]))
session.save("x_plus_1", tsd, description="Random position")
session.save("stimulus-fish", epoch, description="Fish pictures to V1")
We can visualize the newly saved objects.
Out:
📂 ses-A2929-200711
├── 📂 derivatives
│ ├── spikes.npz | TsGroup
│ ├── sleep_ep.npz | IntervalSet
│ ├── position.npz | TsdFrame
│ └── wake_ep.npz | IntervalSet
├── 📂 pynapplenwb
│ ├── A2929-200711_nc | NWB file
│ └── A2929-200711 | NWB file
├── x_plus_1.npz | Tsd
├── x_plus_y.npz | Tsd
└── stimulus-fish.npz | IntervalSet
Out:
╭─ ../../your/path/to/MyProject/sub-A2929/ses-A2929-200711/stimulus─╮
│ time : 2024-04-17 10:39:30.885957 │
│ info : Fish pictures to V1 │
╰───────────────────────────────────────────────────────────────────╯
Out:
Time (s)
---------- --------
670.6407 0.957143
670.649 0.956137
670.65735 0.955147
670.66565 0.954213
670.674 0.953244
670.68235 0.95237
670.69065 0.951604
670.699 0.950934
670.70735 0.950391
670.71565 0.949931
670.724 0.949419
670.73235 0.949029
670.74065 0.948703
670.749 0.948362
670.75735 0.947959
670.76565 0.947551
670.774 0.947114
670.78235 0.946735
670.79065 0.946417
670.799 0.946003
670.80735 0.945554
670.81565 0.945142
670.824 0.944657
670.83235 0.944096
670.84065 0.943503
670.849 0.942849
670.85735 0.942124
670.86565 0.941393
670.874 0.940736
670.88235 0.940054
...
1199.7533 1.02534
1199.7616 1.02496
1199.76995 1.02446
1199.77825 1.02383
1199.7866 1.02317
1199.79495 1.0225
1199.80325 1.0218
1199.8116 1.02097
1199.81995 1.02003
1199.82825 1.01911
1199.8366 1.01826
1199.84495 1.01752
1199.85325 1.0168
1199.8616 1.01619
1199.86995 1.01577
1199.87825 1.01541
1199.8866 1.01507
1199.89495 1.01457
1199.90325 1.01428
1199.9116 1.01379
1199.91995 1.0134
1199.92825 1.0129
1199.9366 1.01242
1199.94495 1.01199
1199.95325 1.01161
1199.9616 1.01124
1199.96995 1.01097
1199.97825 1.01079
1199.9866 1.01066
1199.99495 1.01062
dtype: float64, shape: (63527,)
Total running time of the script: ( 0 minutes 6.528 seconds)
Download Python source code: tutorial_pynapple_io.py