Skip to content

Note

Click here to download the full example code

IO Tutorial

This notebook is designed to demonstrate the pynapple IO. It is build around the specifications of the BIDS standard for sharing datasets. The key ideas are summarized as follow :


The dataset in this example can be found here.

import numpy as np
import pynapple as nap

# mkdocs_gallery_thumbnail_path = '_static/treeview.png'

project_path = "../../your/path/to/MyProject"

project = nap.load_folder(project_path)

print(project)

Out:

📂 MyProject
└── 📂 sub-A2929

The pynapple IO offers a convenient way of visualizing and navigating a folder based dataset. To visualize the whole hierarchy of Folders, you can call the view property or the expand function.

project.view

Out:

📂 MyProject
└── 📂 sub-A2929
    └── 📂 A2929-200711
        ├── 📂 derivatives
           ├── spikes.npz      |        TsGroup
           ├── sleep_ep.npz    |        IntervalSet
           ├── position.npz    |        TsdFrame
           └── wake_ep.npz     |        IntervalSet
        ├── 📂 pynapplenwb
           └── A2929-200711    |        NWB file
        ├── x_plus_1.npz    |        Tsd
        └── stimulus-fish.npz       |        IntervalSet

Here it shows all the subjects (in this case only A2929), all the sessions and all of the derivatives folders. It shows as well all the NPZ files that contains a pynapple object and the NWB files.

The object project behaves like a nested dictionary. It is then easy to loop and navigate through a hierarchy of folders when doing analyses. In this case, we are gonna take only the session A2929-200711.

session = project["sub-A2929"]["A2929-200711"]

print(session)

Out:

📂 A2929-200711
├── 📂 derivatives
├── 📂 pynapplenwb
├── x_plus_1.npz    |        Tsd
└── stimulus-fish.npz       |        IntervalSet

I can expand to see what the folders contains.

print(session.expand())

Out:

📂 A2929-200711
├── 📂 derivatives
   ├── spikes.npz      |        TsGroup
   ├── sleep_ep.npz    |        IntervalSet
   ├── position.npz    |        TsdFrame
   └── wake_ep.npz     |        IntervalSet
├── 📂 pynapplenwb
   └── A2929-200711    |        NWB file
├── x_plus_1.npz    |        Tsd
└── stimulus-fish.npz       |        IntervalSet
None

Loading files

By default, pynapple save objects as NPZ. It is a convenient way to save all the properties of an object such as the time support. The pynapple IO offers an easy way to load any NPZ files that matches the structures defined for a pynapple object.

spikes = session["derivatives"]["spikes"]
position = session["derivatives"]["position"]
wake_ep = session["derivatives"]["wake_ep"]
sleep_ep = session["derivatives"]["sleep_ep"]

Objects are only loaded when they are called.

print(session["derivatives"]["spikes"])

Out:

  Index      rate    group  location
-------  --------  -------  ----------
      0   7.30333        0  adn
      1   5.7325         0  adn
      2   8.11917        0  adn
      3   6.67833        0  adn
      4  10.7708         0  adn
      5  11.0042         0  adn
      6  16.5158         0  adn
      7   2.19667        1  ca1
      8   2.01583        1  ca1
      9   1.06833        1  ca1
     10   3.91833        1  ca1
     11   3.30833        1  ca1
     12   1.09417        1  ca1
     13   1.27917        1  ca1
     14   1.32333        1  ca1

Metadata

A good practice for sharing datasets is to write as many metainformation as possible. Following BIDS specifications, any data files should be accompagned by a JSON sidecar file.

import os

for f in os.listdir(session["derivatives"].path):
    print(f)

Out:

wake_ep.json
spikes.npz
sleep_ep.npz
spikes.json
position.npz
wake_ep.npz

To read the metainformation associated with a file, you can use the functions doc, info or metadata :

session["derivatives"].doc("spikes")


session["derivatives"].doc("position")

Out:

╭─ ../../your/path/to/MyProject/sub-A2929/A2929-200711─╮
 time : 2024-04-29 14:38:34.962905                     info :                                               ╰──────────────────────────────────────────────────────╯
╭─ ../../your/path/to/MyProject/sub-A2929/A2929-200711─╮
 No metadata                                          ╰──────────────────────────────────────────────────────╯

Saving a pynapple object

In this case, we define a new Tsd and a new IntervalSet that we would like to save in the session folder.

tsd = position["x"] + 1
epoch = nap.IntervalSet(start=np.array([0, 3]), end=np.array([1, 6]))

session.save("x_plus_1", tsd, description="Random position")
session.save("stimulus-fish", epoch, description="Fish pictures to V1")

We can visualize the newly saved objects.

session.expand()

Out:

📂 A2929-200711
├── 📂 derivatives
   ├── spikes.npz      |        TsGroup
   ├── sleep_ep.npz    |        IntervalSet
   ├── position.npz    |        TsdFrame
   └── wake_ep.npz     |        IntervalSet
├── 📂 pynapplenwb
   └── A2929-200711    |        NWB file
├── x_plus_1.npz    |        Tsd
└── stimulus-fish.npz       |        IntervalSet
session.doc("stimulus-fish")

Out:

╭─ ../../your/path/to/MyProject/sub-A2929/A2929-200711─╮
 time : 2024-05-28 14:05:56.644983                     info : Fish pictures to V1                           ╰──────────────────────────────────────────────────────╯
session["x_plus_1"]

Out:

Time (s)
----------  --------
670.6407    0.957143
670.649     0.956137
670.65735   0.955147
670.66565   0.954213
670.674     0.953244
670.68235   0.95237
670.69065   0.951604
670.699     0.950934
670.70735   0.950391
670.71565   0.949931
670.724     0.949419
670.73235   0.949029
670.74065   0.948703
670.749     0.948362
670.75735   0.947959
670.76565   0.947551
670.774     0.947114
670.78235   0.946735
670.79065   0.946417
670.799     0.946003
...
1199.8366   1.01826
1199.84495  1.01752
1199.85325  1.0168
1199.8616   1.01619
1199.86995  1.01577
1199.87825  1.01541
1199.8866   1.01507
1199.89495  1.01457
1199.90325  1.01428
1199.9116   1.01379
1199.91995  1.0134
1199.92825  1.0129
1199.9366   1.01242
1199.94495  1.01199
1199.95325  1.01161
1199.9616   1.01124
1199.96995  1.01097
1199.97825  1.01079
1199.9866   1.01066
1199.99495  1.01062
dtype: float64, shape: (63527,)

Total running time of the script: ( 0 minutes 5.546 seconds)

Download Python source code: tutorial_pynapple_io.py

Download Jupyter notebook: tutorial_pynapple_io.ipynb

Gallery generated by mkdocs-gallery