Skip to content

Note

Click here to download the full example code

Core Tutorial

This script will introduce the basics of handling time series data with pynapple.

Warning

This tutorial uses seaborn and matplotlib for displaying the figure.

You can install both with pip install matplotlib seaborn

import numpy as np
import matplotlib.pyplot as plt
import pynapple as nap
import pandas as pd
import seaborn as sns

custom_params = {"axes.spines.right": False, "axes.spines.top": False}
sns.set_theme(style="ticks", palette="colorblind", font_scale=1.5, rc=custom_params)

Time series object

Let's create a Tsd object with artificial data. In this example, every time point is 1 second apart.

tsd = nap.Tsd(t=np.arange(100), d=np.random.rand(100), time_units="s")

print(tsd)

Out:

Time (s)
----------  ---------
0.0         0.923735
1.0         0.362238
2.0         0.961695
3.0         0.559338
4.0         0.058393
5.0         0.854421
6.0         0.55737
7.0         0.560314
8.0         0.312277
9.0         0.664344
10.0        0.559134
11.0        0.653118
12.0        0.47512
13.0        0.915396
14.0        0.31075
15.0        0.315481
16.0        0.360433
17.0        0.225103
18.0        0.23432
19.0        0.0109871
...
80.0        0.787693
81.0        0.0897834
82.0        0.0592212
83.0        0.338823
84.0        0.785288
85.0        0.932136
86.0        0.117659
87.0        0.396095
88.0        0.532803
89.0        0.42059
90.0        0.875323
91.0        0.813546
92.0        0.0844543
93.0        0.33779
94.0        0.973544
95.0        0.848348
96.0        0.858
97.0        0.692393
98.0        0.883817
99.0        0.933647
dtype: float64, shape: (100,)

It is possible to toggle between seconds, milliseconds and microseconds. Note that when using as_units, the returned object is a simple pandas series.

print(tsd.as_units("ms"), "\n")
print(tsd.as_units("us"))

Out:

Time (ms)
0.0        0.923735
1000.0     0.362238
2000.0     0.961695
3000.0     0.559338
4000.0     0.058393
             ...   
95000.0    0.848348
96000.0    0.858000
97000.0    0.692393
98000.0    0.883817
99000.0    0.933647
Length: 100, dtype: float64 

Time (us)
0           0.923735
1000000     0.362238
2000000     0.961695
3000000     0.559338
4000000     0.058393
              ...   
95000000    0.848348
96000000    0.858000
97000000    0.692393
98000000    0.883817
99000000    0.933647
Length: 100, dtype: float64

Pynapple is able to handle data that only contains timestamps, such as an object containing only spike times. To do so, we construct a Ts object which holds only times. In this case, we generate 10 random spike times between 0 and 100 ms.

ts = nap.Ts(t=np.sort(np.random.uniform(0, 100, 10)), time_units="ms")

print(ts)

Out:

Time (s)
0.001594493
0.003159709
0.027978461
0.028130957
0.040479023
0.054769263
0.063307455
0.080485526
0.081644996
0.08769057
shape: 10

If the time series contains multiple columns, we use a TsdFrame.

tsdframe = nap.TsdFrame(
    t=np.arange(100), d=np.random.rand(100, 3), time_units="s", columns=["a", "b", "c"]
)

print(tsdframe)

Out:

Time (s)          a        b        c
----------  -------  -------  -------
0.0         0.8802   0.96314  0.34556
1.0         0.85204  0.14783  0.06106
2.0         0.02749  0.67876  0.36787
3.0         0.04291  0.17927  0.1483
4.0         0.75192  0.5612   0.68331
5.0         0.16817  0.71221  0.0024
6.0         0.95915  0.5725   0.63063
7.0         0.33376  0.77673  0.86595
8.0         0.45981  0.28827  0.02217
9.0         0.07863  0.08664  0.13006
10.0        0.55924  0.60771  0.68963
11.0        0.74923  0.74745  0.56324
12.0        0.48035  0.0018   0.54969
13.0        0.45998  0.54522  0.23229
14.0        0.38662  0.69581  0.8004
15.0        0.74095  0.74812  0.22184
16.0        0.87788  0.11597  0.94661
17.0        0.89642  0.34768  0.42328
18.0        0.98116  0.67483  0.63004
19.0        0.21241  0.39047  0.25312
...
80.0        0.89712  0.41397  0.15206
81.0        0.83342  0.34242  0.01464
82.0        0.92756  0.02424  0.87119
83.0        0.88498  0.28508  0.44408
84.0        0.09853  0.08873  0.14865
85.0        0.56467  0.51266  0.17045
86.0        0.01906  0.9601   0.70918
87.0        0.72724  0.97098  0.42407
88.0        0.45157  0.01844  0.73288
89.0        0.65024  0.51841  0.68307
90.0        0.55023  0.46167  0.10142
91.0        0.1553   0.5792   0.10372
92.0        0.45878  0.04607  0.17074
93.0        0.78169  0.14778  0.59297
94.0        0.8477   0.47356  0.18615
95.0        0.04392  0.93508  0.77993
96.0        0.97546  0.71689  0.97754
97.0        0.40468  0.18595  0.02624
98.0        0.96042  0.41206  0.58147
99.0        0.62321  0.40946  0.90923
dtype: float64, shape: (100, 3)

And if the number of dimension is even larger, we can use the TsdTensor (typically movies).

tsdframe = nap.TsdTensor(
    t=np.arange(100), d=np.random.rand(100, 3, 4)
)

print(tsdframe)

Out:

Time (s)
----------  -----------------------------
0.0         [[0.320655 ... 0.898785] ...]
1.0         [[0.151949 ... 0.553522] ...]
2.0         [[0.6869   ... 0.184668] ...]
3.0         [[0.080449 ... 0.439865] ...]
4.0         [[0.209389 ... 0.348037] ...]
5.0         [[0.718419 ... 0.186291] ...]
6.0         [[0.088709 ... 0.59294 ] ...]
7.0         [[0.614466 ... 0.034238] ...]
8.0         [[0.881002 ... 0.419886] ...]
9.0         [[0.518699 ... 0.25962 ] ...]
10.0        [[0.106632 ... 0.067743] ...]
11.0        [[0.830492 ... 0.592275] ...]
12.0        [[0.552371 ... 0.321746] ...]
13.0        [[0.975908 ... 0.645732] ...]
14.0        [[0.231521 ... 0.571355] ...]
15.0        [[0.246566 ... 0.114151] ...]
16.0        [[0.586287 ... 0.703835] ...]
17.0        [[0.070882 ... 0.600138] ...]
18.0        [[0.684387 ... 0.243488] ...]
19.0        [[0.739166 ... 0.282156] ...]
...
80.0        [[0.343015 ... 0.96702 ] ...]
81.0        [[0.482615 ... 0.005465] ...]
82.0        [[0.657904 ... 0.215169] ...]
83.0        [[0.05642  ... 0.799185] ...]
84.0        [[0.108273 ... 0.368276] ...]
85.0        [[0.750548 ... 0.434113] ...]
86.0        [[0.809891 ... 0.053567] ...]
87.0        [[0.429727 ... 0.882304] ...]
88.0        [[0.989593 ... 0.520176] ...]
89.0        [[0.7904   ... 0.070226] ...]
90.0        [[0.024745 ... 0.328622] ...]
91.0        [[0.492637 ... 0.603764] ...]
92.0        [[0.99485  ... 0.805501] ...]
93.0        [[0.055977 ... 0.524144] ...]
94.0        [[0.366236 ... 0.063774] ...]
95.0        [[0.566298 ... 0.556412] ...]
96.0        [[0.41651  ... 0.120707] ...]
97.0        [[0.940795 ... 0.038079] ...]
98.0        [[0.793171 ... 0.567788] ...]
99.0        [[0.809171 ... 0.041855] ...]
dtype: float64, shape: (100, 3, 4)

Interval Sets object

The IntervalSet object stores multiple epochs with a common time unit. It can then be used to restrict time series to this particular set of epochs.

epochs = nap.IntervalSet(start=[0, 10], end=[5, 15], time_units="s")

new_tsd = tsd.restrict(epochs)

print(epochs)
print("\n")
print(new_tsd)

Out:

            start    end
       0        0      5
       1       10     15
shape: (2, 2), time unit: sec.


Time (s)
----------  --------
0           0.923735
1           0.362238
2           0.961695
3           0.559338
4           0.058393
5           0.854421
10          0.559134
11          0.653118
12          0.47512
13          0.915396
14          0.31075
15          0.315481
dtype: float64, shape: (12,)

Multiple operations are available for IntervalSet. For example, IntervalSet can be merged. See the full documentation of the class here for a list of all the functions that can be used to manipulate IntervalSets.

epoch1 = nap.IntervalSet(start=0, end=10)  # no time units passed. Default is us.
epoch2 = nap.IntervalSet(start=[5, 30], end=[20, 45])

epoch = epoch1.union(epoch2)
print(epoch1, "\n")
print(epoch2, "\n")
print(epoch)

Out:

            start    end
       0        0     10
shape: (1, 2), time unit: sec. 

            start    end
       0        5     20
       1       30     45
shape: (2, 2), time unit: sec. 

            start    end
       0        0     20
       1       30     45
shape: (2, 2), time unit: sec.

TsGroup object

Multiple time series with different time stamps (.i.e. a group of neurons with different spike times from one session) can be grouped with the TsGroup object. The TsGroup behaves like a dictionary but it is also possible to slice with a list of indexes

my_ts = {
    0: nap.Ts(
        t=np.sort(np.random.uniform(0, 100, 1000)), time_units="s"
    ),  # here a simple dictionary
    1: nap.Ts(t=np.sort(np.random.uniform(0, 100, 2000)), time_units="s"),
    2: nap.Ts(t=np.sort(np.random.uniform(0, 100, 3000)), time_units="s"),
}

tsgroup = nap.TsGroup(my_ts)

print(tsgroup, "\n")

Out:

  Index     rate
-------  -------
      0  10.0016
      1  20.0032
      2  30.0047 

Dictionary like indexing returns directly the Ts object

print(tsgroup[0], "\n")  

Out:

Time (s)
0.07083408
0.076736343
0.170599843
0.295250833
0.328204474
0.55759181
0.612005175
0.706360134
0.821872523
0.958285326
0.998823568
1.048722347
1.074615895
1.247032017
1.247561569
1.34075831
1.359668174
1.394835397
1.467814069
1.52970403
...
97.915034046
97.94511462
98.030675333
98.135975338
98.138514674
98.23120439
98.241036005
98.305332053
98.477873168
99.084315381
99.192560652
99.20216068
99.257995846
99.259069944
99.284498842
99.32403227
99.442284178
99.633752425
99.814302614
99.881668021
shape: 1000 

List like indexing

print(tsgroup[[0, 2]])  

Out:

  Index     rate
-------  -------
      0  10.0016
      2  30.0047

Operations such as restrict can thus be directly applied to the TsGroup as well as other operations.

newtsgroup = tsgroup.restrict(epochs)

count = tsgroup.count(
    1, epochs, time_units="s"
)  # Here counting the elements within bins of 1 seconds

print(count)

Out:

Time (s)      0    1    2
----------  ---  ---  ---
0.5          11   12   30
1.5          13   15   23
2.5           5   23   30
3.5          12   21   19
4.5          10   13   28
10.5         12   28   29
11.5         11   29   28
12.5          6   20   29
13.5         13   22   25
14.5         14   24   36
dtype: int64, shape: (10, 3)

One advantage of grouping time series is that metainformation can be added directly on an element-wise basis. In this case, we add labels to each Ts object when instantiating the group and after. We can then use this label to split the group. See the TsGroup documentation for a complete methodology for splitting TsGroup objects.

First we create a pandas Series for the label.

label1 = pd.Series(index=list(my_ts.keys()), data=[0, 1, 0])

print(label1)

Out:

0    0
1    1
2    0
dtype: int64

We can pass label1 at the initialization step.

tsgroup = nap.TsGroup(my_ts, time_units="s", my_label1=label1)

print(tsgroup)

Out:

  Index     rate    my_label1
-------  -------  -----------
      0  10.0016            0
      1  20.0032            1
      2  30.0047            0

Notice how the label has been added as one column when printing tsgroup.

We can also add a label for each items in 2 different ways after initializing the TsGroup object. First with set_info :

tsgroup.set_info(my_label2=np.array(["a", "a", "b"])) 

print(tsgroup)

Out:

  Index     rate    my_label1  my_label2
-------  -------  -----------  -----------
      0  10.0016            0  a
      1  20.0032            1  a
      2  30.0047            0  b

Notice that you can pass directly a numpy array as long as it is the same size as the TsGroup.

We can also add new metadata by passing it as an item of the dictionary with a string key.

tsgroup["my_label3"] = np.random.randn(len(tsgroup))

print(tsgroup)

Out:

  Index     rate    my_label1  my_label2      my_label3
-------  -------  -----------  -----------  -----------
      0  10.0016            0  a               -0.83806
      1  20.0032            1  a               -1.08632
      2  30.0047            0  b               -0.15654

Metadata columns can be viewed as attributes of TsGroup.

tsgroup.my_label1

Out:

0    0
1    1
2    0
Name: my_label1, dtype: int64

or with the get_info method.

tsgroup.get_info("my_label3")

Out:

0   -0.838062
1   -1.086323
2   -0.156541
Name: my_label3, dtype: float64

Finally you can use the metadata to slice through the TsGroup object.

There are multiple methods for it. You can use the TsGroup getter functions :

  • getby_category(col_name) : categorized the metadata column and return a TsGroup for each category.

  • getby_threshold(col_name, value) : threshold the metadata column and return a single TsGroup.

  • getby_intervals(col_name, bins) : digitize the metadata column and return a TsGroup for each bin.

In this example we categorized tsgroup with my_label2.

dict_of_tsgroup = tsgroup.getby_category("my_label2")

print(dict_of_tsgroup["a"], "\n")
print(dict_of_tsgroup["b"])

Out:

  Index     rate    my_label1  my_label2      my_label3
-------  -------  -----------  -----------  -----------
      0  10.0016            0  a               -0.83806
      1  20.0032            1  a               -1.08632 

  Index     rate    my_label1  my_label2      my_label3
-------  -------  -----------  -----------  -----------
      2  30.0047            0  b               -0.15654

Notice that getby_threshold return directly a TsGroup.

tsgroup.getby_threshold("my_label1", 0.5)

Out:

  Index     rate    my_label1  my_label2      my_label3
-------  -------  -----------  -----------  -----------
      1  20.0032            1  a               -1.08632

Similar operations can be performed using directly the attributes of TsGroup. For example, the previous line is equivalent to :

tsgroup[tsgroup.my_label1>0.5]

Out:

  Index     rate    my_label1  my_label2      my_label3
-------  -------  -----------  -----------  -----------
      1  20.0032            1  a               -1.08632

You can also chain queries with attributes.

tsgroup[(tsgroup.my_label1==0) & (tsgroup.my_label2=="a")]

Out:

  Index     rate    my_label1  my_label2      my_label3
-------  -------  -----------  -----------  -----------
      0  10.0016            0  a               -0.83806

Time support

A key feature of how pynapple manipulates time series is an inherent time support object defined for Ts, Tsd, TsdFrame and TsGroup objects. The time support object is defined as an IntervalSet that provides the time serie with a context. For example, the restrict operation will automatically update the time support object for the new time series. Ideally, the time support object should be defined for all time series when instantiating them. If no time series is given, the time support is inferred from the start and end of the time series.

In this example, a TsGroup is instantiated with and without a time support. Notice how the frequency of each Ts element is changed when the time support is defined explicitly.

time_support = nap.IntervalSet(start=0, end=200, time_units="s")

my_ts = {
    0: nap.Ts(
        t=np.sort(np.random.uniform(0, 100, 10)), time_units="s"
    ),  # here a simple dictionary
    1: nap.Ts(t=np.sort(np.random.uniform(0, 100, 20)), time_units="s"),
    2: nap.Ts(t=np.sort(np.random.uniform(0, 100, 30)), time_units="s"),
}

tsgroup = nap.TsGroup(my_ts)

tsgroup_with_time_support = nap.TsGroup(my_ts, time_support=time_support)
print(tsgroup, "\n")

Out:

  Index     rate
-------  -------
      0  0.10659
      1  0.21318
      2  0.31977 
print(tsgroup_with_time_support, "\n")

Out:

  Index    rate
-------  ------
      0    0.05
      1    0.1
      2    0.15 

acceding the time support is an important feature of pynapple

print(tsgroup_with_time_support.time_support)  

Out:

            start    end
       0        0    200
shape: (1, 2), time unit: sec.

We can use value_from which as it indicates assign to every timestamps the closed value in time from another time series. Let's define the time series we want to assign values from.

tsd_sin = nap.Tsd(t=np.arange(0, 100, 1), d=np.sin(np.arange(0, 10, 0.1)))

tsgroup_sin = tsgroup.value_from(tsd_sin)

plt.figure(figsize=(12, 6))
plt.plot(tsgroup[0].fillna(0), "|", markersize=20, mew=3)
plt.plot(tsd_sin, linewidth=2)
plt.plot(tsgroup_sin[0], "o", markersize=20)
plt.title("ts.value_from(tsd)")
plt.xlabel("Time (s)")
plt.yticks([-1, 0, 1])
plt.show()

ts.value_from(tsd)

Total running time of the script: ( 0 minutes 2.286 seconds)

Download Python source code: tutorial_pynapple_core.py

Download Jupyter notebook: tutorial_pynapple_core.ipynb

Gallery generated by mkdocs-gallery