pynaviz.audiovideo#

class pynaviz.audiovideo.AudioHandler(audio_path, stream_index=0, time=None)[source]#

Bases: BaseAudioVideo

Handler for reading and decoding audio frames from a file.

This class uses PyAV to access audio frames from a given file. It allows querying time-aligned audio samples between two timepoints, and provides audio shape and total length information.

Parameters:

  • audio_path (str | pathlib.Path) – Path to the audio file.

  • stream_index (int) – Index of the audio stream to decode (default is 0).

  • time (Optional[NDArray]) – Optional 1D time axis to associate with the samples. Must match the number of sample points in the audio file.

Raises:

ValueError – If the provided time axis is not 1D or does not match the number of sample points in the audio file.

Examples

>>> from pynaviz.audiovideo import AudioHandler
>>> ah = AudioHandler("example.mp3")
>>> # Get audio samples between 1.5 and 2.5 seconds.
>>> audio_trace = ah.get(1.5, 2.5)
>>> # Shape: (n_samples, n_channels)
>>> audio_trace.shape
(44100, 2)
get(start, end)[source]#

Extract decoded frames from a video between two timestamps.

This method decodes and returns the raw video frames corresponding to the time interval [start, end] in seconds. Decoding begins from the nearest keyframe at or before start, and proceeds sequentially until the end timestamp is reached or exceeded. If the last decoded frame extends beyond end, trailing samples are truncated so that the returned array aligns with the requested time range.

Parameters:

  • start (float) – Start time of the segment to extract, in seconds.

  • end (float) – End time of the segment to extract, in seconds. Must be greater than start.

Returns:

A 2D NumPy array containing the decoded frames for the requested interval. The exact shape depends on the video format and frame size, with the first dimension corresponding to time (frame index or samples) and the remaining dimensions containing audio channels.

Return type:

numpy.typing.NDArray

Notes

  • The returned frames are decoded in sequence and concatenated before being transposed so that time is the first dimension.

  • If end falls between two frames, the last frame is partially trimmed to match the requested duration.

See also

av.AudioFrame <https://pyav.org/docs/stable/api/audio.html#module-av.audio.frame>

The PyAV frame object.

property index: numpy.typing.NDArray#

Time axis corresponding to the audio samples.

Returns:

Array of timestamps with shape (num_samples,).

property shape: Tuple[int, int]#

Shape of the audio data.

Returns:

Tuple (num_samples, num_channels) describing the audio shape.

property t: numpy.typing.NDArray#

Time axis corresponding to the audio samples.

Returns:

Array of timestamps with shape (num_samples,).

property tot_length: float#

Total duration of the audio in seconds.

Returns:

Total duration of the audio stream.

class pynaviz.audiovideo.PlotTsdTensor(data, index=None, parent=None)[source]#

Bases: PlotBaseVideoTensor

Visualization for 3-dimensional time series (nap.TsdTensor) displayed as movies.

Parameters:

data (TsdTensor)

class pynaviz.audiovideo.PlotVideo(video, t=None, stream_index=0, index=None, start_worker=True, parent=None)[source]#

Bases: PlotBaseVideoTensor

Visualization for rendering video files synchronized with a time series.

This class uses shared memory and multiprocessing to efficiently stream frames from disk to GPU via pygfx. It also supports real-time interaction and frame-based control.

Parameters:
animate()[source]#

Main render loop callback for pygfx.

Handles texture and text update, syncing, and redraw triggering.

close()[source]#

Cleanly close shared memory, worker, and background thread.

property data#

Read-only access to the video data handler.

class pynaviz.audiovideo.VideoHandler(video_path, stream_index=0, time=None, return_frame_array=True)[source]#

Bases: BaseAudioVideo

Random-access video reader with timestamp-aware seeking.

This class wraps a PyAV container to provide precise, timestamp-based random access to frames. It can return either av.VideoFrame objects or RGB float arrays in the shape (H, W, 3) normalized to [0, 1]. Internally, a background thread builds an index of presentation timestamps (PTS) for fast seeks, with a fallback to on-the-fly indexing when the total frame count is unknown.

Parameters:

  • video_path (str | pathlib.Path) – Path to the video file.

  • stream_index (int) – Index of the video stream to use, default is 0.

  • time (Optional[NDArray]) – Experimental timestamps for each frame (seconds). If None, a uniform grid is generated from the stream’s average rate.

  • return_frame_array (bool) – If True (default), return frames as np.ndarray (RGB, float32 in [0, 1]); otherwise return av.VideoFrame instances.

Examples

>>> from pynaviz.audiovideo import VideoHandler
>>> vh = VideoHandler("example.mp4")
>>> # Get the frame at 1.5 seconds.
>>> frame = vh.get(1.5)
>>> # Shape: (height, width, channels)
>>> frame.shape
(480, 640, 3)
>>> # Get frames from the second to the 10th, every other frame.
>>> frame_sequence = vh[1:10:2]
>>> # Shape: (n_samples, height, width, channels)
>>> frame_sequence.shape
(5, 480, 640, 3)
get(ts)[source]#

Return the frame at (or immediately preceding) a timestamp.

Parameters:

ts (float) – Target time in seconds.

Returns:

If return_frame_array is True, returns an array with shape (H, W, 3) (RGB, float32 in [0, 1]). Otherwise returns an av.VideoFrame.

Return type:

av.VideoFrame | NDArray

Notes

  • Seeks to the closest keyframe behind ts and decodes forward until the target is reached.

  • Uses an internal cache: if the requested frame index matches the previously decoded one, the cached frame is returned.

get_slice(start, end=None)[source]#
Parameters:
property index: numpy.typing.NDArray#

Time index in seconds corresponding to frames.

If time was provided at initialization, that array is returned. Otherwise, a uniformly spaced array derived from the stream rate is used and may be updated as indexing progresses.

property shape: Tuple[int, int, int]#
:

Shape of the video, (n_frames, width, height).

Notes

  • When the total frame count is unknown at initialization, the length may grow while the background indexer discovers frames. A warning is emitted until indexing is complete.

property t: numpy.typing.NDArray#

Time index in seconds corresponding to frames.

If time was provided at initialization, that array is returned. Otherwise, a uniformly spaced array derived from the stream rate is used and may be updated as indexing progresses.

Modules

audio_handling

base_audiovideo

Base class for audio and video handling.

skeleton_plot

video_handling

video_plot

Module for time-synchronized video plotting using pygfx and multiprocessing.

video_worker