Skip to content

Conversation

Copy link

Copilot AI commented Jan 3, 2026

LindiH5pyDataset with compound dtypes raised TypeError on numeric slicing (e.g., dataset[:], dataset[0]), breaking PyNWB DynamicTable HTML rendering.

Changes

  • Modified LindiH5pyDataset._get_item_for_zarr() to handle numeric indexing on compound dtypes
    • Converts zarr's object array representation (lists) to numpy structured arrays
    • Single element access returns numpy.void, multi-element returns structured array
    • Maintains existing field selection behavior (e.g., dataset['fieldname'])

Example

# Previously raised TypeError, now works
compound_dtype = np.dtype([('x', np.int32), ('y', np.float64)])
dataset = lindi_file.create_dataset('dset', data=[(1, 2.2), (3, 4.4)], dtype=compound_dtype)

dataset[:]      # [(1, 2.2) (3, 4.4)]  - now works
dataset[0]      # (1, 2.2)             - now works
dataset['x'][:] # [1 3]                - still works

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • lindi.neurosift.org
    • Triggering command: /usr/bin/python python -m pytest tests/test_lindi_h5py_file.py -v (dns block)
  • tempory.net
    • Triggering command: /usr/bin/python python -m pytest tests/test_lindi_h5py_file.py -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Error slicing LindiH5pyDataset with compound dtype</issue_title>
<issue_description>I get a TypeError: Compound dataset /processing/optotagging/optogenetic_stimulation/timeseries does not support selection with slice(None, None, None) when attempting to get the html representation of a DynamicTable with a compound dtype when using Lindi. Using the local NWB file or streaming with remfile causes no error.

I think an additional case may need to be added for indexing into a LindiH5pyDataset with compound dtype using None values.

Code to reproduce:

import pynwb
import lindi
import remfile
import h5py

# load from remote nwb file
file = remfile.File("https://dandiarchive.s3.amazonaws.com/blobs/d5f/747/d5f74779-e41e-43af-b32c-1310e996a623")
f = h5py.File(file, 'r')
nwbfile = pynwb.NWBHDF5IO(file=f, mode='r').read()

stim_table = nwbfile.processing["optotagging"]["optogenetic_stimulation"]
stim_table._repr_html_()  # this works

# load from lindi file (same file as above, though any ogen nwbfile in dandiset 000253 should reproduce)
f = lindi.LindiH5pyFile.from_lindi_file("https://lindi.neurosift.org/dandi/dandisets/000253/assets/b305ccec-fa84-445e-aea5-0126d6eff558/nwb.lindi.json")
nwbfile = pynwb.NWBHDF5IO(file=f, mode='r').read()

stim_table = nwbfile.processing["optotagging"]["optogenetic_stimulation"]
stim_table._repr_html_()  # this errors

Traceback:

  File "/Users/smprince/Documents/code/Hippie/scratch.py", line 16, in <module>
    stim_table._repr_html_()
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 1279, in _repr_html_
    html_repr += self.generate_html_repr()
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 1289, in generate_html_repr
    inside = f"{self[:min(nrows, len(self))].to_html()}"
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 1011, in __getitem__
    ret = self.get(key)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 1069, in get
    sel = self.__get_selection_as_dict(key, df, index, **kwargs)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 1107, in __get_selection_as_dict
    ret[name] = col.get(arg, df=df, index=index, **kwargs)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 211, in get
    data = self.target.get(slice(None),  **kwargs)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/pynwb/base.py", line 797, in get
    vals = super().get(key)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/common/table.py", line 61, in get
    return super().get(key)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/container.py", line 982, in get
    return self.data[args]
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/backends/hdf5/h5_utils.py", line 231, in __getitem__
    rows = copy(super().__getitem__(arg))
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/hdmf/query.py", line 20, in __getitem__
    return self.dataset[idx]
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/lindi/LindiH5pyFile/LindiH5pyDataset.py", line 180, in __getitem__
    return self._get_item_for_zarr(self._zarr_array, args)
  File "/Users/smprince/anaconda3/envs/hippie/lib/python3.10/site-packages/lindi/LindiH5pyFile/LindiH5pyDataset.py", line 217, in _get_item_for_zarr
    raise TypeError(
TypeError: Compound dataset /processing/optotagging/optogenetic_stimulation/timeseries does not support selection with slice(None, None, None)
```</issue_description>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits January 3, 2026 01:00
Co-authored-by: rly <310197+rly@users.noreply.github.com>
Co-authored-by: rly <310197+rly@users.noreply.github.com>
Co-authored-by: rly <310197+rly@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix error slicing LindiH5pyDataset with compound dtype Fix numeric slicing for compound dtype datasets Jan 3, 2026
Copilot AI requested a review from rly January 3, 2026 01:07
Copilot finished work on behalf of rly January 3, 2026 01:07
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.43%. Comparing base (d2296a4) to head (1b04ba3).

Files with missing lines Patch % Lines
lindi/LindiH5pyFile/LindiH5pyDataset.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #116   +/-   ##
=======================================
  Coverage   81.43%   81.43%           
=======================================
  Files          30       30           
  Lines        2790     2790           
=======================================
  Hits         2272     2272           
  Misses        518      518           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error slicing LindiH5pyDataset with compound dtype

3 participants