Cannot load previously-loadable Open Ephys Neuropixels 2 data (XML file issue)

[jonahpearl]

Hi all — I'm encountering an issue related to #407 and potential issues with older versions of Open Ephys. I'm re-analyzing some data from 2024, recorded on NP2 probes (specifically part no. NP2013). Unfortunately, for whatever reason, some of the recordings are mostly on one shank, but have a single channel activated on a different shank, and when this has occurred, I can no longer open the data in recent versions of Open Ephys — I get the error "Could not match electrode positions to catalogue probe 'NP2013'. The probe part number in settings.xml may be incorrect. See #407 for details.".

For example, here is the beginning of the channels sections of the XML file for one of the recordings that fails:

<CHANNELS CH143="0:0" CH48="0:1" CH49="0:1" CH50="0:1" CH51="0:1" CH52="0:1"....>  # the rest are all 0:1
<ELECTRODE_XPOS CH143="40" CH48="258" CH49="290" CH50="258" CH51="290" CH52="258"...> # the rest are all 258/290
<ELECTRODE_YPOS CH143="705" CH48="0" CH49="0" CH50="15" CH51="15" CH52="30" CH53="30"....>

For different recordings using the same probe on a different day, where there isn't a lone stranded channel, SI can load the recording perfectly fine. So I think this lone channel is causing the error message. What's confusing is that I think (40, 705) is a valid position on these NP2 probes. Plenty of other channels have xpos 40 (it's the second column of shank 1), and 705 is also a valid row y-value. So I think there is something deeper going on, that I'm not fully equipped to unpack.

Any advice for how to load these recordings in this updated SI version, or sanitize the XML files so they can be loaded? Thanks!

Current versions:
probeinterface 0.3.2
spikeinterface 0.104.8

Files that could help:
old_settings_error.xml
old_settings_ok.xml

Traceback

Traceback (most recent call last):
    si.read_openephys("/n/groups/datta/Jonah/20231003_vlPAG_npx/raw_data/J07501/20240823_J07501_tur
    si.read_openephys("/n/groups/datta/Jonah/20231003_vlPAG_npx/raw_data/J07501/20240823_J07501_turbulator_shank3/2024-08-23_16-33-32/Record Node 105")
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/groups/datta/Jonah/miniconda3/envs/spikeinterface_5/lib/python3.13/site-packages/spikeinterface/extractors/neoextractors/openephys.py", line 521, in read_openephys
    recording = OpenEphysBinaryRecordingExtractor(folder_path, **kwargs)
  File "/n/groups/datta/Jonah/miniconda3/envs/spikeinterface_5/lib/python3.13/site-packages/spikeinterface/extractors/neoextractors/openephys.py", line 325, in __init__
    probe = probeinterface.read_openephys(
        settings_file=settings_file, stream_name=oe_stream_name, raise_error=False
    )
  File "/n/groups/datta/Jonah/miniconda3/envs/spikeinterface_5/lib/python3.13/site-packages/probeinterface/neuropixels_tools.py", line 1565, in read_openephys
    probe = _slice_openephys_catalogue_probe(full_probe, probe_info)
  File "/n/groups/datta/Jonah/miniconda3/envs/spikeinterface_5/lib/python3.13/site-packages/probeinterface/neuropixels_tools.py", line 1449, in _slice_openephys_catalogue_probe
    raise ValueError(
    ...<3 lines>...
    )
ValueError: Could not match electrode positions to catalogue probe 'NP2013'. The probe part number in settings.xml may be incorrect. See https://github.com/SpikeInterface/probeinterface/issues/407 for details.

[alejoe91]

@jonahpearl I found the issue and it is quite tricky! In the code, we use the minimum x position from the first shank as an "offset", to center our positions to (0, 0). In your case, you have this single electrode from shank 0 at position 40, but the actual minimum position for NP2 is 8! So the offset is 40 instead of 8 and the electrode assignment becomes messed up!

How many datasets have this issue? The easiest fix is to manually change the XPOS = 40 to 8 ...

[jonahpearl]

Thanks, that's helpful. Only a handful have this issue, so I can easily fix by hand.