ValueError: buffer source array is read-only during ILP solve with pandas 2.0

[allysonryan]

Environment

• ultrack 0.7.2
• Python 3.12
• pandas 2.x
• scikit-image 0.26.0
• Gurobi solver (CPU node)

Error

ValueError: buffer source array is read-only

Full traceback:

File ".../ultrack/core/solve/solver/mip_solver.py", line 181, in add_edges
    sources = self._forward_map[np.asarray(sources, dtype=int)]
              ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../skimage/util/_map_array.py", line 184, in __getitem__
    out = map_array(...)
  File ".../skimage/util/_map_array.py", line 72, in map_array
    _map_array(input_arr, out_view, input_vals, output_vals)
  File "skimage/util/_remap.pyx", line 10, in skimage.util._remap._map_array
  File "<stringsource>", line 352, in View.MemoryView.memoryview.__cinit__
ValueError: buffer source array is read-only

Root cause

In pandas 2.0, the numpy arrays backing DataFrame columns are marked WRITEABLE=False to prevent accidental in-place mutation. When np.asarray(df["source_id"], dtype=int) is called and the column dtype already matches, numpy returns a read-only view of the column's backing array rather than a copy. scikit-image's _remap.pyx Cython code requires a writable buffer to construct its typed memoryview, and raises this error.

The same pattern appears in 8 places across mip_solver.py:

sources = self._forward_map[np.asarray(sources, dtype=int)]   # lines ~181, 239, 300
targets = self._forward_map[np.asarray(targets, dtype=int)]   # lines ~182, 240, 301
indices = self._forward_map[np.asarray(indices, dtype=int)]   # lines ~264, 336

Impact

The .db files (nodes + edges) are written successfully before the crash since segment() and link() complete before solve() is called. Only the ILP solve step fails, so no track assignments are produced and the pipeline cannot continue.

Fix

Add .copy() to force a writable array before passing to ArrayMap.__getitem__:

sources = self._forward_map[np.asarray(sources, dtype=int).copy()]
targets = self._forward_map[np.asarray(targets, dtype=int).copy()]
indices = self._forward_map[np.asarray(indices, dtype=int).copy()]

[allysonryan]

Update: The fix described above (adding .copy() to the _forward_map[np.asarray(...)] calls) is necessary but not sufficient. The root issue is one level deeper.

ArrayMap stores its in_values argument as-is. When ArrayMap.__getitem__ is called, it passes self.in_values to map_array → _remap.pyx, which also requires a writable buffer. Since self.in_values was constructed from a pandas column and is read-only, this fails even after the _forward_map[...] call receives a writable sources array.

The actual root fix is in add_nodes at the point where ArrayMap is constructed (line ~127-129):

# Before (line 127): indices may be read-only if it came from a pandas column
indices = np.asarray(indices, dtype=int)
self._backward_map = np.array(indices, copy=True)
self._forward_map = ArrayMap(indices, np.arange(len(indices)))

# Fix: ensure indices is always a writable copy before being stored in ArrayMap
indices = np.array(indices, dtype=int)        # np.array always copies
self._backward_map = indices.copy()
self._forward_map = ArrayMap(indices, np.arange(len(indices)))

With this fix in place (plus the .copy() on the _forward_map[...] indexing calls as a belt-and-suspenders measure), tracking completes successfully end-to-end on a CPU node with pandas 2.x.

[JoOkuma]

@allysonryan, this has been fixed #260 , can you update your scikit-image?