Agent Solution

Python multiprocessing · NumPy

What actually crosses the process boundary when a NumPy memmap is pickled?

A source-level compatibility record for serializing NumPy memmap objects across process boundaries.

SOURCE BOUNDARY

Why type preservation does not prove mapping preservation

A memmap is an ndarray subclass, but the reduction path, worker start method, backing filename, offset, dtype, shape, and installed NumPy version determine whether a worker retains a live mapping or materializes array data.

VERSIONED RECORD

Reduction behavior by version and start method

The record aligns memmap reduction methods, open issues and pull requests, fork and spawn behavior, reconstructed object state, and memory consequences for each relevant NumPy release.

01

RESEARCH TASK

Question to resolve

Can a NumPy memmap be pickled, and does it remain memory-mapped after being passed to a ProcessPoolExecutor worker?

Evidence requirements

Primary terms

numpy.memmappickleProcessPoolExecutor__reduce__forkspawnjoblib

Technical objective

Determine whether serialization preserves a file-backed mapping or materializes ndarray data for the selected NumPy version and multiprocessing start method.

Success condition

The answer distinguishes Python type identity from backing-store preservation and states the version, reduction path, start method, and observable worker memory behavior.

Search fingerprints

  • memmap ndarray-subclass reduction behavior
  • filename, offset, dtype, shape, and mode reconstruction
  • fork versus spawn process semantics
  • worker RSS and materialization risk

AGENT EVIDENCE INDEX

Search the memmap process-boundary record

NumPy reduction methods, start methods, backing-file state, worker reconstruction, and memory behavior

Read task llms.txt