PILArNetParquetDataset#

class PILArNetParquetDataset(repo_id: str | None = 'DeepLearnPhysics/PILArNet-M-mini', data_root: str | None = None, split='train', transform=None, revision: Literal['v1', 'v2', 'v3'] = 'v2', parquet_revision: str = 'refs/convert/parquet', config_name: str = 'default', loop=1, ignore_index=-1, energy_threshold=0.0, min_points=1024, max_len=-1, remove_low_energy_scatters=False, old_pid_mapping=False, test_mode=False, test_cfg=None, overlay_n_events=1, overlay_prob=1.0, overlay_allow_repeats=True)[source]#

Bases: PILArNetOverlayMixin, Dataset

Map-style PILArNet reader backed by parquet (Arrow-mmap).

Reads the same flat point/cluster/cluster_extra layout as PILArNetH5Dataset from parquet shards – either the HF auto-converted ref of a dataset repo (repo_id) or a local directory of parquet files (data_root) – and emits identical data_dict dictionaries via the shared decode_event(). Intended for fine-tuning, evaluation, probes, and anywhere reproducible random access matters; for large-scale pretraining prefer PILArNetParquetIterableDataset (sequential + shuffle buffer).

The backing table is memory-mapped, so a local split is not resident in RAM. Event overlay is supported with the same overlay_* kwargs as the H5 reader (both mix in PILArNetOverlayMixin), so a config can swap the two readers freely. test_mode is not supported here (use the H5 reader for the voxelized/augmented test path); passing it raises.

Parameters:

  • repo_id (str | None) – HF dataset repo whose auto-converted parquet ref is read when data_root is None. Defaults to "DeepLearnPhysics/PILArNet-M-mini".

  • data_root (str | None) – Local directory of parquet shards; takes precedence over repo_id when set (and over the default via the PILARNET_PARQUET_ROOT_<REV> env var). Defaults to None.

  • split (str) – Split name; "val" is accepted as an alias for "validation". Defaults to "train".

  • transform (list[dict]) – Transform configs (NOT a prebuilt Compose).

  • revision ({"v1","v2","v3"}) – Cluster/extra column layout. Defaults to "v2".

  • parquet_revision (str) – Git ref carrying the parquet export. Defaults to "refs/convert/parquet".

  • config_name (str) – Parquet builder config subdir. Defaults to "default".

  • min_points (int) – Minimum points per event to keep. Defaults to 1024.

See PILArNetH5Dataset for the remaining shared arguments and the emitted dictionary schema.

get_data(idx)[source]#
get_data_name(idx)[source]#