Selection and copies

A selection is an ordered list of ordinals into an existing dataset. Order is preserved and duplicates are kept, so a selection can reorder a dataset, drop records from it, or repeat one:

import signal_dataset as sds

dataset = sds.open("captures.sds")

# Reversed, with observation-0002 repeated so it is seen twice per epoch.
chosen = [2, 2, 1, 0]
view = dataset.select(chosen)

assert [item.id for item in view.record_metadata] == [
    dataset.record_metadata[index].id for index in chosen
]

Publish a lightweight reference dataset pinned to the same immutable shard objects:

subset = view.publish(
    "subset.sds",
    dataset_id="subset",
    snapshot_id="selection-001",
)

Or rewrite independent data and metadata shards:

copy = view.materialize(
    "copy.sds",
    dataset_id="copy",
    snapshot_id="materialized-001",
)

Current annotation sets are projected through the same indices in both cases. Reference chains are flattened during publication.