Internal API

This page is for developers working on Hebog. These modules are not a supported public interface and change without notice. The architecture overview explains how the layers fit together, and How Hebog finds sources explains the science.

The sections follow the order in which hebog.find_sources runs the stages. Each stage module applies pure kernels from hebog.algorithms and hebog.science with tiles, halos and executor batches, and hebog.stages.composition runs the stages in order; its first section documents it. hebog.science holds the reviewed profile and configuration, the composition records and the catalogue-row kernels; it imports no stage.

The stage sequence

hebog.stages.composition

The stage sequence find_sources runs, and the runner of each stage.

Each runner plans its stage's tile grid, opens the generation the stage publishes and runs the stage through the caller's executor. run_stages estimates the background and RMS and then calls run_stages_from_background, which runs every later stage in order on a published background and RMS; a caller that already holds one starts there instead of estimating it again. The sequence returns the generations and the records the terminal catalogues are built from, and builds no catalogue: hebog.public_science does.

ADMITTED_TILE_CORE_PIXELS = 2048 module-attribute

Smallest tile core the scalability contract admits, in pixels.

BackgroundEstimate dataclass

One image's published background and RMS and what they record.

usable_noise is whether any pixel has a usable local noise estimate, and bright_regions how far the estimate refined around bright candidates.

Source code in src/hebog/stages/composition.py

@dataclass(frozen=True, slots=True)
class BackgroundEstimate:
    """One image's published background and RMS and what they record.

    ``usable_noise`` is whether any pixel has a usable local noise estimate,
    and ``bright_regions`` how far the estimate refined around bright
    candidates.
    """

    source: ZarrProductSink
    usable_noise: bool
    bright_regions: BrightRegionCounts

PublishedStageRecords dataclass

The records the stages published, from which the catalogues are built.

They are the inputs of the terminal composition, hebog.public_science.build_configured_continuum_products, under the same names, except that accepted_island_count is its component_count.

Source code in src/hebog/stages/composition.py

@dataclass(frozen=True, slots=True)
class PublishedStageRecords:
    """The records the stages published, from which the catalogues are built.

    They are the inputs of the terminal composition,
    ``hebog.public_science.build_configured_continuum_products``, under the
    same names, except that ``accepted_island_count`` is its
    ``component_count``.
    """

    accepted_island_count: int
    topology: TiledComponentTopology
    measurements: ComponentMeasurements
    association: SourceAssociationResult
    hierarchy: SourceAssociationResult
    component_rows: tuple[CatalogueSource, ...]
    source_rows: tuple[CatalogueSource, ...]
    source_positions: Mapping[int, SourcePositionDiagnostics]
    component_local_rms: Mapping[int, float]
    source_local_rms: Mapping[int, float]
    islands: tuple[CatalogueIsland, ...]
    island_ids_by_owner: Mapping[int, tuple[str, ...]]

PublishedStages dataclass

What the stages after the background and RMS published.

The generations are named for the planes a later caller reads from them: publication_source holds the support labels and the retained mask, component_source the component labels, fit_source the persistent measurement support, hierarchy_source the retained scale support, and the two source generations the source labels and the support each source owns.

Source code in src/hebog/stages/composition.py

@dataclass(frozen=True, slots=True)
class PublishedStages:
    """What the stages after the background and RMS published.

    The generations are named for the planes a later caller reads from
    them: ``publication_source`` holds the support labels and the retained
    mask, ``component_source`` the component labels, ``fit_source`` the
    persistent measurement support, ``hierarchy_source`` the retained
    scale support, and the two source generations the source labels and
    the support each source owns.
    """

    detection_source: ZarrProductSink
    support_source: ZarrProductSink
    publication_source: ZarrProductSink
    component_source: ZarrProductSink
    fit_source: ZarrProductSink
    hierarchy_source: ZarrProductSink
    source_label_source: ZarrProductSink
    source_support_source: ZarrProductSink
    multiscale: TiledMultiscaleDetection
    records: PublishedStageRecords
    wide_object_counts: WideObjectCounts

StageSequenceResult dataclass

The background and RMS generation and what the later stages published.

published is None when no pixel has a usable local noise estimate: the sequence then stops after the background and RMS, and rms_scientific_status is unavailable.

Source code in src/hebog/stages/composition.py

@dataclass(frozen=True, slots=True)
class StageSequenceResult:
    """The background and RMS generation and what the later stages published.

    ``published`` is ``None`` when no pixel has a usable local noise
    estimate: the sequence then stops after the background and RMS, and
    ``rms_scientific_status`` is ``unavailable``.
    """

    background_rms_source: ZarrProductSink
    rms_scientific_status: Literal["valid", "unavailable"]
    published: PublishedStages | None
    bright_regions: BrightRegionCounts

detect_multiscale_products(source, background_rms_source, executor, work_directory, *, image_shape_yx, beam, review, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Run the tiled detection pass and read its published planes.

The composition owns no plane at all: the pass publishes what a later pass reads by window, and reduces each scale feature's peak while its response is still on the task that evaluated it. Only the reconciled island records come back.

tile_core_pixels is the non-overlapping output core each task owns. It defaults to the smallest core the scalability contract admits, and is widened when the widest filter halo would exceed a quarter of it. Tile geometry changes which task computes a value, never the value. One tile per batch is already a coarse task at that core, so batching adds nothing here.

Source code in src/hebog/stages/composition.py

def detect_multiscale_products(  # noqa: PLR0913
    source: WindowReadable,
    background_rms_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    beam: BeamShapePixels,
    review: ContinuumScienceProfile,
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[ZarrProductSink, TiledMultiscaleDetection]:
    """Run the tiled detection pass and read its published planes.

    The composition owns no plane at all: the pass publishes what a later
    pass reads by window, and reduces each scale feature's peak while its
    response is still on the task that evaluated it. Only the reconciled
    island records come back.

    ``tile_core_pixels`` is the non-overlapping output core each task owns.
    It defaults to the smallest core the scalability contract admits, and is
    widened when the widest filter halo would exceed a quarter of it. Tile
    geometry changes which task computes a value, never the value. One tile
    per batch is already a coarse task at that core, so batching adds nothing
    here.
    """
    from hebog.algorithms.multiscale_tiles import (  # noqa: PLC0415
        scale_filter_halo_pixels,
    )
    from hebog.science.continuum import (  # noqa: PLC0415
        residual_detection_config,
    )
    from hebog.science.models import (  # noqa: PLC0415
        TiledMultiscaleDetection,
    )
    from hebog.stages.multiscale import (  # noqa: PLC0415
        MultiscaleStageConfig,
        run_multiscale_stage,
    )

    halo = scale_filter_halo_pixels(beam)
    core = max(tile_core_pixels, 4 * halo + 1)
    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(core, core),
        halo_yx=(halo, halo),
    )
    sink = ZarrProductSink(
        work_directory / "multiscale.zarr",
        manifest,
        generation_id=generation_id,
    )
    result = run_multiscale_stage(
        source,
        background_rms_source,
        manifest,
        config=MultiscaleStageConfig(
            beam=beam,
            detection=residual_detection_config(review),
            maximum_tiles_per_batch=1,
        ),
        executor=executor,
        sink=sink,
    )
    return sink, TiledMultiscaleDetection(
        detection_islands=result.detection_islands,
        scale_islands_by_order=result.scale_islands_by_order,
        scale_nominal_beam_fwhms=result.scale_nominal_beam_fwhms,
    )

estimate_background_rms(source, metadata, config, executor, work_directory, *, beam, review, generation_id)

Run the exact candidate-owned bounded background/RMS stage.

The stage publishes the background and the RMS as tiled planes, and the generation is what comes back: every later pass reads it by window. The one fact the composition needs about the estimate is whether any pixel can use it, which is reduced while streaming rather than kept as a mask. The diagnostics also record how far it refined around bright candidates.

Source code in src/hebog/stages/composition.py

def estimate_background_rms(  # noqa: PLR0913
    source: WindowReadable,
    metadata: ImageMetadata,
    config: SourceFinderConfig,
    executor: Executor,
    work_directory: Path,
    *,
    beam: BeamShapePixels,
    review: ContinuumScienceProfile,
    generation_id: str,
) -> BackgroundEstimate:
    """Run the exact candidate-owned bounded background/RMS stage.

    The stage publishes the background and the RMS as tiled planes, and the
    generation is what comes back: every later pass reads it by window. The
    one fact the composition needs about the estimate is whether any pixel
    can use it, which is reduced while streaming rather than kept as a mask.
    The diagnostics also record how far it refined around bright candidates.
    """
    from hebog.science.configuration import (  # noqa: PLC0415
        source_finder_configs,
    )
    from hebog.stages.background import (  # noqa: PLC0415
        MultiscaleSourceProtection,
    )

    manifest = plan_image_partitions(
        image_shape_yx=metadata.shape_yx,
        tile_core_shape_yx=_TILE_SHAPE_YX,
        halo_yx=(0, 0),
    )
    sink = ZarrProductSink(
        work_directory / "background.zarr",
        manifest,
        generation_id=generation_id,
    )
    candidate_detection = source_finder_configs()[0]
    detection_config = replace(
        candidate_detection,
        source_finder=config,
        background_rms=replace(
            (
                _public_background_config(
                    metadata.shape_yx,
                    candidate_detection.background_rms,
                    source_finder=config,
                )
                if config.profile == "continuum"
                else candidate_detection.background_rms
            ),
            background=config.background,
        ),
    )
    detection = run_detection_stage(
        source,
        manifest,
        detection_config,
        executor,
        sink,
        protect_coarse_source_support=(
            config.profile == "continuum"
            and detection_config.background_rms.coarse.window_shape_yx
            != candidate_detection.background_rms.coarse.window_shape_yx
        ),
        refine_local_noise=(
            config.profile == "continuum"
            and min(metadata.shape_yx)
            >= max(candidate_detection.background_rms.coarse.window_shape_yx)
        ),
        multiscale_protection=(
            MultiscaleSourceProtection(
                beam,
                config,
                review.matrix.support_fraction_bounds[0],
            )
            if config.profile == "continuum"
            else None
        ),
    )
    return BackgroundEstimate(
        source=sink,
        usable_noise=estimate_has_usable_noise(source, sink, metadata),
        bright_regions=BrightRegionCounts(
            candidate_count=len(detection.adaptive_candidate_positions_yx),
            largest_read_pixels=(
                detection.background_rms_grids.largest_bright_region_read_pixels
            ),
        ),
    )

estimate_has_usable_noise(source, background_rms_source, metadata)

Return whether any pixel has a usable local noise estimate.

This is the one decision the estimate makes about itself, and the only thing the composition needs from it: a pixel is usable where the image is valid and the estimate is positive. The stage has already required, on the core that computed it, that the estimate is finite wherever the image is valid, or, when no coarse window held enough samples for a background, nowhere, so validity is the image's own and needs no second opinion. The answer is reduced one canonical tile row at a time, so neither the estimate nor a mask over it is ever held whole.

Raises:

Type	Description
ValueError	If the published rows do not cover the image.

Source code in src/hebog/stages/composition.py

def estimate_has_usable_noise(
    source: WindowReadable,
    background_rms_source: ZarrProductSink,
    metadata: ImageMetadata,
) -> bool:
    """Return whether any pixel has a usable local noise estimate.

    This is the one decision the estimate makes about itself, and the only
    thing the composition needs from it: a pixel is usable where the image is
    valid and the estimate is positive. The stage has already required, on
    the core that computed it, that the estimate is finite wherever the image
    is valid, or, when no coarse window held enough samples for a
    background, nowhere, so validity is the image's own and needs no second
    opinion. The answer is reduced one canonical tile row at a time, so
    neither the estimate nor a mask over it is ever held whole.

    Raises:
        ValueError: If the published rows do not cover the image.
    """
    height, width = metadata.shape_yx
    rows = min(
        height,
        background_rms_source.manifest.tile_core_shape_yx[0],
    )
    usable = False
    start = 0
    for block in background_rms_source.iter_completed_row_blocks(
        "rms",
        max_block_bytes=rows * width * np.dtype(np.float64).itemsize,
    ):
        stop = start + block.shape[0]
        # Once a usable pixel is found, the image is read no further: the
        # remaining rows only confirm that the estimate covers the image.
        if not usable:
            window = source.read_window(ImageBounds(start, stop, 0, width))
            usable = bool(
                np.any(
                    window.valid_pixels
                    & (np.asarray(block, dtype=np.float64) > 0.0)
                )
            )
        start = stop
    if start != height:
        raise ValueError(
            f"background/RMS rows must total {height}; received {start}"
        )
    return usable

publish_component_fits(source, background_rms_source, detection_source, component_source, executor, work_directory, *, image_shape_yx, beam, wcs_header_text, restoring_beam, config, review, component_count, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Fit every measurement parent in its own context window.

Owners whose fit contexts touch need a joint model, so the contexts are reconciled first, a parent no joint fit can hold is split into its islands, and each fit parent is then measured inside the window holding it. The cores combine the persistent measurement support the parents contributed, and the connected features of that support are then grouped one window at a time.

Each parent also describes the direct components it owns, so the records the association decision reads come from the residual the fits already hold rather than from a second pass of per-component reads. component_count is the topology's, and the records must describe every one of those components.

Source code in src/hebog/stages/composition.py

def publish_component_fits(  # noqa: PLR0913, PLR0917
    source: WindowReadable,
    background_rms_source: ZarrProductSink,
    detection_source: ZarrProductSink,
    component_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    beam: BeamShapePixels,
    wcs_header_text: str,
    restoring_beam: RestoringBeam,
    config: SourceFinderConfig,
    review: ContinuumScienceProfile,
    component_count: int,
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[TiledComponentFits, ZarrProductSink]:
    """Fit every measurement parent in its own context window.

    Owners whose fit contexts touch need a joint model, so the contexts are
    reconciled first, a parent no joint fit can hold is split into its
    islands, and each fit parent is then measured inside the window holding
    it. The cores combine the persistent measurement support the
    parents contributed, and the connected features of that support are then
    grouped one window at a time.

    Each parent also describes the direct components it owns, so the records
    the association decision reads come from the residual the fits already
    hold rather than from a second pass of per-component reads.
    ``component_count`` is the topology's, and the records must describe
    every one of those components.
    """
    from hebog.algorithms.component_measurement import (  # noqa: PLC0415
        fit_parent_margin_pixels,
    )
    from hebog.algorithms.multiscale import (  # noqa: PLC0415
        build_residual_atrous_plan,
    )
    from hebog.science.configuration import (  # noqa: PLC0415
        source_finder_configs,
    )
    from hebog.science.continuum import (  # noqa: PLC0415
        compact_deblend_config,
    )
    from hebog.science.models import TiledComponentFits  # noqa: PLC0415
    from hebog.stages.component_fits import (  # noqa: PLC0415
        ComponentFitStageConfig,
        run_component_fit_stage,
    )
    from hebog.stages.extended_groups import (  # noqa: PLC0415
        ExtendedGroupStageConfig,
        run_extended_group_stage,
    )
    from hebog.stages.fit_parents import (  # noqa: PLC0415
        FitParentStageConfig,
        run_fit_parent_stage,
    )

    _, _, moment_config, fit_config = source_finder_configs()
    fit_config = replace(fit_config, integrated_flux_bias_correction_sigma=0.0)
    atrous_plan = build_residual_atrous_plan(beam, noise_correlation=beam)
    context_margin = int(fit_config.context_margin_pixels)
    core = max(tile_core_pixels, 4 * context_margin + 1)
    fit_parent_manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(core, core),
        halo_yx=(context_margin, context_margin),
    )
    fit_parent_sink = ZarrProductSink(
        work_directory / "fit-parents.zarr",
        fit_parent_manifest,
        generation_id=generation_id,
    )
    run_fit_parent_stage(
        component_source,
        fit_parent_manifest,
        config=FitParentStageConfig(
            context_margin_pixels=context_margin,
            maximum_tiles_per_batch=_support_tiles_per_batch(
                fit_parent_manifest, executor
            ),
            read_margin_pixels=fit_parent_margin_pixels(
                fit_config, atrous_plan
            ),
            maximum_bounds_pixels=(
                compact_deblend_config(config).maximum_compact_bounds_pixels
            ),
        ),
        executor=executor,
        sink=fit_parent_sink,
    )
    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    sink = ZarrProductSink(
        work_directory / "component-fits.zarr",
        manifest,
        generation_id=generation_id,
    )
    result = run_component_fit_stage(
        source,
        background_rms_source,
        detection_source,
        component_source,
        fit_parent_sink,
        manifest,
        config=ComponentFitStageConfig(
            moment=moment_config,
            fit=fit_config,
            atrous_plan=atrous_plan,
            detection_sigma=config.detection_threshold_sigma,
            island_sigma=config.island_threshold_sigma,
            minimum_pixels=config.minimum_island_pixels,
            maximum_bounds_pixels=(
                compact_deblend_config(config).maximum_compact_bounds_pixels
            ),
            minimum_support_fraction=(
                review.matrix.support_fraction_bounds[0]
            ),
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        component_count=component_count,
        wcs_header_text=wcs_header_text,
        beam=restoring_beam,
        executor=executor,
        sink=sink,
    )
    groups = run_extended_group_stage(
        source,
        background_rms_source,
        detection_source,
        component_source,
        sink,
        manifest,
        config=ExtendedGroupStageConfig(
            atrous_plan=atrous_plan,
            detection_sigma=config.detection_threshold_sigma,
            island_sigma=config.island_threshold_sigma,
            minimum_pixels=config.minimum_island_pixels,
            maximum_bounds_pixels=(
                compact_deblend_config(config).maximum_compact_bounds_pixels
            ),
            minimum_support_fraction=(
                review.matrix.support_fraction_bounds[0]
            ),
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        parents=result.parents,
        wcs_header_text=wcs_header_text,
        beam=restoring_beam,
        executor=executor,
    )
    return TiledComponentFits(
        parents=result.parents,
        features=groups.features,
        component_records=result.component_records,
        wide_parent_count=result.wide_parent_count,
    ), sink

publish_component_topology(support_source, detection_source, executor, work_directory, *, image_shape_yx, config, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Deblend every parent in its own window and read the components.

Deblending needs a parent's complete support and nothing beyond it, so the object pass decides one parent per task and the cores write the component labels they own.

Source code in src/hebog/stages/composition.py

def publish_component_topology(  # noqa: PLR0913
    support_source: ZarrProductSink,
    detection_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    config: SourceFinderConfig,
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[ZarrProductSink, TiledComponentTopology]:
    """Deblend every parent in its own window and read the components.

    Deblending needs a parent's complete support and nothing beyond it, so
    the object pass decides one parent per task and the cores write the
    component labels they own.
    """
    from hebog.science.continuum import (  # noqa: PLC0415
        compact_deblend_config,
    )
    from hebog.science.models import (  # noqa: PLC0415
        TiledComponentTopology,
    )
    from hebog.stages.component_topology import (  # noqa: PLC0415
        ComponentTopologyStageConfig,
        run_component_topology_stage,
    )

    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    sink = ZarrProductSink(
        work_directory / "components.zarr",
        manifest,
        generation_id=generation_id,
    )
    result = run_component_topology_stage(
        support_source,
        detection_source,
        manifest,
        config=ComponentTopologyStageConfig(
            deblend=compact_deblend_config(config),
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        executor=executor,
        sink=sink,
    )
    return sink, TiledComponentTopology(
        component_count=result.component_count,
        deblended_parent_count=result.deblended_parent_count,
        deferred_parent_count=result.deferred_parent_count,
    )

publish_detection_islands(source, background_rms_source, publication_source, component_source, executor, *, image_shape_yx, beam, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Measure one catalogue row per island of the published retained mask.

The cores label their own mask and observe which owners its islands hold, the fragments reconcile into islands named by canonical first pixel, and one task per batch of islands measures their rows. Nothing is published: the island labels exist only inside these rounds.

Source code in src/hebog/stages/composition.py

def publish_detection_islands(  # noqa: PLR0913
    source: WindowReadable,
    background_rms_source: ZarrProductSink,
    publication_source: ZarrProductSink,
    component_source: ZarrProductSink,
    executor: Executor,
    *,
    image_shape_yx: tuple[int, int],
    beam: BeamShapePixels,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[tuple[Any, ...], Mapping[int, tuple[str, ...]], int]:
    """Measure one catalogue row per island of the published retained mask.

    The cores label their own mask and observe which owners its islands hold,
    the fragments reconcile into islands named by canonical first pixel, and
    one task per batch of islands measures their rows. Nothing is published:
    the island labels exist only inside these rounds.
    """
    from math import log, pi  # noqa: PLC0415

    from hebog.stages.islands import (  # noqa: PLC0415
        DetectionIslandStageConfig,
        run_detection_island_stage,
    )

    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    result = run_detection_island_stage(
        source,
        background_rms_source,
        publication_source,
        component_source,
        manifest,
        config=DetectionIslandStageConfig(
            beam_area_pixels=(
                pi
                * beam.major_fwhm_pixels
                * beam.minor_fwhm_pixels
                / (4.0 * log(2.0))
            ),
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_objects_per_batch=_OWNER_OBJECTS_PER_BATCH,
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        executor=executor,
    )
    return result.islands, result.island_ids_by_owner, result.wide_island_count

publish_hierarchy_overlaps(detection_source, component_source, executor, work_directory, *, image_shape_yx, records, scale_detections, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Reduce every pixel fact the source hierarchy decision needs.

The cores observe which components, features and retained support components meet, one task per feature derives its reviewed B3 influence, and one task per candidate pair decides whether two envelopes overlap. The decision that consumes the result holds no plane, and the component records reach both it and this pass already built.

Source code in src/hebog/stages/composition.py

def publish_hierarchy_overlaps(  # noqa: PLR0913
    detection_source: ZarrProductSink,
    component_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    records: tuple[DetectionComponentRecord, ...],
    scale_detections: Sequence[ScaleDetections],
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[HierarchyOverlaps, ZarrProductSink]:
    """Reduce every pixel fact the source hierarchy decision needs.

    The cores observe which components, features and retained support
    components meet, one task per feature derives its reviewed B3 influence,
    and one task per candidate pair decides whether two envelopes overlap.
    The decision that consumes the result holds no plane, and the component
    records reach both it and this pass already built.
    """
    from hebog.stages.association import (  # noqa: PLC0415
        HierarchyOverlapStageConfig,
        run_hierarchy_overlap_stage,
    )

    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    sink = ZarrProductSink(
        work_directory / "hierarchy.zarr",
        manifest,
        generation_id=generation_id,
    )
    result = run_hierarchy_overlap_stage(
        detection_source,
        component_source,
        manifest,
        config=HierarchyOverlapStageConfig(
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_features_per_batch=_OWNER_OBJECTS_PER_BATCH,
            maximum_pairs_per_batch=_ENVELOPE_PAIRS_PER_BATCH,
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        records=records,
        scale_detections=scale_detections,
        executor=executor,
        sink=sink,
    )
    return result.overlaps, sink

publish_segment_rows(source, background_rms_source, detection_source, label_source, centroid_source, executor, work_directory, *, image_shape_yx, beam, wcs_header_text, restoring_beam, label_product_name, centroid_product_name, aperture_tie_policy, with_position_diagnostics, generation_id, sink_name, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Measure one catalogue row per segment, each in its own window.

The cores write the expanded apertures under the reviewed radius, they observe the bounds each segment and aperture occupies, and one task per batch of segments measures their rows, their local noise and their moment shapes. The aperture plane stays in the published generation: the rows carry what the catalogue needs from it, so the driver never holds it.

Source code in src/hebog/stages/composition.py

def publish_segment_rows(  # noqa: PLR0913, PLR0917
    source: WindowReadable,
    background_rms_source: ZarrProductSink,
    detection_source: ZarrProductSink,
    label_source: ZarrProductSink,
    centroid_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    beam: BeamShapePixels,
    wcs_header_text: str,
    restoring_beam: RestoringBeam,
    label_product_name: str,
    centroid_product_name: str,
    aperture_tie_policy: Literal["nearest-support", "canonical-source"],
    with_position_diagnostics: bool,
    generation_id: str,
    sink_name: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[tuple[Any, ...], Mapping[int, float], Mapping[int, Any], int]:
    """Measure one catalogue row per segment, each in its own window.

    The cores write the expanded apertures under the reviewed radius, they
    observe the bounds each segment and aperture occupies, and one task per
    batch of segments measures their rows, their local noise and their moment
    shapes. The aperture plane stays in the published generation: the rows
    carry what the catalogue needs from it, so the driver never holds it.
    """
    from math import ceil, log, pi  # noqa: PLC0415

    from hebog.science.continuum import (  # noqa: PLC0415
        CONTINUUM_MEASUREMENT_APERTURE_RADIUS_BEAMS,
    )
    from hebog.stages.catalogue_rows import (  # noqa: PLC0415
        SegmentRowStageConfig,
        run_segment_row_stage,
    )

    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    sink = ZarrProductSink(
        work_directory / f"{sink_name}.zarr",
        manifest,
        generation_id=generation_id,
    )
    result = run_segment_row_stage(
        source,
        background_rms_source,
        detection_source,
        label_source,
        centroid_source,
        detection_source,
        manifest,
        config=SegmentRowStageConfig(
            label_product_name=label_product_name,
            centroid_product_name=centroid_product_name,
            aperture_radius_pixels=ceil(
                CONTINUUM_MEASUREMENT_APERTURE_RADIUS_BEAMS
                * beam.major_fwhm_pixels
            ),
            aperture_tie_policy=aperture_tie_policy,
            beam_area_pixels=(
                2.0
                * pi
                / (8.0 * log(2.0))
                * beam.major_fwhm_pixels
                * beam.minor_fwhm_pixels
            ),
            denoised_position_maximum_peak_to_mean_ratio=(
                _DENOISED_POSITION_PEAK_TO_MEAN_RATIO
            ),
            with_position_diagnostics=with_position_diagnostics,
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_objects_per_batch=_OWNER_OBJECTS_PER_BATCH,
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        wcs_header_text=wcs_header_text,
        beam=restoring_beam,
        executor=executor,
        sink=sink,
    )
    return (
        result.rows,
        result.local_rms_by_label,
        result.position_diagnostics,
        result.wide_segment_count,
    )

publish_source_planes(component_source, detection_source, scale_support_source, measurement_support_source, executor, work_directory, *, image_shape_yx, association, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Publish the source labels and the persistent support they own.

The owners a source holds are a record map, so the cores write the labels from the shard that reaches them; the support a source owns spans tiles, so it is reconciled before each connected component divides its unseeded pixels between the sources that seed it.

Source code in src/hebog/stages/composition.py

def publish_source_planes(  # noqa: PLR0913, PLR0917
    component_source: ZarrProductSink,
    detection_source: ZarrProductSink,
    scale_support_source: ZarrProductSink,
    measurement_support_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    association: SourceAssociationResult,
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[ZarrProductSink, ZarrProductSink, int]:
    """Publish the source labels and the persistent support they own.

    The owners a source holds are a record map, so the cores write the
    labels from the shard that reaches them; the support a source owns spans
    tiles, so it is reconciled before each connected component divides its
    unseeded pixels between the sources that seed it.
    """
    from hebog.science.catalogues import (  # noqa: PLC0415
        source_label_by_owner,
    )
    from hebog.stages.sources import (  # noqa: PLC0415
        SourceLabelStageConfig,
        SourceSupportStageConfig,
        run_source_label_stage,
        run_source_support_stage,
    )

    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    label_sink = ZarrProductSink(
        work_directory / "source-labels.zarr",
        manifest,
        generation_id=generation_id,
    )
    run_source_label_stage(
        component_source,
        manifest,
        config=SourceLabelStageConfig(
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
        ),
        source_by_owner=source_label_by_owner(association),
        executor=executor,
        sink=label_sink,
    )
    support_sink = ZarrProductSink(
        work_directory / "source-support.zarr",
        manifest,
        generation_id=generation_id,
    )
    support_result = run_source_support_stage(
        label_sink,
        detection_source,
        scale_support_source,
        measurement_support_source,
        manifest,
        config=SourceSupportStageConfig(
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
            maximum_objects_per_batch=_OWNER_OBJECTS_PER_BATCH,
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        executor=executor,
        sink=support_sink,
    )
    return label_sink, support_sink, support_result.wide_component_count

publish_support_labels(detection_source, support_source, executor, work_directory, *, image_shape_yx, beam, detection_islands, config, review, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Decide owner connectivity and publish the support pass's labels.

Two of the decisions here are scoped to an owner rather than to a tile, so they are taken once per owner from the window holding it and applied by the core that owns each pixel. The caller's island admission is applied in the same write, and the count of islands it accepted is the only thing that comes back: every plane stays in the generation, where the mask product and the object rounds read it by window.

Source code in src/hebog/stages/composition.py

def publish_support_labels(  # noqa: PLR0913
    detection_source: ZarrProductSink,
    support_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    beam: BeamShapePixels,
    detection_islands: tuple[Any, ...],
    config: SourceFinderConfig,
    review: ContinuumScienceProfile,
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> tuple[int, int, ZarrProductSink]:
    """Decide owner connectivity and publish the support pass's labels.

    Two of the decisions here are scoped to an owner rather than to a tile,
    so they are taken once per owner from the window holding it and applied
    by the core that owns each pixel. The caller's island admission is
    applied in the same write, and the count of islands it accepted is the
    only thing that comes back: every plane stays in the generation, where
    the mask product and the object rounds read it by window.
    """
    from hebog.algorithms.extended_measurement import (  # noqa: PLC0415
        segment_refinement_halo_pixels,
    )
    from hebog.stages.publication import (  # noqa: PLC0415
        PublicationStageConfig,
        run_publication_stage,
    )

    halo = segment_refinement_halo_pixels(beam.major_fwhm_pixels)
    core = max(tile_core_pixels, 4 * halo + 1)
    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(core, core),
        halo_yx=(halo, halo),
    )
    sink = ZarrProductSink(
        work_directory / "publication.zarr",
        manifest,
        generation_id=generation_id,
    )
    result = run_publication_stage(
        detection_source,
        support_source,
        manifest,
        detection_islands=detection_islands,
        config=PublicationStageConfig(
            beam=beam,
            island_threshold_sigma=review.matrix.island_sigma,
            minimum_island_pixels=config.minimum_island_pixels,
            maximum_island_pixels=config.maximum_island_pixels,
            maximum_tiles_per_batch=1,
            maximum_batch_read_pixels=_OWNER_BATCH_READ_PIXELS,
        ),
        executor=executor,
        sink=sink,
    )
    return result.accepted_island_count, result.wide_owner_count, sink

reduce_support_topology(detection_source, executor, work_directory, *, image_shape_yx, scale_orders, generation_id, tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Reconcile adjacent-scale persistence and return its plane's store.

Persistence is not bounded by a halo: it is a record graph over the whole image. It is reconciled from compact per-core summaries and published as owned cores, and the generation is returned so the support rounds read it by window.

Source code in src/hebog/stages/composition.py

def reduce_support_topology(  # noqa: PLR0913
    detection_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    scale_orders: tuple[int, ...],
    generation_id: str,
    tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> ZarrProductSink:
    """Reconcile adjacent-scale persistence and return its plane's store.

    Persistence is not bounded by a halo: it is a record graph over the whole
    image. It is reconciled from compact per-core summaries and published as
    owned cores, and the generation is returned so the support rounds read it
    by window.
    """
    from hebog.stages.support import (  # noqa: PLC0415
        SupportTopologyStageConfig,
        run_support_topology_stage,
    )

    manifest = plan_image_partitions(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=(tile_core_pixels, tile_core_pixels),
        halo_yx=(0, 0),
    )
    sink = ZarrProductSink(
        work_directory / "support.zarr",
        manifest,
        generation_id=generation_id,
    )
    run_support_topology_stage(
        detection_source,
        manifest,
        config=SupportTopologyStageConfig(
            scale_orders=scale_orders,
            maximum_tiles_per_batch=_support_tiles_per_batch(
                manifest, executor
            ),
        ),
        executor=executor,
        sink=sink,
    )
    return sink

restoring_beam_from_header(header)

Return the restoring beam a header states, in degrees.

A header without BPA states a beam at position angle zero.

Source code in src/hebog/stages/composition.py

def restoring_beam_from_header(header: fits.Header) -> RestoringBeam:
    """Return the restoring beam a header states, in degrees.

    A header without ``BPA`` states a beam at position angle zero.
    """
    return RestoringBeam(
        cast(float, header["BMAJ"]),
        cast(float, header["BMIN"]),
        cast(float, header["BPA"]) if "BPA" in header else 0.0,
    )

run_stages(source, metadata, executor, work_directory, *, config, wcs_header_text, restoring_beam, beam, review, generation_id)

Estimate the background and RMS, then run every later stage in order.

beam is the restoring beam in local pixel axes and review the installed profile under the caller's thresholds. wcs_header_text is the header text each task rebuilds the celestial WCS from, and restoring_beam the beam that header states. Every stage writes its planes to a generation below work_directory named by generation_id, and later stages read them by window.

Source code in src/hebog/stages/composition.py

def run_stages(  # noqa: PLR0913
    source: WindowReadable,
    metadata: ImageMetadata,
    executor: Executor,
    work_directory: Path,
    *,
    config: SourceFinderConfig,
    wcs_header_text: str,
    restoring_beam: RestoringBeam,
    beam: BeamShapePixels,
    review: ContinuumScienceProfile,
    generation_id: str,
) -> StageSequenceResult:
    """Estimate the background and RMS, then run every later stage in order.

    ``beam`` is the restoring beam in local pixel axes and ``review`` the
    installed profile under the caller's thresholds. ``wcs_header_text`` is
    the header text each task rebuilds the celestial WCS from, and
    ``restoring_beam`` the beam that header states. Every stage writes its
    planes to a generation below ``work_directory`` named by
    ``generation_id``, and later stages read them by window.
    """
    estimate = estimate_background_rms(
        source,
        metadata,
        config,
        executor,
        work_directory,
        beam=beam,
        review=review,
        generation_id=generation_id,
    )
    if not estimate.usable_noise:
        return StageSequenceResult(
            background_rms_source=estimate.source,
            rms_scientific_status="unavailable",
            published=None,
            bright_regions=estimate.bright_regions,
        )
    return StageSequenceResult(
        background_rms_source=estimate.source,
        rms_scientific_status="valid",
        bright_regions=estimate.bright_regions,
        published=run_stages_from_background(
            source,
            estimate.source,
            executor,
            work_directory,
            image_shape_yx=metadata.shape_yx,
            config=config,
            wcs_header_text=wcs_header_text,
            restoring_beam=restoring_beam,
            beam=beam,
            review=review,
            generation_id=generation_id,
        ),
    )

run_stages_from_background(source, background_rms_source, executor, work_directory, *, image_shape_yx, config, wcs_header_text, restoring_beam, beam, review, generation_id, multiscale_tile_core_pixels=ADMITTED_TILE_CORE_PIXELS, support_tile_core_pixels=ADMITTED_TILE_CORE_PIXELS)

Run every stage after the background and RMS, in order.

background_rms_source is a published background and RMS generation, which every stage reads by window rather than estimating again. multiscale_tile_core_pixels is the multiscale pass's core and support_tile_core_pixels the core of every pass from the support reduction on; both default to the smallest core the scalability contract admits, as find_sources runs them, and no core changes a product.

Source code in src/hebog/stages/composition.py

def run_stages_from_background(  # noqa: PLR0913
    source: WindowReadable,
    background_rms_source: ZarrProductSink,
    executor: Executor,
    work_directory: Path,
    *,
    image_shape_yx: tuple[int, int],
    config: SourceFinderConfig,
    wcs_header_text: str,
    restoring_beam: RestoringBeam,
    beam: BeamShapePixels,
    review: ContinuumScienceProfile,
    generation_id: str,
    multiscale_tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
    support_tile_core_pixels: int = ADMITTED_TILE_CORE_PIXELS,
) -> PublishedStages:
    """Run every stage after the background and RMS, in order.

    ``background_rms_source`` is a published background and RMS generation,
    which every stage reads by window rather than estimating again.
    ``multiscale_tile_core_pixels`` is the multiscale pass's core and
    ``support_tile_core_pixels`` the core of every pass from the support
    reduction on; both default to the smallest
    core the scalability contract admits, as ``find_sources`` runs them, and
    no core changes a product.
    """
    detection_source, multiscale = detect_multiscale_products(
        source,
        background_rms_source,
        executor,
        work_directory,
        image_shape_yx=image_shape_yx,
        beam=beam,
        review=review,
        generation_id=generation_id,
        tile_core_pixels=multiscale_tile_core_pixels,
    )
    support_source = reduce_support_topology(
        detection_source,
        executor,
        work_directory,
        image_shape_yx=image_shape_yx,
        scale_orders=tuple(
            range(1, len(multiscale.scale_islands_by_order) + 1)
        ),
        generation_id=generation_id,
        tile_core_pixels=support_tile_core_pixels,
    )
    accepted_island_count, wide_owner_count, labels_source = (
        publish_support_labels(
            detection_source,
            support_source,
            executor,
            work_directory,
            image_shape_yx=image_shape_yx,
            beam=beam,
            detection_islands=multiscale.detection_islands,
            config=config,
            review=review,
            generation_id=generation_id,
            tile_core_pixels=support_tile_core_pixels,
        )
    )
    component_source, topology = publish_component_topology(
        labels_source,
        detection_source,
        executor,
        work_directory,
        image_shape_yx=image_shape_yx,
        config=config,
        generation_id=generation_id,
        tile_core_pixels=support_tile_core_pixels,
    )
    scale_detections = retained_scale_detections(multiscale)
    component_fits, fit_source = publish_component_fits(
        source,
        background_rms_source,
        detection_source,
        component_source,
        executor,
        work_directory,
        image_shape_yx=image_shape_yx,
        beam=beam,
        wcs_header_text=wcs_header_text,
        restoring_beam=restoring_beam,
        config=config,
        review=review,
        component_count=topology.component_count,
        generation_id=generation_id,
        tile_core_pixels=support_tile_core_pixels,
    )
    records = component_fits.component_records
    overlaps, hierarchy_source = publish_hierarchy_overlaps(
        detection_source,
        component_source,
        executor,
        work_directory,
        image_shape_yx=image_shape_yx,
        records=records,
        scale_detections=scale_detections,
        generation_id=generation_id,
        tile_core_pixels=support_tile_core_pixels,
    )
    measurements = reconcile_component_measurements(
        parents=component_fits.parents,
        features=component_fits.features,
    )
    hierarchy, association = _source_association(
        records,
        scale_detections,
        overlaps,
        (*measurements.compact_groups, *measurements.extended_groups),
    )
    source_label_source, source_support_source, wide_component_count = (
        publish_source_planes(
            component_source,
            detection_source,
            hierarchy_source,
            fit_source,
            executor,
            work_directory,
            image_shape_yx=image_shape_yx,
            association=association,
            generation_id=generation_id,
            tile_core_pixels=support_tile_core_pixels,
        )
    )
    islands, island_ids_by_owner, wide_island_count = (
        publish_detection_islands(
            source,
            background_rms_source,
            labels_source,
            component_source,
            executor,
            image_shape_yx=image_shape_yx,
            beam=beam,
            tile_core_pixels=support_tile_core_pixels,
        )
    )
    component_rows, component_local_rms, _, wide_component_segment_count = (
        publish_segment_rows(
            source,
            background_rms_source,
            detection_source,
            component_source,
            component_source,
            executor,
            work_directory,
            image_shape_yx=image_shape_yx,
            beam=beam,
            wcs_header_text=wcs_header_text,
            restoring_beam=restoring_beam,
            label_product_name="component-measurement-labels",
            centroid_product_name="component-measurement-labels",
            aperture_tie_policy="nearest-support",
            with_position_diagnostics=False,
            generation_id=generation_id,
            sink_name="component-rows",
            tile_core_pixels=support_tile_core_pixels,
        )
    )
    (
        source_rows,
        source_local_rms,
        source_positions,
        wide_source_segment_count,
    ) = publish_segment_rows(
        source,
        background_rms_source,
        detection_source,
        source_support_source,
        source_label_source,
        executor,
        work_directory,
        image_shape_yx=image_shape_yx,
        beam=beam,
        wcs_header_text=wcs_header_text,
        restoring_beam=restoring_beam,
        label_product_name="source-measurement-labels",
        centroid_product_name="source-labels",
        aperture_tie_policy="canonical-source",
        with_position_diagnostics=True,
        generation_id=generation_id,
        sink_name="source-rows",
        tile_core_pixels=support_tile_core_pixels,
    )
    return PublishedStages(
        detection_source=detection_source,
        support_source=support_source,
        publication_source=labels_source,
        component_source=component_source,
        fit_source=fit_source,
        hierarchy_source=hierarchy_source,
        source_label_source=source_label_source,
        source_support_source=source_support_source,
        multiscale=multiscale,
        records=PublishedStageRecords(
            accepted_island_count=accepted_island_count,
            topology=topology,
            measurements=measurements,
            association=association,
            hierarchy=hierarchy,
            component_rows=component_rows,
            source_rows=source_rows,
            source_positions=source_positions,
            component_local_rms=component_local_rms,
            source_local_rms=source_local_rms,
            islands=islands,
            island_ids_by_owner=island_ids_by_owner,
        ),
        wide_object_counts=WideObjectCounts(
            publication_owners=wide_owner_count,
            support_components=wide_component_count,
            deferred_fit_parents=component_fits.wide_parent_count,
            islands=wide_island_count,
            segments=wide_component_segment_count + wide_source_segment_count,
        ),
    )

Configuration and the reviewed profile

hebog.config

Immutable scientific and execution configuration.

AdaptiveRmsConfig dataclass

Fine-grid and deterministic blend policy around bright candidates.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class AdaptiveRmsConfig:
    """Fine-grid and deterministic blend policy around bright candidates."""

    grid: RmsGridConfig
    candidate_threshold_sigma: float
    influence_radius_pixels: float
    transition_width_pixels: float

    def __post_init__(self) -> None:
        """Require finite positive radii and a contained transition zone."""
        if (
            not isfinite(self.candidate_threshold_sigma)
            or self.candidate_threshold_sigma <= 0
        ):
            raise ValueError(
                "candidate_threshold_sigma must be finite and positive"
            )
        if (
            not isfinite(self.influence_radius_pixels)
            or self.influence_radius_pixels <= 0
        ):
            raise ValueError(
                "influence_radius_pixels must be finite and positive"
            )
        if (
            not isfinite(self.transition_width_pixels)
            or self.transition_width_pixels <= 0
            or self.transition_width_pixels > self.influence_radius_pixels
        ):
            raise ValueError(
                "transition_width_pixels must be finite, positive, and no "
                "larger than influence_radius_pixels"
            )

__post_init__()

Require finite positive radii and a contained transition zone.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Require finite positive radii and a contained transition zone."""
    if (
        not isfinite(self.candidate_threshold_sigma)
        or self.candidate_threshold_sigma <= 0
    ):
        raise ValueError(
            "candidate_threshold_sigma must be finite and positive"
        )
    if (
        not isfinite(self.influence_radius_pixels)
        or self.influence_radius_pixels <= 0
    ):
        raise ValueError(
            "influence_radius_pixels must be finite and positive"
        )
    if (
        not isfinite(self.transition_width_pixels)
        or self.transition_width_pixels <= 0
        or self.transition_width_pixels > self.influence_radius_pixels
    ):
        raise ValueError(
            "transition_width_pixels must be finite, positive, and no "
            "larger than influence_radius_pixels"
        )

BackgroundRmsConfig dataclass

Complete coarse, adaptive, interpolation, and memory policy.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class BackgroundRmsConfig:
    """Complete coarse, adaptive, interpolation, and memory policy."""

    coarse: RmsGridConfig
    adaptive: AdaptiveRmsConfig | None
    maximum_spatial_window_fraction: float
    maximum_constant_map_pixels: int
    background: Literal["estimated", "zero"] = "estimated"

    def __post_init__(self) -> None:
        """Validate automatic constant-map fallback and its memory bound."""
        if self.background not in {"estimated", "zero"}:
            raise ValueError("background must be 'estimated' or 'zero'")
        if (
            not isfinite(self.maximum_spatial_window_fraction)
            or not 0 < self.maximum_spatial_window_fraction <= 1
        ):
            raise ValueError(
                "maximum_spatial_window_fraction must be finite and in (0, 1]"
            )
        if (
            isinstance(self.maximum_constant_map_pixels, bool)
            or not isinstance(self.maximum_constant_map_pixels, Integral)
            or self.maximum_constant_map_pixels < 1
        ):
            raise ValueError(
                "maximum_constant_map_pixels must be a positive integer"
            )

__post_init__()

Validate automatic constant-map fallback and its memory bound.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Validate automatic constant-map fallback and its memory bound."""
    if self.background not in {"estimated", "zero"}:
        raise ValueError("background must be 'estimated' or 'zero'")
    if (
        not isfinite(self.maximum_spatial_window_fraction)
        or not 0 < self.maximum_spatial_window_fraction <= 1
    ):
        raise ValueError(
            "maximum_spatial_window_fraction must be finite and in (0, 1]"
        )
    if (
        isinstance(self.maximum_constant_map_pixels, bool)
        or not isinstance(self.maximum_constant_map_pixels, Integral)
        or self.maximum_constant_map_pixels < 1
    ):
        raise ValueError(
            "maximum_constant_map_pixels must be a positive integer"
        )

CompactDeblendConfig dataclass

Scientific cuts plus preferred and hard compact-work bounds.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class CompactDeblendConfig:
    """Scientific cuts plus preferred and hard compact-work bounds."""

    minimum_peak_signal_to_noise: float
    minimum_peak_separation_pixels: int
    minimum_saddle_depth_sigma: float
    minimum_region_pixels: int
    maximum_compact_island_pixels: int
    maximum_compact_bounds_pixels: int

    def __post_init__(self) -> None:
        """Require explicit finite science cuts and bounded region costs."""
        if (
            not isfinite(self.minimum_peak_signal_to_noise)
            or self.minimum_peak_signal_to_noise <= 0
        ):
            raise ValueError(
                "minimum_peak_signal_to_noise must be finite and positive"
            )
        if (
            isinstance(self.minimum_peak_separation_pixels, bool)
            or not isinstance(self.minimum_peak_separation_pixels, Integral)
            or self.minimum_peak_separation_pixels < 1
        ):
            raise ValueError(
                "minimum_peak_separation_pixels must be a positive integer"
            )
        if (
            not isfinite(self.minimum_saddle_depth_sigma)
            or self.minimum_saddle_depth_sigma < 0
        ):
            raise ValueError(
                "minimum_saddle_depth_sigma must be finite and non-negative"
            )
        if (
            isinstance(self.minimum_region_pixels, bool)
            or not isinstance(self.minimum_region_pixels, Integral)
            or self.minimum_region_pixels < 1
        ):
            raise ValueError(
                "minimum_region_pixels must be a positive integer"
            )
        for value, name in (
            (
                self.maximum_compact_island_pixels,
                "maximum_compact_island_pixels",
            ),
            (
                self.maximum_compact_bounds_pixels,
                "maximum_compact_bounds_pixels",
            ),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")

__post_init__()

Require explicit finite science cuts and bounded region costs.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Require explicit finite science cuts and bounded region costs."""
    if (
        not isfinite(self.minimum_peak_signal_to_noise)
        or self.minimum_peak_signal_to_noise <= 0
    ):
        raise ValueError(
            "minimum_peak_signal_to_noise must be finite and positive"
        )
    if (
        isinstance(self.minimum_peak_separation_pixels, bool)
        or not isinstance(self.minimum_peak_separation_pixels, Integral)
        or self.minimum_peak_separation_pixels < 1
    ):
        raise ValueError(
            "minimum_peak_separation_pixels must be a positive integer"
        )
    if (
        not isfinite(self.minimum_saddle_depth_sigma)
        or self.minimum_saddle_depth_sigma < 0
    ):
        raise ValueError(
            "minimum_saddle_depth_sigma must be finite and non-negative"
        )
    if (
        isinstance(self.minimum_region_pixels, bool)
        or not isinstance(self.minimum_region_pixels, Integral)
        or self.minimum_region_pixels < 1
    ):
        raise ValueError(
            "minimum_region_pixels must be a positive integer"
        )
    for value, name in (
        (
            self.maximum_compact_island_pixels,
            "maximum_compact_island_pixels",
        ),
        (
            self.maximum_compact_bounds_pixels,
            "maximum_compact_bounds_pixels",
        ),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")

CompactGaussianFitConfig dataclass

Explicit bounded nonlinear policy for one compact Gaussian fit.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class CompactGaussianFitConfig:
    """Explicit bounded nonlinear policy for one compact Gaussian fit."""

    minimum_fit_pixels: int
    maximum_function_evaluations: int
    minimum_sigma_pixels: float
    maximum_sigma_pixels: float
    maximum_amplitude_factor: float
    center_margin_pixels: float
    convergence_tolerance: float
    maximum_axis_ratio: float
    context_margin_pixels: int = 8
    extension_significance_sigma: float = 5.0
    component_extension_significance_sigma: float = 5.0
    integrated_flux_bias_correction_sigma: float = 0.0
    maximum_information_condition_number: float = 1e8
    pixel_support: Literal["bounded-context", "owned-region"] = (
        "bounded-context"
    )
    point_estimator: Literal["diagonal-weighted", "correlated-gls"] = (
        "diagonal-weighted"
    )
    maximum_gls_pixels: int = 512
    model_selection: Literal["free-only", "beam-or-free"] = "free-only"

    def __post_init__(self) -> None:
        """Validate scientific parameter bounds and finite work limits."""
        if (
            isinstance(self.minimum_fit_pixels, bool)
            or not isinstance(self.minimum_fit_pixels, Integral)
            or self.minimum_fit_pixels < _MINIMUM_GAUSSIAN_FIT_PIXELS
        ):
            raise ValueError("minimum_fit_pixels must be an integer >= 7")
        if (
            isinstance(self.maximum_function_evaluations, bool)
            or not isinstance(self.maximum_function_evaluations, Integral)
            or self.maximum_function_evaluations < 1
        ):
            raise ValueError(
                "maximum_function_evaluations must be a positive integer"
            )
        if (
            not isfinite(self.minimum_sigma_pixels)
            or not isfinite(self.maximum_sigma_pixels)
            or self.minimum_sigma_pixels <= 0
            or self.maximum_sigma_pixels <= self.minimum_sigma_pixels
        ):
            raise ValueError("sigma bounds must be finite, positive, ordered")
        if (
            not isfinite(self.maximum_amplitude_factor)
            or self.maximum_amplitude_factor <= 1
        ):
            raise ValueError("maximum_amplitude_factor must exceed one")
        if (
            not isfinite(self.center_margin_pixels)
            or self.center_margin_pixels < 0
        ):
            raise ValueError("center_margin_pixels must be finite and >= 0")
        if (
            not isfinite(self.convergence_tolerance)
            or not 0 < self.convergence_tolerance < 1
        ):
            raise ValueError(
                "convergence_tolerance must be finite and in (0, 1)"
            )
        if (
            not isfinite(self.maximum_axis_ratio)
            or self.maximum_axis_ratio <= 1
        ):
            raise ValueError("maximum_axis_ratio must be finite and > 1")
        if (
            isinstance(self.context_margin_pixels, bool)
            or not isinstance(self.context_margin_pixels, Integral)
            or self.context_margin_pixels < 0
        ):
            raise ValueError(
                "context_margin_pixels must be a non-negative integer"
            )
        self._validate_selection_policy()

    def _validate_selection_policy(self) -> None:
        """Validate extension evidence and identifiability thresholds."""
        if self.pixel_support not in {"bounded-context", "owned-region"}:
            raise ValueError("pixel_support is not a supported policy")
        if self.point_estimator not in {
            "diagonal-weighted",
            "correlated-gls",
        }:
            raise ValueError("point_estimator is not a supported policy")
        if self.model_selection not in {"free-only", "beam-or-free"}:
            raise ValueError("model_selection is not a supported policy")
        if (
            isinstance(self.maximum_gls_pixels, bool)
            or not isinstance(self.maximum_gls_pixels, Integral)
            or self.maximum_gls_pixels < _MINIMUM_GAUSSIAN_FIT_PIXELS
        ):
            raise ValueError("maximum_gls_pixels must be an integer >= 7")
        if (
            not isfinite(self.extension_significance_sigma)
            or self.extension_significance_sigma <= 0
        ):
            raise ValueError(
                "extension_significance_sigma must be finite and positive"
            )
        self._validate_component_selection_policy()
        if (
            not isfinite(self.integrated_flux_bias_correction_sigma)
            or not 0.0
            <= self.integrated_flux_bias_correction_sigma
            < _MAXIMUM_INTEGRATED_FLUX_BIAS_CORRECTION_SIGMA
        ):
            raise ValueError(
                "integrated_flux_bias_correction_sigma must be finite and "
                "in [0, 0.5)"
            )
        if (
            not isfinite(self.maximum_information_condition_number)
            or self.maximum_information_condition_number <= 1
        ):
            raise ValueError(
                "maximum_information_condition_number must be finite and "
                "greater than one"
            )

    def _validate_component_selection_policy(self) -> None:
        """Keep component evidence no stricter than source evidence."""
        if (
            not isfinite(self.component_extension_significance_sigma)
            or self.component_extension_significance_sigma <= 0
        ):
            raise ValueError(
                "component_extension_significance_sigma must be finite and "
                "positive"
            )
        if (
            self.component_extension_significance_sigma
            > self.extension_significance_sigma
        ):
            raise ValueError(
                "component_extension_significance_sigma cannot exceed "
                "extension_significance_sigma"
            )

__post_init__()

Validate scientific parameter bounds and finite work limits.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Validate scientific parameter bounds and finite work limits."""
    if (
        isinstance(self.minimum_fit_pixels, bool)
        or not isinstance(self.minimum_fit_pixels, Integral)
        or self.minimum_fit_pixels < _MINIMUM_GAUSSIAN_FIT_PIXELS
    ):
        raise ValueError("minimum_fit_pixels must be an integer >= 7")
    if (
        isinstance(self.maximum_function_evaluations, bool)
        or not isinstance(self.maximum_function_evaluations, Integral)
        or self.maximum_function_evaluations < 1
    ):
        raise ValueError(
            "maximum_function_evaluations must be a positive integer"
        )
    if (
        not isfinite(self.minimum_sigma_pixels)
        or not isfinite(self.maximum_sigma_pixels)
        or self.minimum_sigma_pixels <= 0
        or self.maximum_sigma_pixels <= self.minimum_sigma_pixels
    ):
        raise ValueError("sigma bounds must be finite, positive, ordered")
    if (
        not isfinite(self.maximum_amplitude_factor)
        or self.maximum_amplitude_factor <= 1
    ):
        raise ValueError("maximum_amplitude_factor must exceed one")
    if (
        not isfinite(self.center_margin_pixels)
        or self.center_margin_pixels < 0
    ):
        raise ValueError("center_margin_pixels must be finite and >= 0")
    if (
        not isfinite(self.convergence_tolerance)
        or not 0 < self.convergence_tolerance < 1
    ):
        raise ValueError(
            "convergence_tolerance must be finite and in (0, 1)"
        )
    if (
        not isfinite(self.maximum_axis_ratio)
        or self.maximum_axis_ratio <= 1
    ):
        raise ValueError("maximum_axis_ratio must be finite and > 1")
    if (
        isinstance(self.context_margin_pixels, bool)
        or not isinstance(self.context_margin_pixels, Integral)
        or self.context_margin_pixels < 0
    ):
        raise ValueError(
            "context_margin_pixels must be a non-negative integer"
        )
    self._validate_selection_policy()

CompactMomentConfig dataclass

Numerical availability policy for compact moment ellipses.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class CompactMomentConfig:
    """Numerical availability policy for compact moment ellipses."""

    minimum_shape_pixels: int
    covariance_relative_tolerance: float

    def __post_init__(self) -> None:
        """Require enough pixels for 2-D shape and a strict tolerance."""
        if (
            isinstance(self.minimum_shape_pixels, bool)
            or not isinstance(self.minimum_shape_pixels, Integral)
            or self.minimum_shape_pixels < _MINIMUM_SHAPE_PIXELS
        ):
            raise ValueError("minimum_shape_pixels must be an integer >= 3")
        if (
            not isfinite(self.covariance_relative_tolerance)
            or not 0 < self.covariance_relative_tolerance < 1
        ):
            raise ValueError(
                "covariance_relative_tolerance must be finite and in (0, 1)"
            )

__post_init__()

Require enough pixels for 2-D shape and a strict tolerance.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Require enough pixels for 2-D shape and a strict tolerance."""
    if (
        isinstance(self.minimum_shape_pixels, bool)
        or not isinstance(self.minimum_shape_pixels, Integral)
        or self.minimum_shape_pixels < _MINIMUM_SHAPE_PIXELS
    ):
        raise ValueError("minimum_shape_pixels must be an integer >= 3")
    if (
        not isfinite(self.covariance_relative_tolerance)
        or not 0 < self.covariance_relative_tolerance < 1
    ):
        raise ValueError(
            "covariance_relative_tolerance must be finite and in (0, 1)"
        )

DetectionStageConfig dataclass

Scientific policies required by the compact-detection stage.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class DetectionStageConfig:
    """Scientific policies required by the compact-detection stage."""

    background_rms: BackgroundRmsConfig
    source_finder: SourceFinderConfig

ResidualMultiscaleDetectionConfig dataclass

Promoted residual-B3 segmentation thresholds and topology rules.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class ResidualMultiscaleDetectionConfig:
    """Promoted residual-B3 segmentation thresholds and topology rules."""

    detection_threshold_sigma: float
    island_threshold_sigma: float
    minimum_scale_support_fraction: float
    minimum_island_area_beams: float
    connectivity: Literal["eight-neighbour"] = "eight-neighbour"
    persistence: Literal["adjacent-scales"] = "adjacent-scales"
    seed_growth: Literal["original-residual"] = "original-residual"
    subarea_island_policy: Literal["retain-direct-detection-seed"] = (
        "retain-direct-detection-seed"
    )
    edge_support: Literal["normalized-minimum-fraction"] = (
        "normalized-minimum-fraction"
    )
    invalid_pixels: Literal["excluded"] = "excluded"

    def __post_init__(self) -> None:
        """Reject thresholds or topology outside the promoted policy."""
        if (
            not isfinite(self.detection_threshold_sigma)
            or self.detection_threshold_sigma <= 0
            or not isfinite(self.island_threshold_sigma)
            or self.island_threshold_sigma <= 0
            or self.island_threshold_sigma >= self.detection_threshold_sigma
        ):
            raise ValueError(
                "multiscale thresholds must be finite, positive, and ordered"
            )
        if (
            not isfinite(self.minimum_scale_support_fraction)
            or not 0 < self.minimum_scale_support_fraction <= 1
        ):
            raise ValueError(
                "minimum scale support fraction must be finite and in (0, 1]"
            )
        if (
            not isfinite(self.minimum_island_area_beams)
            or self.minimum_island_area_beams <= 0
        ):
            raise ValueError(
                "minimum island area must be finite and positive in beams"
            )
        if self.connectivity != "eight-neighbour":
            raise ValueError("multiscale connectivity must be eight-neighbour")
        if self.persistence != "adjacent-scales":
            raise ValueError("multiscale persistence must use adjacent-scales")
        if self.seed_growth != "original-residual":
            raise ValueError(
                "multiscale seed growth must use original residual"
            )
        if self.subarea_island_policy != "retain-direct-detection-seed":
            raise ValueError(
                "subarea islands require a direct detection-threshold seed"
            )
        if self.edge_support != "normalized-minimum-fraction":
            raise ValueError(
                "edge support must use normalized minimum support"
            )
        if self.invalid_pixels != "excluded":
            raise ValueError("invalid pixels must be excluded")

__post_init__()

Reject thresholds or topology outside the promoted policy.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Reject thresholds or topology outside the promoted policy."""
    if (
        not isfinite(self.detection_threshold_sigma)
        or self.detection_threshold_sigma <= 0
        or not isfinite(self.island_threshold_sigma)
        or self.island_threshold_sigma <= 0
        or self.island_threshold_sigma >= self.detection_threshold_sigma
    ):
        raise ValueError(
            "multiscale thresholds must be finite, positive, and ordered"
        )
    if (
        not isfinite(self.minimum_scale_support_fraction)
        or not 0 < self.minimum_scale_support_fraction <= 1
    ):
        raise ValueError(
            "minimum scale support fraction must be finite and in (0, 1]"
        )
    if (
        not isfinite(self.minimum_island_area_beams)
        or self.minimum_island_area_beams <= 0
    ):
        raise ValueError(
            "minimum island area must be finite and positive in beams"
        )
    if self.connectivity != "eight-neighbour":
        raise ValueError("multiscale connectivity must be eight-neighbour")
    if self.persistence != "adjacent-scales":
        raise ValueError("multiscale persistence must use adjacent-scales")
    if self.seed_growth != "original-residual":
        raise ValueError(
            "multiscale seed growth must use original residual"
        )
    if self.subarea_island_policy != "retain-direct-detection-seed":
        raise ValueError(
            "subarea islands require a direct detection-threshold seed"
        )
    if self.edge_support != "normalized-minimum-fraction":
        raise ValueError(
            "edge support must use normalized minimum support"
        )
    if self.invalid_pixels != "excluded":
        raise ValueError("invalid pixels must be excluded")

RmsGridConfig dataclass

Window geometry and bounded-batch policy for one RMS grid.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class RmsGridConfig:
    """Window geometry and bounded-batch policy for one RMS grid."""

    window_shape_yx: tuple[int, int]
    step_yx: tuple[int, int]
    statistics: RmsWindowStatisticsConfig
    maximum_batch_cells: int

    def __post_init__(self) -> None:
        """Require meaningful windows and an explicit positive batch bound."""
        _validate_positive_shape(self.window_shape_yx, "window shape")
        _validate_positive_shape(self.step_yx, "step")
        if any(
            step > window
            for step, window in zip(
                self.step_yx,
                self.window_shape_yx,
                strict=True,
            )
        ):
            raise ValueError(
                "RMS grid step cannot exceed its window dimension"
            )
        if (
            isinstance(self.maximum_batch_cells, bool)
            or not isinstance(self.maximum_batch_cells, Integral)
            or self.maximum_batch_cells < 1
        ):
            raise ValueError("maximum_batch_cells must be a positive integer")

__post_init__()

Require meaningful windows and an explicit positive batch bound.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Require meaningful windows and an explicit positive batch bound."""
    _validate_positive_shape(self.window_shape_yx, "window shape")
    _validate_positive_shape(self.step_yx, "step")
    if any(
        step > window
        for step, window in zip(
            self.step_yx,
            self.window_shape_yx,
            strict=True,
        )
    ):
        raise ValueError(
            "RMS grid step cannot exceed its window dimension"
        )
    if (
        isinstance(self.maximum_batch_cells, bool)
        or not isinstance(self.maximum_batch_cells, Integral)
        or self.maximum_batch_cells < 1
    ):
        raise ValueError("maximum_batch_cells must be a positive integer")

RmsWindowStatisticsConfig dataclass

Robust-statistics policy for one batch of bounded RMS windows.

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class RmsWindowStatisticsConfig:
    """Robust-statistics policy for one batch of bounded RMS windows."""

    clipping_sigma: float
    maximum_iterations: int
    minimum_samples: int

    def __post_init__(self) -> None:
        """Require a finite clipping threshold and usable sample counts."""
        if not isfinite(self.clipping_sigma) or self.clipping_sigma <= 0:
            raise ValueError("clipping_sigma must be finite and positive")
        if isinstance(self.maximum_iterations, bool) or not isinstance(
            self.maximum_iterations,
            Integral,
        ):
            raise ValueError("maximum_iterations must be an integer")
        if self.maximum_iterations < 1:
            raise ValueError("maximum_iterations must be positive")
        if isinstance(self.minimum_samples, bool) or not isinstance(
            self.minimum_samples,
            Integral,
        ):
            raise ValueError("minimum_samples must be an integer")
        if self.minimum_samples < _MINIMUM_RMS_SAMPLES:
            raise ValueError("minimum_samples must be at least two")

__post_init__()

Require a finite clipping threshold and usable sample counts.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Require a finite clipping threshold and usable sample counts."""
    if not isfinite(self.clipping_sigma) or self.clipping_sigma <= 0:
        raise ValueError("clipping_sigma must be finite and positive")
    if isinstance(self.maximum_iterations, bool) or not isinstance(
        self.maximum_iterations,
        Integral,
    ):
        raise ValueError("maximum_iterations must be an integer")
    if self.maximum_iterations < 1:
        raise ValueError("maximum_iterations must be positive")
    if isinstance(self.minimum_samples, bool) or not isinstance(
        self.minimum_samples,
        Integral,
    ):
        raise ValueError("minimum_samples must be an integer")
    if self.minimum_samples < _MINIMUM_RMS_SAMPLES:
        raise ValueError("minimum_samples must be at least two")

SourceFinderConfig dataclass

Pipeline-neutral scientific thresholds for one image analysis.

Thresholds are explicit because a value appropriate for one survey, image product, or pipeline stage is not a universal scientific default. Island-size cuts are likewise explicit pixel counts; a compatibility adapter may derive them from reviewed beam metadata before constructing this scheduler-independent configuration. background is estimated, Hebog's spatial background, or zero, which subtracts nothing while the RMS is still estimated, as PyBDSF's mean_map="zero" does; which one a workflow uses is that workflow's adapter's choice, as are its filtering options. The residual multiscale segmentation policy has its own explicit configuration because its area and support rules use beam rather than pixel units. continuum is the general source-association profile; compact is an explicit component-level profile that is incomplete for extended emission.

A threshold or pixel count may be any real number or integer, a NumPy scalar included, and is held as the Python float or int it equals, so equal configurations serialize and hash alike.

SourceFinderConfig(5, 3, 7) SourceFinderConfig(detection_threshold_sigma=5.0, island_threshold_sigma=3.0, minimum_island_pixels=7, maximum_island_pixels=None, profile='continuum', background='estimated')

Source code in src/hebog/config.py

@dataclass(frozen=True, slots=True)
class SourceFinderConfig:
    """Pipeline-neutral scientific thresholds for one image analysis.

    Thresholds are explicit because a value appropriate for one survey,
    image product, or pipeline stage is not a universal scientific default.
    Island-size cuts are likewise explicit pixel counts; a compatibility
    adapter may derive them from reviewed beam metadata before constructing
    this scheduler-independent configuration.
    ``background`` is ``estimated``, Hebog's spatial background, or
    ``zero``, which subtracts nothing while the RMS is still estimated, as
    PyBDSF's ``mean_map="zero"`` does; which one a workflow uses is that
    workflow's adapter's choice, as are its filtering options. The residual
    multiscale segmentation policy has its own explicit configuration because
    its area and support rules use beam rather than pixel units.
    ``continuum`` is the general source-association profile; ``compact`` is an
    explicit component-level profile that is incomplete for extended emission.

    A threshold or pixel count may be any real number or integer, a NumPy
    scalar included, and is held as the Python ``float`` or ``int`` it
    equals, so equal configurations serialize and hash alike.

    >>> SourceFinderConfig(5, 3, 7)
    SourceFinderConfig(detection_threshold_sigma=5.0, \
island_threshold_sigma=3.0, minimum_island_pixels=7, \
maximum_island_pixels=None, profile='continuum', background='estimated')
    """

    detection_threshold_sigma: float
    island_threshold_sigma: float
    minimum_island_pixels: int
    maximum_island_pixels: int | None = None
    profile: Literal["continuum", "compact"] = "continuum"
    background: Literal["estimated", "zero"] = "estimated"

    def __post_init__(self) -> None:
        """Validate thresholds and pixel cuts, and hold plain values."""
        if self.profile not in {"continuum", "compact"}:
            raise ValueError(
                "source-finder profile must be 'continuum' or 'compact'"
            )
        if self.background not in {"estimated", "zero"}:
            raise ValueError(
                "source-finder background must be 'estimated' or 'zero'"
            )
        for name in ("detection_threshold_sigma", "island_threshold_sigma"):
            object.__setattr__(
                self, name, _sigma_threshold(name, getattr(self, name))
            )
        if self.island_threshold_sigma >= self.detection_threshold_sigma:
            raise ValueError(
                "island_threshold_sigma must be lower than "
                "detection_threshold_sigma"
            )
        if (
            isinstance(self.minimum_island_pixels, bool)
            or not isinstance(self.minimum_island_pixels, Integral)
            or self.minimum_island_pixels < 1
        ):
            raise ValueError(
                "minimum_island_pixels must be a positive integer"
            )
        maximum = self.maximum_island_pixels
        if maximum is not None and (
            isinstance(maximum, bool)
            or not isinstance(maximum, Integral)
            or maximum < self.minimum_island_pixels
        ):
            raise ValueError(
                "maximum_island_pixels must be an integer no smaller than "
                "minimum_island_pixels"
            )
        object.__setattr__(
            self, "minimum_island_pixels", int(self.minimum_island_pixels)
        )
        if maximum is not None:
            object.__setattr__(self, "maximum_island_pixels", int(maximum))

__post_init__()

Validate thresholds and pixel cuts, and hold plain values.

Source code in src/hebog/config.py

def __post_init__(self) -> None:
    """Validate thresholds and pixel cuts, and hold plain values."""
    if self.profile not in {"continuum", "compact"}:
        raise ValueError(
            "source-finder profile must be 'continuum' or 'compact'"
        )
    if self.background not in {"estimated", "zero"}:
        raise ValueError(
            "source-finder background must be 'estimated' or 'zero'"
        )
    for name in ("detection_threshold_sigma", "island_threshold_sigma"):
        object.__setattr__(
            self, name, _sigma_threshold(name, getattr(self, name))
        )
    if self.island_threshold_sigma >= self.detection_threshold_sigma:
        raise ValueError(
            "island_threshold_sigma must be lower than "
            "detection_threshold_sigma"
        )
    if (
        isinstance(self.minimum_island_pixels, bool)
        or not isinstance(self.minimum_island_pixels, Integral)
        or self.minimum_island_pixels < 1
    ):
        raise ValueError(
            "minimum_island_pixels must be a positive integer"
        )
    maximum = self.maximum_island_pixels
    if maximum is not None and (
        isinstance(maximum, bool)
        or not isinstance(maximum, Integral)
        or maximum < self.minimum_island_pixels
    ):
        raise ValueError(
            "maximum_island_pixels must be an integer no smaller than "
            "minimum_island_pixels"
        )
    object.__setattr__(
        self, "minimum_island_pixels", int(self.minimum_island_pixels)
    )
    if maximum is not None:
        object.__setattr__(self, "maximum_island_pixels", int(maximum))

hebog.science.configuration

Current scientific configuration for installed source finding.

source_finder_configs()

Return the current source-finder scientific configuration.

Source code in src/hebog/science/configuration.py

def source_finder_configs() -> tuple[
    DetectionStageConfig,
    CompactDeblendConfig,
    CompactMomentConfig,
    CompactGaussianFitConfig,
]:
    """Return the current source-finder scientific configuration."""
    statistics = RmsWindowStatisticsConfig(3.0, 10, 6)
    detection = DetectionStageConfig(
        background_rms=BackgroundRmsConfig(
            coarse=RmsGridConfig((150, 150), (50, 50), statistics, 32),
            adaptive=AdaptiveRmsConfig(
                grid=RmsGridConfig((35, 35), (7, 7), statistics, 32),
                candidate_threshold_sigma=75.0,
                influence_radius_pixels=75.0,
                transition_width_pixels=20.0,
            ),
            maximum_spatial_window_fraction=0.25,
            maximum_constant_map_pixels=1_000_000,
        ),
        source_finder=SourceFinderConfig(5.0, 3.0, 7),
    )
    return (
        detection,
        CompactDeblendConfig(
            5.0,
            2,
            # Judged at the true pass between two peaks, the highest level
            # at which one eight-connected part of the island holds both.
            # At 1.0 smooth extended emission in beam-correlated noise
            # splits on noise about twice as often as at 1.5 (`LOG.md`,
            # 3 October).
            1.5,
            7,
            100_000,
            250_000,
        ),
        CompactMomentConfig(3, 1e-12),
        CompactGaussianFitConfig(
            7,
            300,
            0.2,
            30.0,
            5.0,
            1.0,
            1e-8,
            30.0,
            component_extension_significance_sigma=1.5,
            integrated_flux_bias_correction_sigma=0.075,
            pixel_support="owned-region",
            # Beam-correlated GLS weighting amplifies pixel-independent
            # noise and lost most components on such images; diagonal
            # weighting is robust to either noise model.
            point_estimator="diagonal-weighted",
            model_selection="beam-or-free",
        ),
    )

hebog.science.profile

Minimal installed profile for the reviewed continuum science.

ContinuumCorrectionProfile dataclass

Reviewed continuum correction settings used at runtime.

Source code in src/hebog/science/profile.py

@dataclass(frozen=True, slots=True)
class ContinuumCorrectionProfile:
    """Reviewed continuum correction settings used at runtime."""

    minimum_island_area_beams: float

ContinuumDetectionProfile dataclass

Threshold, scale, and valid-support settings used at runtime.

Source code in src/hebog/science/profile.py

@dataclass(frozen=True, slots=True)
class ContinuumDetectionProfile:
    """Threshold, scale, and valid-support settings used at runtime."""

    scale_orders: tuple[int, ...]
    support_fraction_bounds: tuple[float, float]
    detection_sigma: float
    island_sigma: float

ContinuumScienceProfile dataclass

Small phase-neutral projection of the historical review record.

Source code in src/hebog/science/profile.py

@dataclass(frozen=True, slots=True)
class ContinuumScienceProfile:
    """Small phase-neutral projection of the historical review record."""

    matrix: ContinuumDetectionProfile
    corrections: ContinuumCorrectionProfile

configured_science_profile(profile, config)

Apply caller thresholds without mutating the installed profile.

Source code in src/hebog/science/profile.py

def configured_science_profile(
    profile: ContinuumScienceProfile,
    config: SourceFinderConfig,
) -> ContinuumScienceProfile:
    """Apply caller thresholds without mutating the installed profile."""
    return replace(
        profile,
        matrix=replace(
            profile.matrix,
            detection_sigma=config.detection_threshold_sigma,
            island_sigma=config.island_threshold_sigma,
        ),
    )

load_continuum_science_profile(payload)

Project the reviewed JSON record onto fields used by source finding.

Source code in src/hebog/science/profile.py

def load_continuum_science_profile(payload: bytes) -> ContinuumScienceProfile:
    """Project the reviewed JSON record onto fields used by source finding."""
    document = _object(json.loads(payload), name="document")
    matrix = _object(document.get("matrix"), name="matrix")
    corrections = _object(document.get("corrections"), name="corrections")
    scale_orders = tuple(int(value) for value in matrix["scale_orders"])
    support_bounds = tuple(
        float(value) for value in matrix["support_fraction_bounds"]
    )
    if scale_orders != (1, 2, 3):
        raise ValueError(
            "continuum science profile scales must be 1, 2, and 3"
        )
    if support_bounds != (0.5, 1.0):
        raise ValueError(
            "continuum science profile support fraction must span 0.5--1"
        )
    detection_sigma = float(matrix["detection_sigma"])
    island_sigma = float(matrix["island_sigma"])
    minimum_area = float(corrections["minimum_island_area_beams"])
    if (
        not isfinite(detection_sigma)
        or not isfinite(island_sigma)
        or detection_sigma <= island_sigma
        or island_sigma <= 0.0
    ):
        raise ValueError(
            "continuum science thresholds must be positive and ordered"
        )
    if not isfinite(minimum_area) or minimum_area <= 0.0:
        raise ValueError("continuum minimum island area must be positive")
    return ContinuumScienceProfile(
        matrix=ContinuumDetectionProfile(
            scale_orders=scale_orders,
            support_fraction_bounds=(support_bounds[0], support_bounds[1]),
            detection_sigma=detection_sigma,
            island_sigma=island_sigma,
        ),
        corrections=ContinuumCorrectionProfile(
            minimum_island_area_beams=minimum_area
        ),
    )

Partition planning

hebog.algorithms.partitioning

Deterministic image partition planning.

plan_image_partitions(*, image_shape_yx, tile_core_shape_yx, halo_yx, partition_origin_yx=(0, 0))

Plan canonical row-major cores and clipped stage read halos.

Source code in src/hebog/algorithms/partitioning.py

def plan_image_partitions(
    *,
    image_shape_yx: tuple[int, int],
    tile_core_shape_yx: tuple[int, int],
    halo_yx: tuple[int, int],
    partition_origin_yx: tuple[int, int] = (0, 0),
) -> PartitionManifest:
    """Plan canonical row-major cores and clipped stage read halos."""
    return PartitionManifest.create(
        image_shape_yx=image_shape_yx,
        tile_core_shape_yx=tile_core_shape_yx,
        halo_yx=halo_yx,
        partition_origin_yx=partition_origin_yx,
    )

Background and RMS

hebog.algorithms.background

Robust serial background and RMS statistics for bounded window batches.

BackgroundRmsTile dataclass

One bounded interpolated background and RMS output core.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class BackgroundRmsTile:
    """One bounded interpolated background and RMS output core."""

    bounds: ImageBounds
    background: npt.NDArray[np.float64]
    rms: npt.NDArray[np.float64]
    scientifically_available: bool
    fallback_cell_count: int

PreparedRmsGrid dataclass

Coarse samples filled once and ready for repeated interpolation.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class PreparedRmsGrid:
    """Coarse samples filled once and ready for repeated interpolation."""

    geometry: RmsGridGeometry
    background: npt.NDArray[np.float64]
    rms: npt.NDArray[np.float64]
    fallback_cells: npt.NDArray[np.bool_]
    scientifically_available: bool

    @property
    def fallback_cell_count(self) -> int:
        """Return the number of unavailable cells filled from neighbours."""
        return int(np.count_nonzero(self.fallback_cells))

fallback_cell_count property

Return the number of unavailable cells filled from neighbours.

RmsGridBatchStatistics dataclass

Small coarse statistics returned by one bounded window batch.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class RmsGridBatchStatistics:
    """Small coarse statistics returned by one bounded window batch."""

    batch: RmsWindowBatch
    statistics: RmsWindowStatistics
    protected_window_count: int = 0

RmsGridGeometry dataclass

Globally anchored coarse-grid geometry with edge-aligned windows.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class RmsGridGeometry:
    """Globally anchored coarse-grid geometry with edge-aligned windows."""

    image_shape_yx: tuple[int, int]
    configured_window_shape_yx: tuple[int, int]
    effective_window_shape_yx: tuple[int, int]
    step_yx: tuple[int, int]
    window_starts_y: tuple[int, ...]
    window_starts_x: tuple[int, ...]
    sample_coordinates_y: tuple[float, ...]
    sample_coordinates_x: tuple[float, ...]

    @property
    def shape_yx(self) -> tuple[int, int]:
        """Return the coarse-grid shape in NumPy axis order."""
        return (len(self.window_starts_y), len(self.window_starts_x))

    @property
    def cell_count(self) -> int:
        """Return the number of independently estimated window cells."""
        return self.shape_yx[0] * self.shape_yx[1]

cell_count property

Return the number of independently estimated window cells.

shape_yx property

Return the coarse-grid shape in NumPy axis order.

RmsGridStatistics dataclass

Canonical coarse background and RMS summaries for one image.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class RmsGridStatistics:
    """Canonical coarse background and RMS summaries for one image."""

    geometry: RmsGridGeometry
    background: npt.NDArray[np.float64]
    rms: npt.NDArray[np.float64]
    available: npt.NDArray[np.bool_]
    valid_sample_count: npt.NDArray[np.int64]
    retained_sample_count: npt.NDArray[np.int64]
    protected_window_count: int = 0

RmsWindowBatch dataclass

One rectangular block of coarse cells sharing a bounded image read.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class RmsWindowBatch:
    """One rectangular block of coarse cells sharing a bounded image read."""

    grid_y_start: int
    grid_y_stop: int
    grid_x_start: int
    grid_x_stop: int
    read_bounds: ImageBounds

    @property
    def shape_yx(self) -> tuple[int, int]:
        """Return the rectangular coarse-cell block shape."""
        return (
            self.grid_y_stop - self.grid_y_start,
            self.grid_x_stop - self.grid_x_start,
        )

    @property
    def cell_count(self) -> int:
        """Return the number of windows evaluated in this batch."""
        return self.shape_yx[0] * self.shape_yx[1]

cell_count property

Return the number of windows evaluated in this batch.

shape_yx property

Return the rectangular coarse-cell block shape.

RmsWindowBatchGroup dataclass

Neighbouring window batches estimated from one read of their union.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class RmsWindowBatchGroup:
    """Neighbouring window batches estimated from one read of their union."""

    batches: tuple[RmsWindowBatch, ...]
    read_bounds: ImageBounds

RmsWindowStatistics dataclass

Background and RMS estimates for a batch of independent windows.

Source code in src/hebog/algorithms/background.py

@dataclass(frozen=True, slots=True)
class RmsWindowStatistics:
    """Background and RMS estimates for a batch of independent windows."""

    background: npt.NDArray[np.float64]
    rms: npt.NDArray[np.float64]
    available: npt.NDArray[np.bool_]
    valid_sample_count: npt.NDArray[np.int64]
    retained_sample_count: npt.NDArray[np.int64]

assemble_rms_grid_statistics(grid, batch_results)

Assemble complete coarse summaries independently of completion order.

Source code in src/hebog/algorithms/background.py

def assemble_rms_grid_statistics(
    grid: RmsGridGeometry,
    batch_results: Any,
) -> RmsGridStatistics:
    """Assemble complete coarse summaries independently of completion order."""
    shape = grid.shape_yx
    background = np.full(shape, np.nan, dtype=np.float64)
    rms = np.full(shape, np.nan, dtype=np.float64)
    available = np.zeros(shape, dtype=np.bool_)
    valid_sample_count = np.zeros(shape, dtype=np.int64)
    retained_sample_count = np.zeros(shape, dtype=np.int64)
    visits = np.zeros(shape, dtype=np.uint8)
    protected_window_count = 0
    for result in batch_results:
        if not isinstance(result, RmsGridBatchStatistics):
            raise ValueError("coarse-grid results contain an invalid batch")
        batch = result.batch
        _require_canonical_batch(grid, batch)
        selection = (
            slice(batch.grid_y_start, batch.grid_y_stop),
            slice(batch.grid_x_start, batch.grid_x_stop),
        )
        expected_size = batch.cell_count
        if any(
            values.size != expected_size
            for values in (
                result.statistics.background,
                result.statistics.rms,
                result.statistics.available,
                result.statistics.valid_sample_count,
                result.statistics.retained_sample_count,
            )
        ):
            raise ValueError("coarse-grid batch statistics are misaligned")
        if np.any(visits[selection]):
            raise ValueError("duplicate coarse-grid cells were returned")
        background[selection] = result.statistics.background.reshape(
            batch.shape_yx
        )
        rms[selection] = result.statistics.rms.reshape(batch.shape_yx)
        available[selection] = result.statistics.available.reshape(
            batch.shape_yx
        )
        valid_sample_count[selection] = (
            result.statistics.valid_sample_count.reshape(batch.shape_yx)
        )
        retained_sample_count[selection] = (
            result.statistics.retained_sample_count.reshape(batch.shape_yx)
        )
        visits[selection] += 1
        protected_window_count += result.protected_window_count
    if np.any(visits == 0):
        raise ValueError("missing coarse-grid cells prevent interpolation")
    return RmsGridStatistics(
        geometry=grid,
        background=cast(
            npt.NDArray[np.float64],
            _read_only(background),
        ),
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
        available=cast(npt.NDArray[np.bool_], _read_only(available)),
        valid_sample_count=cast(
            npt.NDArray[np.int64],
            _read_only(valid_sample_count),
        ),
        retained_sample_count=cast(
            npt.NDArray[np.int64],
            _read_only(retained_sample_count),
        ),
        protected_window_count=protected_window_count,
    )

blend_adaptive_background_rms(coarse, adaptive, positions_yx, *, influence_radius_pixels, transition_width_pixels)

Blend cached fine estimates smoothly around bright candidates.

The background is blended both ways. The fine RMS only raises the coarse one: near a bright source it measures the raised noise of its artefacts, which a coarse window dilutes, and where it reads lower the coarse RMS stands. PyBDSF's adaptive small box likewise applies near a bright source only out to where it stops reading well above its large one. So a quieter region's noise, measured by a fine window that missed the protected sources there, never lowers the noise beside it.

Source code in src/hebog/algorithms/background.py

def blend_adaptive_background_rms(
    coarse: BackgroundRmsTile,
    adaptive: BackgroundRmsTile,
    positions_yx: tuple[tuple[float, float], ...],
    *,
    influence_radius_pixels: float,
    transition_width_pixels: float,
) -> BackgroundRmsTile:
    """Blend cached fine estimates smoothly around bright candidates.

    The background is blended both ways. The fine RMS only raises the
    coarse one: near a bright source it measures the raised noise of its
    artefacts, which a coarse window dilutes, and where it reads lower the
    coarse RMS stands. PyBDSF's adaptive small box likewise applies near a
    bright source only out to where it stops reading well above its large
    one. So a quieter region's noise, measured by a fine window that missed
    the protected sources there, never lowers the noise beside it.
    """
    if coarse.bounds != adaptive.bounds:
        raise ValueError("coarse and adaptive RMS tiles must share bounds")
    if not positions_yx or not adaptive.scientifically_available:
        return coarse
    bounds = coarse.bounds
    output_y = np.arange(bounds.y_start, bounds.y_stop, dtype=np.float64)
    output_x = np.arange(bounds.x_start, bounds.x_stop, dtype=np.float64)
    y_coordinates, x_coordinates = np.meshgrid(
        output_y,
        output_x,
        indexing="ij",
    )
    minimum_distance = np.full(bounds.shape_yx, np.inf, dtype=np.float64)
    for y_position, x_position in positions_yx:
        np.minimum(
            minimum_distance,
            np.hypot(
                y_coordinates - y_position,
                x_coordinates - x_position,
            ),
            out=minimum_distance,
        )
    transition = np.clip(
        (influence_radius_pixels - minimum_distance) / transition_width_pixels,
        0.0,
        1.0,
    )
    weight = transition * transition * (3.0 - 2.0 * transition)
    usable_fine = np.isfinite(adaptive.background) & np.isfinite(adaptive.rms)
    weight[~usable_fine] = 0.0
    use_adaptive = weight > 0.0
    background = np.array(coarse.background, copy=True)
    rms = np.array(coarse.rms, copy=True)
    background[use_adaptive] = (
        1.0 - weight[use_adaptive]
    ) * coarse.background[use_adaptive] + weight[
        use_adaptive
    ] * adaptive.background[use_adaptive]
    fine_weight = weight[use_adaptive]
    coarse_rms = coarse.rms[use_adaptive]
    fine_rms = adaptive.rms[use_adaptive]
    rms[use_adaptive] = np.where(
        fine_rms > coarse_rms,
        # The floor only absorbs the blend's rounding.
        np.maximum(
            (1.0 - fine_weight) * coarse_rms + fine_weight * fine_rms,
            coarse_rms,
        ),
        coarse_rms,
    )
    return BackgroundRmsTile(
        bounds=bounds,
        background=cast(
            npt.NDArray[np.float64],
            _read_only(background),
        ),
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
        scientifically_available=(
            coarse.scientifically_available
            or adaptive.scientifically_available
        ),
        fallback_cell_count=(
            coarse.fallback_cell_count + adaptive.fallback_cell_count
        ),
    )

clean_cell_coverage(available, reach_yx)

Return the fraction of grid cells with a clean cell within reach.

A cell is covered when a clean cell lies within reach_yx cells of it along each axis, itself included; the grid's edge clips the reach. An empty grid covers nothing.

Source code in src/hebog/algorithms/background.py

def clean_cell_coverage(
    available: npt.NDArray[np.bool_], reach_yx: tuple[int, int]
) -> float:
    """Return the fraction of grid cells with a clean cell within reach.

    A cell is covered when a clean cell lies within ``reach_yx`` cells of it
    along each axis, itself included; the grid's edge clips the reach. An
    empty grid covers nothing.
    """
    if available.size == 0:
        return 0.0
    reach_y, reach_x = reach_yx
    covered = np.asarray(
        maximum_filter(
            np.asarray(available, dtype=np.uint8),
            size=(2 * reach_y + 1, 2 * reach_x + 1),
            mode="constant",
            cval=0,
        ),
        dtype=np.uint8,
    )
    return float(np.count_nonzero(covered) / available.size)

estimate_rms_grid_batch(values, valid_pixels, grid, batch, config, *, protected_pixels=None)

Estimate a rectangular window block without loops over grid cells.

A fine-grid window that intersects protected_pixels is deliberately unavailable as a whole. This prevents connected bright-source support from contributing to either the background or RMS statistic while retaining the established deterministic interpolation fallback.

Source code in src/hebog/algorithms/background.py

def estimate_rms_grid_batch(  # noqa: PLR0913
    values: npt.NDArray[np.floating[Any]],
    valid_pixels: npt.NDArray[np.bool_],
    grid: RmsGridGeometry,
    batch: RmsWindowBatch,
    config: RmsWindowStatisticsConfig,
    *,
    protected_pixels: npt.NDArray[np.bool_] | None = None,
) -> RmsGridBatchStatistics:
    """Estimate a rectangular window block without loops over grid cells.

    A fine-grid window that intersects ``protected_pixels`` is deliberately
    unavailable as a whole. This prevents connected bright-source support
    from contributing to either the background or RMS statistic while
    retaining the established deterministic interpolation fallback.
    """
    _require_canonical_batch(grid, batch)
    bounded_values = np.asarray(values)
    bounded_validity = np.asarray(valid_pixels, dtype=np.bool_)
    if (
        bounded_values.shape != batch.read_bounds.shape_yx
        or bounded_validity.shape != batch.read_bounds.shape_yx
    ):
        raise ValueError(
            "RMS batch values and validity must match read bounds"
        )
    bounded_protection: npt.NDArray[np.bool_] | None = None
    if protected_pixels is not None:
        bounded_protection = np.asarray(protected_pixels, dtype=np.bool_)
        if bounded_protection.shape != batch.read_bounds.shape_yx:
            raise ValueError(
                "protected pixels must match RMS batch read bounds"
            )

    window_shape = grid.effective_window_shape_yx
    value_views = np.lib.stride_tricks.sliding_window_view(
        bounded_values,
        window_shape,
    )
    validity_views = np.lib.stride_tricks.sliding_window_view(
        bounded_validity,
        window_shape,
    )
    y_offsets = (
        np.asarray(
            grid.window_starts_y[batch.grid_y_start : batch.grid_y_stop],
            dtype=np.int64,
        )
        - batch.read_bounds.y_start
    )
    x_offsets = (
        np.asarray(
            grid.window_starts_x[batch.grid_x_start : batch.grid_x_stop],
            dtype=np.int64,
        )
        - batch.read_bounds.x_start
    )
    selected_values = value_views[
        y_offsets[:, np.newaxis],
        x_offsets[np.newaxis, :],
    ]
    selected_validity = validity_views[
        y_offsets[:, np.newaxis],
        x_offsets[np.newaxis, :],
    ]
    all_values = np.reshape(
        selected_values,
        (batch.cell_count, *window_shape),
    )
    all_validity = np.reshape(
        selected_validity,
        (batch.cell_count, *window_shape),
    )
    statistics = estimate_rms_window_statistics(
        all_values,
        all_validity,
        config,
    )
    protected_window_count = 0
    if bounded_protection is not None:
        protection_views = np.lib.stride_tricks.sliding_window_view(
            bounded_protection,
            window_shape,
        )
        selected_protection = protection_views[
            y_offsets[:, np.newaxis],
            x_offsets[np.newaxis, :],
        ]
        protected_windows = np.any(
            np.reshape(
                selected_protection,
                (batch.cell_count, *window_shape),
            ),
            axis=_STATISTIC_AXES,
        )
        protected_window_count = int(np.count_nonzero(protected_windows))
        if protected_window_count:
            background = np.array(statistics.background, copy=True)
            rms = np.array(statistics.rms, copy=True)
            available = np.array(statistics.available, copy=True)
            retained_sample_count = np.array(
                statistics.retained_sample_count,
                copy=True,
            )
            background[protected_windows] = np.nan
            rms[protected_windows] = np.nan
            available[protected_windows] = False
            retained_sample_count[protected_windows] = 0
            statistics = RmsWindowStatistics(
                background=cast(
                    npt.NDArray[np.float64],
                    _read_only(background),
                ),
                rms=cast(
                    npt.NDArray[np.float64],
                    _read_only(rms),
                ),
                available=cast(
                    npt.NDArray[np.bool_],
                    _read_only(available),
                ),
                valid_sample_count=statistics.valid_sample_count,
                retained_sample_count=cast(
                    npt.NDArray[np.int64],
                    _read_only(retained_sample_count),
                ),
            )
    return RmsGridBatchStatistics(
        batch=batch,
        statistics=statistics,
        protected_window_count=protected_window_count,
    )

estimate_rms_window_statistics(windows, valid_pixels, config)

Estimate robust background and RMS for a batch of 2-D windows.

Non-finite or explicitly invalid pixels do not contribute. A window with too few retained samples, or whose clipped spread is no greater than its noise floor, has NaN estimates and available=False so a later interpolation stage can apply its documented fallback policy. An available window therefore has a positive RMS.

The noise floor is the window's largest absolute valid value, taken before clipping, times single precision's machine epsilon (2**-23, about 1.2e-7): single precision's resolution at that value. Radio images are commonly made and stored in single precision, so a spread finer than that resolution beside the window's brightest pixel is below what such an image can be relied on to hold, and is not taken for noise. Samples of one value, whose spread is exactly zero, are the limit case; another is a window holding a noise-free source, whose tails' clipped spread lies near 1e-12 of its peak. A window of a noise-free source's far tails alone has a spread close to its own values, which no floor relative to them can tell from noise. Real noise falls under the floor only within a window whose brightest pixel is more than eight million times the noise, and that window then takes the fallback, as a window over a protected source does. The floor scales with the values, so the units and absolute level of an image do not change which windows it rejects.

The kernel runs on executor threads, so it emits no warning and never touches the process's warning filters: an invalid or empty window is ordinary input, reported through available.

Source code in src/hebog/algorithms/background.py

def estimate_rms_window_statistics(
    windows: npt.NDArray[np.floating[Any]],
    valid_pixels: npt.NDArray[np.bool_],
    config: RmsWindowStatisticsConfig,
) -> RmsWindowStatistics:
    """Estimate robust background and RMS for a batch of 2-D windows.

    Non-finite or explicitly invalid pixels do not contribute. A window with
    too few retained samples, or whose clipped spread is no greater than its
    noise floor, has NaN estimates and ``available=False`` so a later
    interpolation stage can apply its documented fallback policy. An
    available window therefore has a positive RMS.

    The noise floor is the window's largest absolute valid value, taken
    before clipping, times single precision's machine epsilon (2**-23,
    about 1.2e-7): single precision's resolution at that value. Radio
    images are commonly made and stored in single precision, so a spread
    finer than that resolution beside the window's brightest pixel is below
    what such an image can be relied on to hold, and is not taken for
    noise. Samples of
    one value, whose spread is exactly zero, are the limit case; another is
    a window holding a noise-free source, whose tails' clipped spread lies
    near 1e-12 of its peak. A window of a noise-free source's far tails
    alone has a spread close to its own values, which no floor relative to
    them can tell from noise. Real noise falls under the floor only within
    a window whose brightest pixel is more than eight million times the
    noise, and that window then takes the fallback, as a window over a
    protected source does. The floor scales with the values, so the units
    and absolute level of an image do not change which windows it rejects.

    The kernel runs on executor threads, so it emits no warning and never
    touches the process's warning filters: an invalid or empty window is
    ordinary input, reported through ``available``.
    """
    values = np.asarray(windows, dtype=np.float64)
    validity = np.asarray(valid_pixels, dtype=np.bool_)
    if values.ndim != _WINDOW_DIMENSIONS:
        raise ValueError(
            "RMS window values must be a three-dimensional "
            "(window, y, x) batch"
        )
    if min(values.shape) < 1:
        raise ValueError("RMS window batches and windows must be non-empty")
    if validity.shape != values.shape:
        raise ValueError(
            "RMS window values and valid pixels need the same shape"
        )

    effective_validity = validity & np.isfinite(values)
    valid_sample_count = np.count_nonzero(
        effective_validity,
        axis=_STATISTIC_AXES,
    ).astype(np.int64, copy=False)
    # The floor is taken before clipping, so a source the clipping removes
    # still sets the window's scale; a window without a valid sample has a
    # floor of zero, and too few samples anyway.
    noise_floor = _NOISE_FLOOR_PER_WINDOW_SCALE * np.max(
        np.abs(values),
        axis=_STATISTIC_AXES,
        where=effective_validity,
        initial=0.0,
    )
    # Excluded samples are carried as NaN, which the reductions below skip
    # about twice as fast as a mask. They are also masked for the clipping,
    # which otherwise warns about every non-finite sample, and which returns
    # the plain array with every excluded or clipped sample NaN. A window of
    # subnormal values, as the tails of a noise-free source are, makes its
    # compiled spread invalid; ``errstate`` is local to the calling thread's
    # context, unlike a warning filter, and that spread is then NaN, which
    # leaves the window unavailable below.
    retained = np.where(effective_validity, values, np.nan)
    with np.errstate(invalid="ignore", divide="ignore"):
        retained = cast(
            npt.NDArray[np.float64],
            sigma_clip(
                np.ma.MaskedArray(retained, mask=~effective_validity),
                sigma=config.clipping_sigma,
                maxiters=config.maximum_iterations,
                cenfunc="median",
                stdfunc="std",
                axis=_STATISTIC_AXES,
                masked=False,
                copy=False,
            ),
        )
    retained_sample_count = np.count_nonzero(
        np.isfinite(retained),
        axis=_STATISTIC_AXES,
    ).astype(np.int64, copy=False)
    enough_samples = retained_sample_count >= config.minimum_samples
    # Only windows with enough samples are reduced, because NumPy warns about
    # a window with no sample. Selecting them copies, so a batch whose
    # windows all have enough is reduced as it is.
    samples = retained if np.all(enough_samples) else retained[enough_samples]
    window_background = np.nanmedian(samples, axis=_STATISTIC_AXES)
    window_rms = np.nanstd(samples, axis=_STATISTIC_AXES)
    # A spread no greater than the floor, such as that of samples which all
    # hold one value, is no noise estimate, and the window takes the
    # fallback of one that holds too few samples.
    measures_noise = window_rms > noise_floor[enough_samples]
    available = enough_samples.copy()
    available[enough_samples] = measures_noise
    background = np.full(available.shape, np.nan, dtype=np.float64)
    rms = np.full(available.shape, np.nan, dtype=np.float64)
    background[available] = window_background[measures_noise]
    rms[available] = window_rms[measures_noise]

    return RmsWindowStatistics(
        background=cast(npt.NDArray[np.float64], _read_only(background)),
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
        available=cast(npt.NDArray[np.bool_], _read_only(available)),
        valid_sample_count=cast(
            npt.NDArray[np.int64],
            _read_only(valid_sample_count),
        ),
        retained_sample_count=cast(
            npt.NDArray[np.int64],
            _read_only(retained_sample_count),
        ),
    )

group_rms_window_batches(batches, *, read_side_pixels)

Group batches whose reads start in the same square of the image.

The squares are read_side_pixels wide from the image origin. A group's read is the union of its batches' reads, which tile a rectangle of the grid, so it is at most one batch read wider than the square along each axis. Groups follow the raster order of their squares, and batches keep their order within a group. The cells a batch estimates come from its own windows only, so grouping changes the reads and the tasks, never a statistic.

Source code in src/hebog/algorithms/background.py

def group_rms_window_batches(
    batches: Sequence[RmsWindowBatch],
    *,
    read_side_pixels: int,
) -> tuple[RmsWindowBatchGroup, ...]:
    """Group batches whose reads start in the same square of the image.

    The squares are ``read_side_pixels`` wide from the image origin. A
    group's read is the union of its batches' reads, which tile a
    rectangle of the grid, so it is at most one batch read wider than the
    square along each axis. Groups follow the raster order of their
    squares, and batches keep their order within a group. The cells a
    batch estimates come from its own windows only, so grouping changes
    the reads and the tasks, never a statistic.
    """
    if (
        isinstance(read_side_pixels, bool)
        or not isinstance(read_side_pixels, Integral)
        or read_side_pixels < 1
    ):
        raise ValueError("read_side_pixels must be a positive integer")
    squares: dict[tuple[int, int], list[RmsWindowBatch]] = {}
    for batch in batches:
        square = (
            batch.read_bounds.y_start // read_side_pixels,
            batch.read_bounds.x_start // read_side_pixels,
        )
        squares.setdefault(square, []).append(batch)
    return tuple(
        RmsWindowBatchGroup(
            batches=tuple(members),
            read_bounds=ImageBounds(
                min(batch.read_bounds.y_start for batch in members),
                max(batch.read_bounds.y_stop for batch in members),
                min(batch.read_bounds.x_start for batch in members),
                max(batch.read_bounds.x_stop for batch in members),
            ),
        )
        for _, members in sorted(squares.items())
    )

interpolate_prepared_rms_grid(grid, bounds, valid_pixels, *, extrapolate_rms=True)

Interpolate cached samples, optionally extending RMS edge values.

Fine noise cells have stochastic slopes, not a measured noise gradient beyond their centres. Constant edge extension preserves positive convex weights there. Background edge slopes span the extrapolation distance, not a possibly tiny gap between the final two window centres. Extended RMS follows the same secants but never falls below the edge cell: a noise level falling towards the image edge is held there, one rising is extended. At a corner the second axis' secant runs through samples the first has already extended, so a corner value can be lower than before the bound, but not below the corner cell.

Source code in src/hebog/algorithms/background.py

def interpolate_prepared_rms_grid(
    grid: PreparedRmsGrid,
    bounds: ImageBounds,
    valid_pixels: npt.NDArray[np.bool_],
    *,
    extrapolate_rms: bool = True,
) -> BackgroundRmsTile:
    """Interpolate cached samples, optionally extending RMS edge values.

    Fine noise cells have stochastic slopes, not a measured noise gradient
    beyond their centres. Constant edge extension preserves positive convex
    weights there. Background edge slopes span the extrapolation distance,
    not a possibly tiny gap between the final two window centres. Extended
    RMS follows the same secants but never falls below the edge cell: a noise
    level falling towards the image edge is held there, one rising is
    extended. At a corner the second axis' secant runs through samples the
    first has already extended, so a corner value can be lower than before
    the bound, but not below the corner cell.
    """
    bounds.require_inside(grid.geometry.image_shape_yx)
    validity = np.asarray(valid_pixels, dtype=np.bool_)
    if validity.shape != bounds.shape_yx:
        raise ValueError("tile valid pixels must match the requested bounds")
    if not grid.scientifically_available:
        background = np.full(bounds.shape_yx, np.nan, dtype=np.float64)
        rms = np.full(bounds.shape_yx, np.nan, dtype=np.float64)
    else:
        (
            sample_y,
            sample_x,
            coarse_background,
            coarse_rms,
        ) = _expand_singleton_grid_axes(grid)
        image_shape_yx = grid.geometry.image_shape_yx
        output_y = np.arange(bounds.y_start, bounds.y_stop, dtype=np.float64)
        output_x = np.arange(bounds.x_start, bounds.x_stop, dtype=np.float64)
        y_coordinates, x_coordinates = np.meshgrid(
            output_y,
            output_x,
            indexing="ij",
        )
        query_points = np.stack((y_coordinates, x_coordinates), axis=-1)
        background_y, background_x, background_samples = (
            _extend_grid_to_image_edges(
                sample_y,
                sample_x,
                coarse_background,
                image_shape_yx,
                bounded_below_by_edge_cell=False,
            )
        )
        background = np.asarray(
            RegularGridInterpolator(
                (background_y, background_x),
                background_samples,
                method="linear",
                bounds_error=False,
                fill_value=None,  # pyright: ignore[reportArgumentType]
            )(query_points),
            dtype=np.float64,
        )
        if extrapolate_rms:
            rms_interpolator = _extended_rms_interpolator(grid)
            rms_points = query_points
        else:
            rms_interpolator = RegularGridInterpolator(
                (sample_y, sample_x),
                coarse_rms,
                method="linear",
                bounds_error=False,
                fill_value=None,  # pyright: ignore[reportArgumentType]
            )
            rms_points = np.stack(
                (
                    np.clip(y_coordinates, sample_y[0], sample_y[-1]),
                    np.clip(x_coordinates, sample_x[0], sample_x[-1]),
                ),
                axis=-1,
            )
        rms = np.asarray(rms_interpolator(rms_points), dtype=np.float64)
        # Every sample is a non-negative cell value or an extension held at
        # or above one, so this only guards the interpolation's rounding.
        np.maximum(rms, 0.0, out=rms)
        background[~validity] = np.nan
        rms[~validity] = np.nan
    return BackgroundRmsTile(
        bounds=bounds,
        background=cast(
            npt.NDArray[np.float64],
            _read_only(background),
        ),
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
        scientifically_available=grid.scientifically_available,
        fallback_cell_count=grid.fallback_cell_count,
    )

plan_rms_grid(*, image_shape_yx, window_shape_yx, step_yx)

Plan deterministic global RMS windows independent of tile geometry.

Source code in src/hebog/algorithms/background.py

def plan_rms_grid(
    *,
    image_shape_yx: tuple[int, int],
    window_shape_yx: tuple[int, int],
    step_yx: tuple[int, int],
) -> RmsGridGeometry:
    """Plan deterministic global RMS windows independent of tile geometry."""
    if min(image_shape_yx) < 1:
        raise ValueError("image shape dimensions must be positive")
    if min(window_shape_yx) < 1:
        raise ValueError("window shape dimensions must be positive")
    if min(step_yx) < 1:
        raise ValueError("RMS grid step dimensions must be positive")
    if any(
        step > window
        for step, window in zip(step_yx, window_shape_yx, strict=True)
    ):
        raise ValueError("RMS grid step cannot exceed its window dimension")

    effective_window_shape_yx = (
        min(image_shape_yx[0], window_shape_yx[0]),
        min(image_shape_yx[1], window_shape_yx[1]),
    )
    starts_y = _axis_window_starts(
        length=image_shape_yx[0],
        window=window_shape_yx[0],
        step=step_yx[0],
    )
    starts_x = _axis_window_starts(
        length=image_shape_yx[1],
        window=window_shape_yx[1],
        step=step_yx[1],
    )
    y_offset = (effective_window_shape_yx[0] - 1) / 2.0
    x_offset = (effective_window_shape_yx[1] - 1) / 2.0
    return RmsGridGeometry(
        image_shape_yx=image_shape_yx,
        configured_window_shape_yx=window_shape_yx,
        effective_window_shape_yx=effective_window_shape_yx,
        step_yx=step_yx,
        window_starts_y=starts_y,
        window_starts_x=starts_x,
        sample_coordinates_y=tuple(start + y_offset for start in starts_y),
        sample_coordinates_x=tuple(start + x_offset for start in starts_x),
    )

plan_rms_window_batches(grid, *, maximum_cells)

Group coarse cells into bounded rectangular image-read batches.

Source code in src/hebog/algorithms/background.py

def plan_rms_window_batches(
    grid: RmsGridGeometry,
    *,
    maximum_cells: int,
) -> tuple[RmsWindowBatch, ...]:
    """Group coarse cells into bounded rectangular image-read batches."""
    if (
        isinstance(maximum_cells, bool)
        or not isinstance(maximum_cells, Integral)
        or maximum_cells < 1
    ):
        raise ValueError("maximum_cells must be a positive integer")
    cells_y, cells_x = grid.shape_yx
    block_y = min(cells_y, max(1, isqrt(maximum_cells)))
    block_x = min(cells_x, max(1, maximum_cells // block_y))
    window_y, window_x = grid.effective_window_shape_yx
    batches: list[RmsWindowBatch] = []
    for grid_y_start in range(0, cells_y, block_y):
        grid_y_stop = min(cells_y, grid_y_start + block_y)
        for grid_x_start in range(0, cells_x, block_x):
            grid_x_stop = min(cells_x, grid_x_start + block_x)
            y_start = grid.window_starts_y[grid_y_start]
            x_start = grid.window_starts_x[grid_x_start]
            y_stop = grid.window_starts_y[grid_y_stop - 1] + window_y
            x_stop = grid.window_starts_x[grid_x_stop - 1] + window_x
            batches.append(
                RmsWindowBatch(
                    grid_y_start=grid_y_start,
                    grid_y_stop=grid_y_stop,
                    grid_x_start=grid_x_start,
                    grid_x_stop=grid_x_stop,
                    read_bounds=ImageBounds(y_start, y_stop, x_start, x_stop),
                )
            )
    return tuple(batches)

prepare_crowded_local_noise_rms_grid(statistics, coarse, *, maximum_noise_ratio, clean_cell_reach_yx)

Hold a crowded field's coarse RMS to its clean windows' noise.

In a field too crowded for its clean fine windows to reach most of its cells, filling every cell from a few distant windows would copy their noise across the field. A cell with a clean window within clean_cell_reach_yx cells takes its nearest clean window's RMS, as local noise does; beyond that reach the unprotected coarse RMS stands, as PyBDSF's does, wherever it reads at most maximum_noise_ratio times the nearest clean window's RMS. Above that, a coarse window holds emission rather than noise, as over a source wider than one, and the clean window's noise stands there, as local noise fills a source wider than its reach. The background is filled from the clean windows, as any filled grid's; a grid without a clean window stays unavailable.

Parameters:

Name	Type	Description	Default
statistics	RmsGridStatistics	The local-noise grid's assembled window statistics.	required
coarse	PreparedRmsGrid	The prepared unprotected coarse grid of the same image.	required
maximum_noise_ratio	float	How many times the nearest clean window's RMS the coarse RMS may read and still stand.	required
clean_cell_reach_yx	tuple[int, int]	How many cells along each axis, either side of a cell, a clean window may lie and set its RMS.	required

Raises:

Type	Description
ValueError	If the ratio is not finite or below 1, or the local noise grid is available and the coarse grid is not.

Source code in src/hebog/algorithms/background.py

def prepare_crowded_local_noise_rms_grid(
    statistics: RmsGridStatistics,
    coarse: PreparedRmsGrid,
    *,
    maximum_noise_ratio: float,
    clean_cell_reach_yx: tuple[int, int],
) -> PreparedRmsGrid:
    """Hold a crowded field's coarse RMS to its clean windows' noise.

    In a field too crowded for its clean fine windows to reach most of its
    cells, filling every cell from a few distant windows would copy their
    noise across the field. A cell with a clean window within
    ``clean_cell_reach_yx`` cells takes its nearest clean window's RMS, as
    local noise does; beyond that reach the unprotected coarse RMS stands, as
    PyBDSF's does, wherever it reads at most ``maximum_noise_ratio`` times
    the nearest clean window's RMS. Above that, a coarse window holds
    emission rather than noise, as over a source wider than one, and the
    clean window's noise stands there, as local noise fills a source wider
    than its reach. The background is filled from the clean windows, as any
    filled grid's; a grid without a clean window stays unavailable.

    Args:
        statistics: The local-noise grid's assembled window statistics.
        coarse: The prepared unprotected coarse grid of the same image.
        maximum_noise_ratio: How many times the nearest clean window's RMS
            the coarse RMS may read and still stand.
        clean_cell_reach_yx: How many cells along each axis, either side of
            a cell, a clean window may lie and set its RMS.

    Raises:
        ValueError: If the ratio is not finite or below 1, or the local
            noise grid is available and the coarse grid is not.
    """
    if not isfinite(maximum_noise_ratio) or maximum_noise_ratio < 1:
        raise ValueError("maximum_noise_ratio must be finite and at least 1")
    prepared = prepare_rms_grid_for_interpolation(statistics)
    if not prepared.scientifically_available:
        return prepared
    if not coarse.scientifically_available:
        raise ValueError("a crowded local-noise grid needs a coarse grid")
    cell_y, cell_x = np.meshgrid(
        np.asarray(statistics.geometry.sample_coordinates_y, dtype=np.float64),
        np.asarray(statistics.geometry.sample_coordinates_x, dtype=np.float64),
        indexing="ij",
    )
    coarse_rms = np.asarray(
        _extended_rms_interpolator(coarse)(
            np.stack((cell_y, cell_x), axis=-1)
        ),
        dtype=np.float64,
    )
    reach_y, reach_x = clean_cell_reach_yx
    within_reach = maximum_filter(
        np.asarray(statistics.available, dtype=np.uint8),
        size=(2 * reach_y + 1, 2 * reach_x + 1),
        mode="constant",
        cval=0,
    ).astype(np.bool_)
    rms = np.where(
        within_reach | (coarse_rms > maximum_noise_ratio * prepared.rms),
        prepared.rms,
        coarse_rms,
    )
    return replace(
        prepared,
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
    )

prepare_local_noise_rms_grid(statistics, coarse, *, minimum_coarse_fraction, clean_cell_reach_yx)

Fill a local-noise grid without letting a fill fall far below its noise.

Cells are filled from their nearest available cell, as :func:prepare_rms_grid_for_interpolation fills any grid, except that a filled cell with an available cell within clean_cell_reach_yx cells of it never falls below minimum_coarse_fraction times a reference: the coarse RMS at its centre, or the largest available cell in that reach if that is smaller. Local noise blanks every window that touches guarded source support, and beside a sharp step in the noise the nearest clean window can lie on the quieter side, where its estimate would lower the noise beside the step. A coarse window over extended emission includes emission the clipping keeps, so where no clean window nearby reads as high as the coarse RMS, the excess is not noise and the clean windows set the reference; and a cell with no clean window in reach, as inside a source wider than twice the reach, keeps its nearest fill, because the coarse RMS alone is not a reference there. A cell that a clean window measured keeps its own estimate, however low, since that is what local noise measures. The background is filled as before, and an unavailable grid stays unavailable.

Parameters:

Name	Type	Description	Default
statistics	RmsGridStatistics	The local-noise grid's assembled window statistics.	required
coarse	PreparedRmsGrid	The prepared coarse grid of the same image, available.	required
minimum_coarse_fraction	float	The fraction of the reference below which a filled cell's RMS never falls, from 0, no floor, to 1.	required
clean_cell_reach_yx	tuple[int, int]	How many cells along each axis, either side of a filled cell, a clean cell may lie and still floor it and cap its reference.	required

Raises:

Type	Description
ValueError	If the fraction is not finite or lies outside 0 to 1, a reach is not a non-negative integer, or the local-noise grid is available and the coarse grid is not.

Source code in src/hebog/algorithms/background.py

def prepare_local_noise_rms_grid(
    statistics: RmsGridStatistics,
    coarse: PreparedRmsGrid,
    *,
    minimum_coarse_fraction: float,
    clean_cell_reach_yx: tuple[int, int],
) -> PreparedRmsGrid:
    """Fill a local-noise grid without letting a fill fall far below its noise.

    Cells are filled from their nearest available cell, as
    :func:`prepare_rms_grid_for_interpolation` fills any grid, except that a
    filled cell with an available cell within ``clean_cell_reach_yx`` cells
    of it never falls below ``minimum_coarse_fraction`` times a reference:
    the coarse RMS at its centre, or the largest available cell in that
    reach if that is smaller. Local noise blanks every window that touches
    guarded source support, and beside a sharp step in the noise the nearest
    clean window can lie on the quieter side, where its estimate would lower
    the noise beside the step. A coarse window over extended emission
    includes emission the clipping keeps, so where no clean window nearby
    reads as high as the coarse RMS, the excess is not noise and the clean
    windows set the reference; and a cell with no clean window in reach,
    as inside a source wider than twice the reach, keeps its nearest fill,
    because the coarse RMS alone is not a reference there. A cell that a
    clean window measured keeps its own estimate, however low, since that
    is what local noise measures. The background is filled as before, and
    an unavailable grid stays unavailable.

    Args:
        statistics: The local-noise grid's assembled window statistics.
        coarse: The prepared coarse grid of the same image, available.
        minimum_coarse_fraction: The fraction of the reference below which a
            filled cell's RMS never falls, from 0, no floor, to 1.
        clean_cell_reach_yx: How many cells along each axis, either side of
            a filled cell, a clean cell may lie and still floor it and cap
            its reference.

    Raises:
        ValueError: If the fraction is not finite or lies outside 0 to 1, a
            reach is not a non-negative integer, or the local-noise grid is
            available and the coarse grid is not.
    """
    if (
        not isfinite(minimum_coarse_fraction)
        or not 0 <= minimum_coarse_fraction <= 1
    ):
        raise ValueError(
            "minimum_coarse_fraction must be finite and between 0 and 1"
        )
    if len(clean_cell_reach_yx) != _REACH_AXES or any(
        isinstance(reach, bool) or not isinstance(reach, Integral) or reach < 0
        for reach in clean_cell_reach_yx
    ):
        raise ValueError(
            "clean_cell_reach_yx must be two non-negative integers"
        )
    prepared = prepare_rms_grid_for_interpolation(statistics)
    if not prepared.scientifically_available:
        return prepared
    if not coarse.scientifically_available:
        raise ValueError("a local-noise grid needs an available coarse grid")
    filled_y, filled_x = np.nonzero(~statistics.available)
    if filled_y.size == 0:
        return prepared
    reach_y, reach_x = (int(reach) for reach in clean_cell_reach_yx)
    largest_clean = np.asarray(
        maximum_filter(
            np.where(statistics.available, statistics.rms, -np.inf),
            size=(2 * reach_y + 1, 2 * reach_x + 1),
            mode="constant",
            cval=-np.inf,
        ),
        dtype=np.float64,
    )[filled_y, filled_x]
    # The coarse RMS is evaluated at filled cells only, so the driver holds
    # one point per filled cell, not two coordinates per cell of the grid.
    centres = np.stack(
        (
            np.asarray(
                statistics.geometry.sample_coordinates_y, dtype=np.float64
            )[filled_y],
            np.asarray(
                statistics.geometry.sample_coordinates_x, dtype=np.float64
            )[filled_x],
        ),
        axis=-1,
    )
    coarse_rms = np.asarray(
        _extended_rms_interpolator(coarse)(centres), dtype=np.float64
    )
    # With no clean cell in reach the maximum is -inf and there is no floor:
    # the coarse RMS alone is not a reference, because over a source wider
    # than the reach the unprotected coarse window holds the source's
    # emission.
    reference = np.where(
        np.isfinite(largest_clean),
        np.minimum(coarse_rms, largest_clean),
        0.0,
    )
    rms = np.array(prepared.rms, copy=True)
    rms[filled_y, filled_x] = np.maximum(
        rms[filled_y, filled_x], minimum_coarse_fraction * reference
    )
    return replace(
        prepared,
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
    )

prepare_refinement_rms_grid(statistics, coarse, *, fill_reach_pixels)

Fill a refinement grid without carrying its RMS beyond a reach.

Cells are filled from their nearest available cell, as :func:prepare_rms_grid_for_interpolation fills any grid, except that a cell whose nearest available cell's centre lies more than fill_reach_pixels from its own has no fine RMS: it takes the coarse RMS at its centre, so the coarse estimate stands there. A refinement grid over a crowded field keeps few windows clear of protected sources, and without the reach one of them would set the noise hundreds of pixels from where it was measured, across any step in the noise between. The background is filled as before, and an unavailable grid stays unavailable.

Parameters:

Name	Type	Description	Default
statistics	RmsGridStatistics	The refinement grid's assembled window statistics.	required
coarse	PreparedRmsGrid	The prepared coarse grid of the same image, available.	required
fill_reach_pixels	float	The farthest distance, in pixels between window centres, over which a clean window's RMS fills another cell; a cell exactly at it is filled.	required

Raises:

Type	Description
ValueError	If the reach is negative or not finite, or the refinement grid is available and the coarse grid is not.

Source code in src/hebog/algorithms/background.py

def prepare_refinement_rms_grid(
    statistics: RmsGridStatistics,
    coarse: PreparedRmsGrid,
    *,
    fill_reach_pixels: float,
) -> PreparedRmsGrid:
    """Fill a refinement grid without carrying its RMS beyond a reach.

    Cells are filled from their nearest available cell, as
    :func:`prepare_rms_grid_for_interpolation` fills any grid, except that a
    cell whose nearest available cell's centre lies more than
    ``fill_reach_pixels`` from its own has no fine RMS: it takes the coarse
    RMS at its centre, so the coarse estimate stands there. A refinement grid
    over a crowded field keeps few windows clear of protected sources, and
    without the reach one of them would set the noise hundreds of pixels
    from where it was measured, across any step in the noise between. The
    background is filled as before, and an unavailable grid stays
    unavailable.

    Args:
        statistics: The refinement grid's assembled window statistics.
        coarse: The prepared coarse grid of the same image, available.
        fill_reach_pixels: The farthest distance, in pixels between window
            centres, over which a clean window's RMS fills another cell; a
            cell exactly at it is filled.

    Raises:
        ValueError: If the reach is negative or not finite, or the
            refinement grid is available and the coarse grid is not.
    """
    if not isfinite(fill_reach_pixels) or fill_reach_pixels < 0:
        raise ValueError("fill_reach_pixels must be finite and non-negative")
    prepared = prepare_rms_grid_for_interpolation(statistics)
    if not prepared.scientifically_available:
        return prepared
    if not coarse.scientifically_available:
        raise ValueError("a refinement grid needs an available coarse grid")
    nearest_y, nearest_x = _nearest_available_indices(statistics.available)
    sample_y = np.asarray(
        statistics.geometry.sample_coordinates_y, dtype=np.float64
    )
    sample_x = np.asarray(
        statistics.geometry.sample_coordinates_x, dtype=np.float64
    )
    cell_y, cell_x = np.meshgrid(sample_y, sample_x, indexing="ij")
    beyond_reach = (
        np.hypot(sample_y[nearest_y] - cell_y, sample_x[nearest_x] - cell_x)
        > fill_reach_pixels
    )
    if not np.any(beyond_reach):
        return prepared
    coarse_rms = np.asarray(
        _extended_rms_interpolator(coarse)(
            np.stack((cell_y, cell_x), axis=-1)
        ),
        dtype=np.float64,
    )
    rms = np.where(beyond_reach, coarse_rms, prepared.rms)
    return replace(
        prepared,
        rms=cast(npt.NDArray[np.float64], _read_only(rms)),
    )

prepare_rms_grid_for_interpolation(statistics)

Fill sparse cells once without recomputing window statistics.

Source code in src/hebog/algorithms/background.py

def prepare_rms_grid_for_interpolation(
    statistics: RmsGridStatistics,
) -> PreparedRmsGrid:
    """Fill sparse cells once without recomputing window statistics."""
    available = statistics.available
    scientifically_available = bool(np.any(available))
    fallback_cells = np.asarray(~available, dtype=np.bool_)
    if scientifically_available:
        background = _nearest_available_values(
            statistics.background,
            available,
        )
        rms = _nearest_available_values(statistics.rms, available)
    else:
        background = np.full(statistics.geometry.shape_yx, np.nan)
        rms = np.full(statistics.geometry.shape_yx, np.nan)
        fallback_cells = np.zeros_like(available)
    return PreparedRmsGrid(
        geometry=statistics.geometry,
        background=cast(
            npt.NDArray[np.float64],
            _read_only(np.array(background, copy=True)),
        ),
        rms=cast(
            npt.NDArray[np.float64],
            _read_only(np.array(rms, copy=True)),
        ),
        fallback_cells=cast(
            npt.NDArray[np.bool_],
            _read_only(np.array(fallback_cells, copy=True)),
        ),
        scientifically_available=scientifically_available,
    )

subset_prepared_rms_grid(grid, bounds)

Copy the bounded coarse summary needed to interpolate one tile.

Source code in src/hebog/algorithms/background.py

def subset_prepared_rms_grid(
    grid: PreparedRmsGrid,
    bounds: ImageBounds,
) -> PreparedRmsGrid:
    """Copy the bounded coarse summary needed to interpolate one tile."""
    bounds.require_inside(grid.geometry.image_shape_yx)
    y_selection = _interpolation_axis_slice(
        grid.geometry.sample_coordinates_y,
        output_start=bounds.y_start,
        output_stop=bounds.y_stop,
        image_length=grid.geometry.image_shape_yx[0],
    )
    x_selection = _interpolation_axis_slice(
        grid.geometry.sample_coordinates_x,
        output_start=bounds.x_start,
        output_stop=bounds.x_stop,
        image_length=grid.geometry.image_shape_yx[1],
    )
    geometry = subset_rms_grid_geometry(grid.geometry, bounds)
    selection = (y_selection, x_selection)
    return PreparedRmsGrid(
        geometry=geometry,
        background=cast(
            npt.NDArray[np.float64],
            _read_only(np.array(grid.background[selection], copy=True)),
        ),
        rms=cast(
            npt.NDArray[np.float64],
            _read_only(np.array(grid.rms[selection], copy=True)),
        ),
        fallback_cells=cast(
            npt.NDArray[np.bool_],
            _read_only(np.array(grid.fallback_cells[selection], copy=True)),
        ),
        scientifically_available=grid.scientifically_available,
    )

subset_rms_grid_geometry(grid, bounds)

Return only globally anchored cells bracketing one bounded region.

Source code in src/hebog/algorithms/background.py

def subset_rms_grid_geometry(
    grid: RmsGridGeometry,
    bounds: ImageBounds,
) -> RmsGridGeometry:
    """Return only globally anchored cells bracketing one bounded region."""
    bounds.require_inside(grid.image_shape_yx)
    y_selection = _interpolation_axis_slice(
        grid.sample_coordinates_y,
        output_start=bounds.y_start,
        output_stop=bounds.y_stop,
        image_length=grid.image_shape_yx[0],
    )
    x_selection = _interpolation_axis_slice(
        grid.sample_coordinates_x,
        output_start=bounds.x_start,
        output_stop=bounds.x_stop,
        image_length=grid.image_shape_yx[1],
    )
    return RmsGridGeometry(
        image_shape_yx=grid.image_shape_yx,
        configured_window_shape_yx=grid.configured_window_shape_yx,
        effective_window_shape_yx=grid.effective_window_shape_yx,
        step_yx=grid.step_yx,
        window_starts_y=grid.window_starts_y[y_selection],
        window_starts_x=grid.window_starts_x[x_selection],
        sample_coordinates_y=grid.sample_coordinates_y[y_selection],
        sample_coordinates_x=grid.sample_coordinates_x[x_selection],
    )

hebog.stages.background

Bounded source and executor orchestration for background estimation.

AdaptiveRmsRegion dataclass

One merged bright-candidate region and its local fine-grid cache.

Source code in src/hebog/stages/background.py

@dataclass(frozen=True, slots=True)
class AdaptiveRmsRegion:
    """One merged bright-candidate region and its local fine-grid cache."""

    grid: PreparedRmsGrid
    bright_candidate_positions_yx: tuple[tuple[float, float], ...]
    protected_pixel_count: int
    protected_window_count: int

AdaptiveRmsTileSummary dataclass

One tile-local adaptive interpolant and its candidate positions.

Source code in src/hebog/stages/background.py

@dataclass(frozen=True, slots=True)
class AdaptiveRmsTileSummary:
    """One tile-local adaptive interpolant and its candidate positions."""

    grid: PreparedRmsGrid
    bright_candidate_positions_yx: tuple[tuple[float, float], ...]

BackgroundRmsGrids dataclass

Prepared coarse and optional sparse adaptive interpolation caches.

largest_bright_region_read_pixels is the most pixels one task read to estimate the fine grids around bright candidates, zero if none did.

Source code in src/hebog/stages/background.py

@dataclass(frozen=True, slots=True)
class BackgroundRmsGrids:
    """Prepared coarse and optional sparse adaptive interpolation caches.

    ``largest_bright_region_read_pixels`` is the most pixels one task read
    to estimate the fine grids around bright candidates, zero if none did.
    """

    coarse: PreparedRmsGrid
    adaptive_regions: tuple[AdaptiveRmsRegion, ...]
    coarse_protected_pixel_count: int = 0
    local_noise: PreparedRmsGrid | None = None
    local_noise_protected_window_count: int = 0
    largest_bright_region_read_pixels: int = 0

    @property
    def adaptive_estimated_cell_count(self) -> int:
        """Return the total fine cells retained across local regions."""
        return sum(
            region.grid.geometry.cell_count for region in self.adaptive_regions
        )

    @property
    def adaptive_protected_pixel_count(self) -> int:
        """Return bounded source-support pixels seen across fine regions."""
        return sum(
            region.protected_pixel_count for region in self.adaptive_regions
        )

    @property
    def adaptive_protected_window_count(self) -> int:
        """Return fine windows rejected for intersecting source support."""
        return sum(
            region.protected_window_count for region in self.adaptive_regions
        )

adaptive_estimated_cell_count property

Return the total fine cells retained across local regions.

adaptive_protected_pixel_count property

Return bounded source-support pixels seen across fine regions.

adaptive_protected_window_count property

Return fine windows rejected for intersecting source support.

BackgroundRmsTileRequest dataclass

Bounded interpolation summaries and blend metadata for one tile.

Source code in src/hebog/stages/background.py

@dataclass(frozen=True, slots=True)
class BackgroundRmsTileRequest:
    """Bounded interpolation summaries and blend metadata for one tile."""

    partition: TilePartition
    coarse: PreparedRmsGrid
    adaptive_regions: tuple[AdaptiveRmsTileSummary, ...]
    influence_radius_pixels: float | None
    transition_width_pixels: float | None
    local_noise: PreparedRmsGrid | None = None
    background: Literal["estimated", "zero"] = "estimated"

MultiscaleSourceProtection dataclass

Caller-supplied beam and detection policy for fine-grid protection.

Source code in src/hebog/stages/background.py

@dataclass(frozen=True, slots=True)
class MultiscaleSourceProtection:
    """Caller-supplied beam and detection policy for fine-grid protection."""

    beam: BeamShapePixels
    source_finder: SourceFinderConfig
    minimum_support_fraction: float

    def __post_init__(self) -> None:
        """Validate filter admission before any image or candidate exists."""
        if not isfinite(self.minimum_support_fraction) or not (
            0.0 < self.minimum_support_fraction <= 1.0
        ):
            raise ValueError(
                "multiscale protection support fraction is invalid"
            )

__post_init__()

Validate filter admission before any image or candidate exists.

Source code in src/hebog/stages/background.py

def __post_init__(self) -> None:
    """Validate filter admission before any image or candidate exists."""
    if not isfinite(self.minimum_support_fraction) or not (
        0.0 < self.minimum_support_fraction <= 1.0
    ):
        raise ValueError(
            "multiscale protection support fraction is invalid"
        )

estimate_background_rms_grids(source, image_shape_yx, config, executor, *, bright_candidate_positions_yx, source_protection_island_threshold_sigma=None, multiscale_protection=None)

Estimate cached global coarse and sparse adaptive RMS summaries.

Source code in src/hebog/stages/background.py

def estimate_background_rms_grids(  # noqa: PLR0913
    source: WindowReadable,
    image_shape_yx: tuple[int, int],
    config: BackgroundRmsConfig,
    executor: Executor,
    *,
    bright_candidate_positions_yx: tuple[tuple[float, float], ...],
    source_protection_island_threshold_sigma: float | None = None,
    multiscale_protection: MultiscaleSourceProtection | None = None,
) -> BackgroundRmsGrids:
    """Estimate cached global coarse and sparse adaptive RMS summaries."""
    if min(image_shape_yx) < 1:
        raise ValueError("image shape dimensions must be positive")
    use_constant_map = _use_constant_map(image_shape_yx, config)
    if (
        use_constant_map
        and prod(image_shape_yx) > config.maximum_constant_map_pixels
    ):
        raise ValueError(
            "constant-map pixel limit would require an unbounded image read"
        )
    coarse_geometry = plan_rms_grid(
        image_shape_yx=image_shape_yx,
        window_shape_yx=(
            image_shape_yx
            if use_constant_map
            else config.coarse.window_shape_yx
        ),
        step_yx=(
            image_shape_yx if use_constant_map else config.coarse.step_yx
        ),
    )
    coarse_statistics = estimate_rms_grid(
        source,
        coarse_geometry,
        config.coarse,
        executor,
    )
    coarse = prepare_rms_grid_for_interpolation(coarse_statistics)

    return refine_background_rms_grids(
        source,
        BackgroundRmsGrids(coarse=coarse, adaptive_regions=()),
        config,
        executor,
        bright_candidate_positions_yx=bright_candidate_positions_yx,
        source_protection_island_threshold_sigma=(
            source_protection_island_threshold_sigma
        ),
        multiscale_protection=multiscale_protection,
    )

estimate_background_rms_tile(source, request)

Read validity and interpolate one deterministic owned output core.

Source code in src/hebog/stages/background.py

def estimate_background_rms_tile(
    source: WindowReadable,
    request: BackgroundRmsTileRequest,
) -> BackgroundRmsTile:
    """Read validity and interpolate one deterministic owned output core."""
    bounds = request.partition.core_bounds
    image_window = source.read_window(bounds)
    return interpolate_background_rms_tile(image_window, request)

estimate_rms_grid(source, grid, config, executor)

Estimate one complete coarse grid through bounded executor batches.

One task estimates the batches whose reads start in one square of _SOURCE_READ_SIDE_PIXELS, from one read.

Source code in src/hebog/stages/background.py

def estimate_rms_grid(
    source: WindowReadable,
    grid: RmsGridGeometry,
    config: RmsGridConfig,
    executor: Executor,
) -> RmsGridStatistics:
    """Estimate one complete coarse grid through bounded executor batches.

    One task estimates the batches whose reads start in one square of
    ``_SOURCE_READ_SIDE_PIXELS``, from one read.
    """
    groups = group_rms_window_batches(
        plan_rms_window_batches(
            grid,
            maximum_cells=config.maximum_batch_cells,
        ),
        read_side_pixels=_SOURCE_READ_SIDE_PIXELS,
    )
    estimate_batches = partial(
        _estimate_source_batches,
        source=source,
        grid=grid,
        config=config,
    )
    results = executor.map_batches(
        estimate_batches,
        groups,
        requirement=read_requirement(
            _SOURCE_BATCH_BYTES_PER_READ_PIXEL,
            (group.read_bounds for group in groups),
        ),
    )
    return assemble_rms_grid_statistics(grid, chain.from_iterable(results))

interpolate_background_rms_tile(image_window, request)

Interpolate one owned core using an already-read source window.

Source code in src/hebog/stages/background.py

def interpolate_background_rms_tile(
    image_window: ImageWindow,
    request: BackgroundRmsTileRequest,
) -> BackgroundRmsTile:
    """Interpolate one owned core using an already-read source window."""
    bounds = request.partition.core_bounds
    if image_window.bounds != bounds:
        raise ValueError("image source returned different tile bounds")
    if (
        image_window.values.shape != bounds.shape_yx
        or image_window.valid_pixels.shape != bounds.shape_yx
    ):
        raise ValueError("image source returned a misaligned window")
    coarse = interpolate_prepared_rms_grid(
        request.coarse,
        bounds,
        image_window.valid_pixels,
    )
    if request.adaptive_regions and (
        request.influence_radius_pixels is None
        or request.transition_width_pixels is None
    ):
        raise ValueError("adaptive tile request is missing blend metadata")
    result = coarse
    for region in request.adaptive_regions:
        adaptive = interpolate_prepared_rms_grid(
            region.grid,
            bounds,
            image_window.valid_pixels,
        )
        result = blend_adaptive_background_rms(
            result,
            adaptive,
            region.bright_candidate_positions_yx,
            influence_radius_pixels=cast(
                float, request.influence_radius_pixels
            ),
            transition_width_pixels=cast(
                float, request.transition_width_pixels
            ),
        )
    # Local noise refused for a field too crowded to measure it leaves the
    # unprotected coarse RMS that refine_background_rms_grids keeps. The fine
    # windows around a bright candidate protect only its own support, so in
    # such a field they read neighbouring sources as noise, and raising the
    # RMS from them cuts into those sources on the candidate's side.
    if (
        request.local_noise is not None
        and request.local_noise.scientifically_available
    ):
        noise = interpolate_prepared_rms_grid(
            request.local_noise,
            bounds,
            image_window.valid_pixels,
            extrapolate_rms=False,
        )
        result = replace(
            result,
            rms=noise.rms,
            fallback_cell_count=result.fallback_cell_count
            + noise.fallback_cell_count,
        )
    elif request.local_noise is not None:
        result = replace(result, rms=coarse.rms)
    if request.background == "zero":
        # PyBDSF's mean_map="zero": the RMS above stands, and nothing is
        # subtracted where a background was defined.
        result = replace(
            result,
            background=np.where(
                np.isfinite(result.background), 0.0, result.background
            ),
        )
    return result

prepare_background_rms_tile_request(partition, grids, config)

Build one local interpolation request without global grid payloads.

Source code in src/hebog/stages/background.py

def prepare_background_rms_tile_request(
    partition: TilePartition,
    grids: BackgroundRmsGrids,
    config: BackgroundRmsConfig,
) -> BackgroundRmsTileRequest:
    """Build one local interpolation request without global grid payloads."""
    coarse = subset_prepared_rms_grid(grids.coarse, partition.core_bounds)
    local_noise = (
        subset_prepared_rms_grid(grids.local_noise, partition.core_bounds)
        if grids.local_noise is not None
        else None
    )
    adaptive_config = config.adaptive
    if not grids.adaptive_regions or adaptive_config is None:
        return BackgroundRmsTileRequest(
            partition=partition,
            coarse=coarse,
            adaptive_regions=(),
            influence_radius_pixels=None,
            transition_width_pixels=None,
            local_noise=local_noise,
            background=config.background,
        )
    bounds = partition.core_bounds
    radius = adaptive_config.influence_radius_pixels
    summaries: list[AdaptiveRmsTileSummary] = []
    for region in grids.adaptive_regions:
        nearby_positions = tuple(
            (y_position, x_position)
            for y_position, x_position in (
                region.bright_candidate_positions_yx
            )
            if bounds.y_start - radius < y_position < bounds.y_stop + radius
            and bounds.x_start - radius < x_position < bounds.x_stop + radius
        )
        if nearby_positions:
            summaries.append(
                AdaptiveRmsTileSummary(
                    grid=subset_prepared_rms_grid(region.grid, bounds),
                    bright_candidate_positions_yx=nearby_positions,
                )
            )
    adaptive_regions = tuple(summaries)
    return BackgroundRmsTileRequest(
        partition=partition,
        coarse=coarse,
        adaptive_regions=adaptive_regions,
        influence_radius_pixels=(radius if adaptive_regions else None),
        transition_width_pixels=(
            adaptive_config.transition_width_pixels
            if adaptive_regions
            else None
        ),
        local_noise=local_noise,
        background=config.background,
    )

refine_background_rms_grids(source, coarse_grids, config, executor, *, bright_candidate_positions_yx, source_protection_island_threshold_sigma=None, multiscale_protection=None, protect_coarse_source_support=False, refine_local_noise=False)

Estimate sparse adaptive cells while reusing a prepared coarse grid.

Source code in src/hebog/stages/background.py

def refine_background_rms_grids(  # noqa: PLR0913
    source: WindowReadable,
    coarse_grids: BackgroundRmsGrids,
    config: BackgroundRmsConfig,
    executor: Executor,
    *,
    bright_candidate_positions_yx: tuple[tuple[float, float], ...],
    source_protection_island_threshold_sigma: float | None = None,
    multiscale_protection: MultiscaleSourceProtection | None = None,
    protect_coarse_source_support: bool = False,
    refine_local_noise: bool = False,
) -> BackgroundRmsGrids:
    """Estimate sparse adaptive cells while reusing a prepared coarse grid."""
    if coarse_grids.adaptive_regions or coarse_grids.local_noise is not None:
        raise ValueError("adaptive refinement requires a coarse-only cache")
    image_shape_yx = coarse_grids.coarse.geometry.image_shape_yx
    adaptive_config = config.adaptive
    if refine_local_noise:
        _require_local_noise_policy(
            config,
            multiscale_protection,
            source_protection_island_threshold_sigma,
        )
    adaptive_margin = (
        adaptive_config.influence_radius_pixels
        + max(adaptive_config.grid.step_yx)
        if adaptive_config is not None
        else 0.0
    )
    candidate_regions = _merge_candidate_regions(
        bright_candidate_positions_yx,
        image_shape_yx=image_shape_yx,
        margin_pixels=adaptive_margin,
    )

    if adaptive_config is None or (
        not candidate_regions
        and not protect_coarse_source_support
        and not refine_local_noise
    ):
        return coarse_grids
    _require_protection_threshold(
        source_protection_island_threshold_sigma, adaptive_config
    )
    if protect_coarse_source_support:
        _require_bounded_coarse_protection(
            image_shape_yx, config, source_protection_island_threshold_sigma
        )
    # Without a coarse estimate no pixel has a background, so nothing here
    # can refine it and the estimate stays unavailable as a whole; the fine
    # pilot alone would be 4.8 million cells at 15,402 pixels.
    if not coarse_grids.coarse.scientifically_available:
        return coarse_grids
    # A coarse RMS can dilute noise excursions and misclassify their peaks
    # as sources. This unmasked pilot only admits source protection.
    detection_rms = (
        prepare_rms_grid_for_interpolation(
            estimate_rms_grid(
                source,
                plan_rms_grid(
                    image_shape_yx=image_shape_yx,
                    window_shape_yx=adaptive_config.grid.window_shape_yx,
                    step_yx=adaptive_config.grid.step_yx,
                ),
                adaptive_config.grid,
                executor,
            )
        )
        if multiscale_protection is not None
        and (protect_coarse_source_support or refine_local_noise)
        else None
    )
    unprotected_grids, unprotected_regions = coarse_grids, candidate_regions
    if protect_coarse_source_support:
        assert source_protection_island_threshold_sigma is not None
        estimate_coarse = partial(
            _estimate_source_protected_grid,
            source=source,
            config=config.coarse,
            island_threshold_sigma=source_protection_island_threshold_sigma,
            multiscale_protection=multiscale_protection,
        )
        image_height, image_width = coarse_grids.coarse.geometry.image_shape_yx
        (protected_coarse,) = executor.map_batches(
            estimate_coarse,
            (
                _AdaptiveRegionRequest(
                    grid=coarse_grids.coarse.geometry,
                    coarse=coarse_grids.coarse,
                    positions_yx=bright_candidate_positions_yx,
                    protection="exclude-samples",
                    guard_window_shape_yx=adaptive_config.grid.window_shape_yx,
                    detection_rms=detection_rms,
                ),
            ),
            # The protected coarse grid is estimated over the whole image in
            # one task, so it declares the whole image.
            requirement=read_requirement(
                _PROTECTED_BYTES_PER_READ_PIXEL,
                (ImageBounds(0, image_height, 0, image_width),),
            ),
        )
        coarse_grids, candidate_regions = _adopt_protected_coarse(
            protected_coarse,
            coarse_grids,
            candidate_regions,
            source,
            executor,
            island_threshold_sigma=source_protection_island_threshold_sigma,
            margin_pixels=adaptive_margin,
        )
    if refine_local_noise:
        assert detection_rms is not None and multiscale_protection is not None
        statistics = _estimate_local_noise_grid(
            source,
            coarse_grids.coarse,
            detection_rms,
            config,
            executor,
            policy=multiscale_protection,
        )
        local_noise = _prepare_local_noise(
            statistics, coarse_grids.coarse, config
        )
        # A field so crowded that no fine window anywhere clears source
        # support is too crowded to protect: a protected coarse estimate
        # there rests on the few pixels farthest from every source. The
        # unprotected sigma-clipped estimate stands instead, as PyBDSF's
        # does; its anchors refine the background but not the RMS
        # (interpolate_background_rms_tile), and the local noise refines
        # nothing.
        if not local_noise.scientifically_available:
            coarse_grids, candidate_regions = (
                unprotected_grids,
                unprotected_regions,
            )
        coarse_grids = replace(
            coarse_grids,
            local_noise=local_noise,
            local_noise_protected_window_count=statistics.protected_window_count,
        )
    if not candidate_regions:
        return coarse_grids
    return _refine_bright_regions(
        source,
        coarse_grids,
        candidate_regions,
        config,
        executor,
        source_protection_island_threshold_sigma=source_protection_island_threshold_sigma,
        multiscale_protection=multiscale_protection,
    )

Detection and island topology

hebog.algorithms.detection

Pure bounded normalization and two-threshold detection kernels.

DetectionThresholdMasks dataclass

Immutable island-membership and strict detection-seed masks.

Source code in src/hebog/algorithms/detection.py

@dataclass(frozen=True, slots=True)
class DetectionThresholdMasks:
    """Immutable island-membership and strict detection-seed masks."""

    normalized_residual: npt.NDArray[np.float64]
    island_membership: npt.NDArray[np.bool_]
    detection_seeds: npt.NDArray[np.bool_]
    valid_pixel_count: int

detect_high_significance_candidates(image, valid_pixels, background, rms, *, threshold_sigma)

Apply one strict threshold for adaptive-RMS candidate topology.

Source code in src/hebog/algorithms/detection.py

def detect_high_significance_candidates(
    image: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    background: npt.ArrayLike,
    rms: npt.ArrayLike,
    *,
    threshold_sigma: float,
) -> DetectionThresholdMasks:
    """Apply one strict threshold for adaptive-RMS candidate topology."""
    if not np.isfinite(threshold_sigma) or threshold_sigma <= 0:
        raise ValueError("candidate threshold must be finite and positive")
    normalized, scientifically_valid = normalize_residual(
        image,
        valid_pixels,
        background,
        rms,
    )
    candidate_pixels = scientifically_valid & (normalized > threshold_sigma)
    candidate_pixels.setflags(write=False)
    return DetectionThresholdMasks(
        normalized_residual=normalized,
        island_membership=candidate_pixels,
        detection_seeds=candidate_pixels,
        valid_pixel_count=int(np.count_nonzero(scientifically_valid)),
    )

detect_threshold_masks(image, valid_pixels, background, rms, config)

Threshold one bounded image tile without persisting an SNR plane.

Island membership includes pixels exactly at the island threshold. Detection seeds require a value strictly above the detection threshold. This explicit asymmetry reproduces the reviewed initial compact-detection contract without introducing workflow defaults into the kernel.

Source code in src/hebog/algorithms/detection.py

def detect_threshold_masks(
    image: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    background: npt.ArrayLike,
    rms: npt.ArrayLike,
    config: SourceFinderConfig,
) -> DetectionThresholdMasks:
    """Threshold one bounded image tile without persisting an SNR plane.

    Island membership includes pixels exactly at the island threshold.
    Detection seeds require a value strictly above the detection threshold.
    This explicit asymmetry reproduces the reviewed initial compact-detection
    contract without introducing workflow defaults into the kernel.
    """
    normalized, scientifically_valid = normalize_residual(
        image,
        valid_pixels,
        background,
        rms,
    )
    island_membership = scientifically_valid & (
        normalized >= config.island_threshold_sigma
    )
    detection_seeds = scientifically_valid & (
        normalized > config.detection_threshold_sigma
    )
    island_membership.setflags(write=False)
    detection_seeds.setflags(write=False)
    return DetectionThresholdMasks(
        normalized_residual=normalized,
        island_membership=island_membership,
        detection_seeds=detection_seeds,
        valid_pixel_count=int(np.count_nonzero(scientifically_valid)),
    )

normalize_residual(image, valid_pixels, background, rms)

Return immutable normalized residuals and scientific validity.

Source code in src/hebog/algorithms/detection.py

def normalize_residual(
    image: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    background: npt.ArrayLike,
    rms: npt.ArrayLike,
) -> tuple[npt.NDArray[np.float64], npt.NDArray[np.bool_]]:
    """Return immutable normalized residuals and scientific validity."""
    image_array = _as_float_plane(image, name="image")
    background_array = _as_float_plane(background, name="background")
    rms_array = _as_float_plane(rms, name="rms")
    validity = np.asarray(valid_pixels)
    if validity.ndim != _IMAGE_DIMENSIONS:
        raise ValueError("valid_pixels must be two-dimensional")
    if not np.issubdtype(validity.dtype, np.bool_):
        raise TypeError("valid_pixels must be a boolean array")
    if not (
        image_array.shape
        == background_array.shape
        == rms_array.shape
        == validity.shape
    ):
        raise ValueError("detection arrays must have the same shape")

    scientifically_valid = (
        np.asarray(validity, dtype=np.bool_)
        & np.isfinite(image_array)
        & np.isfinite(background_array)
        & np.isfinite(rms_array)
        & (rms_array > 0)
    )
    normalized = np.full(image_array.shape, np.nan, dtype=np.float64)
    np.divide(
        image_array - background_array,
        rms_array,
        out=normalized,
        where=scientifically_valid,
    )
    normalized.setflags(write=False)
    scientifically_valid.setflags(write=False)
    return normalized, scientifically_valid

hebog.algorithms.labelling

Eight-connected local island labelling over deterministic tile cores.

LocalIslandSummary dataclass

Small mergeable facts for one tile-local connected component.

Source code in src/hebog/algorithms/labelling.py

@dataclass(frozen=True, slots=True)
class LocalIslandSummary:
    """Small mergeable facts for one tile-local connected component."""

    local_label: int
    pixel_count: int
    bounds: ImageBounds
    peak_signal_to_noise: float
    peak_position_yx: tuple[int, int]
    first_pixel_yx: tuple[int, int]
    touches_image_edge: bool
    contains_detection_seed: bool
    peak_response_jy_per_beam: float | None = None

LocalIslandTile dataclass

One locally labelled core plus mergeable summaries and boundaries.

Source code in src/hebog/algorithms/labelling.py

@dataclass(frozen=True, slots=True)
class LocalIslandTile:
    """One locally labelled core plus mergeable summaries and boundaries."""

    partition: TilePartition
    labels: npt.NDArray[np.int32]
    islands: tuple[LocalIslandSummary, ...]
    boundary_labels: TileBoundaryLabels

    def compact_summary(self) -> LocalIslandTileSummary:
        """Drop the image-sized label core before scheduler reconciliation."""
        return LocalIslandTileSummary(
            partition=self.partition,
            islands=self.islands,
            boundary_labels=self.boundary_labels,
        )

compact_summary()

Drop the image-sized label core before scheduler reconciliation.

Source code in src/hebog/algorithms/labelling.py

def compact_summary(self) -> LocalIslandTileSummary:
    """Drop the image-sized label core before scheduler reconciliation."""
    return LocalIslandTileSummary(
        partition=self.partition,
        islands=self.islands,
        boundary_labels=self.boundary_labels,
    )

LocalIslandTileSummary dataclass

Compact tile topology safe to return through an executor.

Source code in src/hebog/algorithms/labelling.py

@dataclass(frozen=True, slots=True)
class LocalIslandTileSummary:
    """Compact tile topology safe to return through an executor."""

    partition: TilePartition
    islands: tuple[LocalIslandSummary, ...]
    boundary_labels: TileBoundaryLabels

TileBoundaryLabels dataclass

Local labels on four owned core boundaries for reconciliation.

Source code in src/hebog/algorithms/labelling.py

@dataclass(frozen=True, slots=True)
class TileBoundaryLabels:
    """Local labels on four owned core boundaries for reconciliation."""

    top: npt.NDArray[np.int32]
    bottom: npt.NDArray[np.int32]
    left: npt.NDArray[np.int32]
    right: npt.NDArray[np.int32]

label_detection_tile(masks, partition, *, image_shape_yx, response_jy_per_beam=None)

Label one owned core and reduce component properties without copies.

response_jy_per_beam is the optional filtered response the island was detected on. When it is given, each summary also carries that plane's per-island maximum, so a caller that needs the peak response of a feature does not have to store the response plane for a later pass.

Source code in src/hebog/algorithms/labelling.py

def label_detection_tile(
    masks: DetectionThresholdMasks,
    partition: TilePartition,
    *,
    image_shape_yx: tuple[int, int],
    response_jy_per_beam: npt.NDArray[np.float64] | None = None,
) -> LocalIslandTile:
    """Label one owned core and reduce component properties without copies.

    ``response_jy_per_beam`` is the optional filtered response the island was
    detected on. When it is given, each summary also carries that plane's
    per-island maximum, so a caller that needs the peak response of a feature
    does not have to store the response plane for a later pass.
    """
    partition.core_bounds.require_inside(image_shape_yx)
    expected_shape = partition.core_bounds.shape_yx
    arrays = (
        masks.normalized_residual,
        masks.island_membership,
        masks.detection_seeds,
    )
    if any(array.shape != expected_shape for array in arrays):
        raise ValueError("detection masks must match the tile core shape")
    if (
        response_jy_per_beam is not None
        and response_jy_per_beam.shape != expected_shape
    ):
        raise ValueError("detection response must match the tile core shape")
    if np.any(masks.detection_seeds & ~masks.island_membership):
        raise ValueError("detection seeds must be island members")

    connectivity = np.ones((3, 3), dtype=np.bool_)
    raw_labels, count = cast(
        tuple[npt.NDArray[np.int32], int],
        ndimage.label(
            masks.island_membership,
            structure=connectivity,
        ),
    )
    labels = np.asarray(raw_labels, dtype=np.int32)
    bounds = partition.core_bounds
    image_width = image_shape_yx[1]

    if count:
        flat_labels = labels.ravel()
        member_indices = np.flatnonzero(flat_labels)
        member_labels = flat_labels[member_indices]
        member_values = masks.normalized_residual.ravel()[member_indices]
        pixel_counts = np.bincount(
            member_labels,
            minlength=count + 1,
        )[1:]
        local_y, local_x = np.divmod(member_indices, labels.shape[1])
        member_global_linear = np.asarray(
            (local_y + bounds.y_start) * image_width
            + local_x
            + bounds.x_start,
            dtype=np.int64,
        )
        maximum_lookup = np.full(count + 1, -np.inf, dtype=np.float64)
        np.maximum.at(maximum_lookup, member_labels, member_values)
        peak_values = maximum_lookup[1:]
        peak_responses = _peak_responses(
            response_jy_per_beam,
            member_indices=member_indices,
            member_labels=member_labels,
            label_count=count,
        )
        sentinel = np.iinfo(np.int64).max
        first_lookup = np.full(count + 1, sentinel, dtype=np.int64)
        np.minimum.at(first_lookup, member_labels, member_global_linear)
        first_linear = first_lookup[1:]
        peak_members = member_values == maximum_lookup[member_labels]
        peak_lookup = np.full(count + 1, sentinel, dtype=np.int64)
        np.minimum.at(
            peak_lookup,
            member_labels[peak_members],
            member_global_linear[peak_members],
        )
        peak_linear = peak_lookup[1:]
        object_slices = ndimage.find_objects(labels, max_label=count)
        first_positions = _positions_from_linear_indices(
            first_linear,
            image_width=image_width,
        )
        peak_positions = _positions_from_linear_indices(
            peak_linear,
            image_width=image_width,
        )
        seed_labels = {
            int(value)
            for value in np.unique(labels[masks.detection_seeds])
            if value > 0
        }
        summaries = tuple(
            LocalIslandSummary(
                local_label=local_label,
                pixel_count=int(pixel_counts[local_label - 1]),
                bounds=ImageBounds(
                    bounds.y_start + object_slices[local_label - 1][0].start,
                    bounds.y_start + object_slices[local_label - 1][0].stop,
                    bounds.x_start + object_slices[local_label - 1][1].start,
                    bounds.x_start + object_slices[local_label - 1][1].stop,
                ),
                peak_signal_to_noise=float(peak_values[local_label - 1]),
                peak_position_yx=peak_positions[local_label - 1],
                first_pixel_yx=first_positions[local_label - 1],
                touches_image_edge=(
                    object_slices[local_label - 1][0].start + bounds.y_start
                    == 0
                    or object_slices[local_label - 1][0].stop + bounds.y_start
                    == image_shape_yx[0]
                    or object_slices[local_label - 1][1].start + bounds.x_start
                    == 0
                    or object_slices[local_label - 1][1].stop + bounds.x_start
                    == image_shape_yx[1]
                ),
                contains_detection_seed=local_label in seed_labels,
                peak_response_jy_per_beam=(
                    None
                    if peak_responses is None
                    else float(peak_responses[local_label - 1])
                ),
            )
            for local_label in range(1, count + 1)
        )
    else:
        summaries = ()

    labels = cast(npt.NDArray[np.int32], _read_only_copy(labels))
    return LocalIslandTile(
        partition=partition,
        labels=labels,
        islands=summaries,
        boundary_labels=TileBoundaryLabels(
            top=cast(npt.NDArray[np.int32], _read_only_copy(labels[0, :])),
            bottom=cast(
                npt.NDArray[np.int32],
                _read_only_copy(labels[-1, :]),
            ),
            left=cast(npt.NDArray[np.int32], _read_only_copy(labels[:, 0])),
            right=cast(
                npt.NDArray[np.int32],
                _read_only_copy(labels[:, -1]),
            ),
        ),
    )

relabelled(labels, value_by_label, *, otherwise=None)

Return the value each pixel's label maps to, in one pass over them.

A pixel whose label maps to nothing keeps its value in otherwise, or 0 when none is given. Comparing the plane with each label in turn costs a full pass per label, and a core holds thousands; this looks every pixel up once, a block of rows at a time, so the lookup adds memory for one block rather than for the plane.

Examples:

>>> relabelled(
...     np.array([[0, 3], [5, 3]], dtype=np.int32), {3: 7, 5: 9}
... ).tolist()
[[0, 7], [9, 7]]

Source code in src/hebog/algorithms/labelling.py

def relabelled(
    labels: npt.NDArray[np.integer[Any]],
    value_by_label: Mapping[int, int],
    *,
    otherwise: npt.NDArray[np.int32] | None = None,
) -> npt.NDArray[np.int32]:
    """Return the value each pixel's label maps to, in one pass over them.

    A pixel whose label maps to nothing keeps its value in ``otherwise``, or
    0 when none is given. Comparing the plane with each label in turn costs a
    full pass per label, and a core holds thousands; this looks every pixel
    up once, a block of rows at a time, so the lookup adds memory for one
    block rather than for the plane.

    Examples:
        >>> relabelled(
        ...     np.array([[0, 3], [5, 3]], dtype=np.int32), {3: 7, 5: 9}
        ... ).tolist()
        [[0, 7], [9, 7]]
    """
    keys = np.array(sorted(value_by_label), dtype=np.int64)
    values = np.array(
        [value_by_label[int(key)] for key in keys], dtype=np.int32
    )
    result = (
        np.zeros(labels.shape, dtype=np.int32)
        if otherwise is None
        else np.array(otherwise, dtype=np.int32, copy=True)
    )
    if not keys.size:
        return result
    for start in range(0, labels.shape[0], _RELABEL_BLOCK_ROWS):
        block = labels[start : start + _RELABEL_BLOCK_ROWS]
        position = np.minimum(np.searchsorted(keys, block), keys.size - 1)
        found = keys[position] == block
        result[start : start + _RELABEL_BLOCK_ROWS][found] = values[
            position[found]
        ]
    return result

hebog.algorithms.reconciliation

Deterministic reconciliation of tile-local connected island labels.

DetectedIsland dataclass

Canonical detection-stage topology before scientific measurement.

Source code in src/hebog/algorithms/reconciliation.py

@dataclass(frozen=True, slots=True)
class DetectedIsland:
    """Canonical detection-stage topology before scientific measurement."""

    island_id: str
    global_label: int
    pixel_count: int
    bounds: ImageBounds
    peak_signal_to_noise: float
    peak_position_yx: tuple[int, int]
    first_pixel_yx: tuple[int, int]
    touches_image_edge: bool
    peak_response_jy_per_beam: float | None = None

ReconciledIslands dataclass

Stable accepted islands and per-tile mappings for mask publication.

Source code in src/hebog/algorithms/reconciliation.py

@dataclass(frozen=True, slots=True)
class ReconciledIslands:
    """Stable accepted islands and per-tile mappings for mask publication."""

    islands: tuple[DetectedIsland, ...]
    tile_mappings: tuple[TileLabelMapping, ...]
    reduction_round_count: int

    def mapping_for_tile(self, tile_id: str) -> TileLabelMapping:
        """Return the exact mapping for one canonical tile."""
        for mapping in self.tile_mappings:
            if mapping.tile_id == tile_id:
                return mapping
        raise ValueError(f"reconciliation has no mapping for tile {tile_id!r}")

mapping_for_tile(tile_id)

Return the exact mapping for one canonical tile.

Source code in src/hebog/algorithms/reconciliation.py

def mapping_for_tile(self, tile_id: str) -> TileLabelMapping:
    """Return the exact mapping for one canonical tile."""
    for mapping in self.tile_mappings:
        if mapping.tile_id == tile_id:
            return mapping
    raise ValueError(f"reconciliation has no mapping for tile {tile_id!r}")

TileLabelMapping dataclass

Small mapping from one tile's local labels to accepted global labels.

Source code in src/hebog/algorithms/reconciliation.py

@dataclass(frozen=True, slots=True)
class TileLabelMapping:
    """Small mapping from one tile's local labels to accepted global labels."""

    tile_id: str
    local_labels: tuple[int, ...]
    global_labels: tuple[int, ...]

apply_reconciled_labels(tile, reconciliation)

Map one local label core to immutable stable global labels.

Source code in src/hebog/algorithms/reconciliation.py

def apply_reconciled_labels(
    tile: LocalIslandTile,
    reconciliation: ReconciledIslands,
) -> npt.NDArray[np.int64]:
    """Map one local label core to immutable stable global labels."""
    return apply_tile_label_mapping(
        tile,
        reconciliation.mapping_for_tile(tile.partition.tile_id),
    )

apply_tile_label_mapping(tile, mapping)

Map one local core using only its small executor-safe mapping.

Source code in src/hebog/algorithms/reconciliation.py

def apply_tile_label_mapping(
    tile: LocalIslandTile,
    mapping: TileLabelMapping,
) -> npt.NDArray[np.int64]:
    """Map one local core using only its small executor-safe mapping."""
    if mapping.tile_id != tile.partition.tile_id:
        raise ValueError("label mapping belongs to a different tile")
    maximum_local_label = int(np.max(tile.labels, initial=0))
    lookup = np.zeros(maximum_local_label + 1, dtype=np.int64)
    for local_label, global_label in zip(
        mapping.local_labels,
        mapping.global_labels,
        strict=True,
    ):
        if local_label > maximum_local_label:
            raise ValueError("reconciliation local label is outside tile")
        lookup[local_label] = global_label
    labels = lookup[tile.labels]
    labels.setflags(write=False)
    return labels

reconcile_candidate_tiles(manifest, tiles)

Merge strict high-significance candidates without a size cut.

Source code in src/hebog/algorithms/reconciliation.py

def reconcile_candidate_tiles(
    manifest: PartitionManifest,
    tiles: tuple[LocalIslandTileSummary, ...],
) -> ReconciledIslands:
    """Merge strict high-significance candidates without a size cut."""
    return _reconcile_island_tiles(
        manifest,
        tiles,
        minimum_island_pixels=1,
        maximum_island_pixels=None,
    )

reconcile_island_tiles(manifest, tiles, config)

Merge source-detection fragments under the configured size policy.

Source code in src/hebog/algorithms/reconciliation.py

def reconcile_island_tiles(
    manifest: PartitionManifest,
    tiles: tuple[_ReconciliationTile, ...],
    config: SourceFinderConfig,
) -> ReconciledIslands:
    """Merge source-detection fragments under the configured size policy."""
    return _reconcile_island_tiles(
        manifest,
        tiles,
        minimum_island_pixels=config.minimum_island_pixels,
        maximum_island_pixels=config.maximum_island_pixels,
    )

hebog.stages.detection

Bounded compact-detection execution and Zarr product publication.

DetectionStageResult dataclass

Published compact topology and bounded execution evidence.

Source code in src/hebog/stages/detection.py

@dataclass(frozen=True, slots=True)
class DetectionStageResult:
    """Published compact topology and bounded execution evidence."""

    generation: ProductGenerationManifest
    islands: tuple[DetectedIsland, ...]
    adaptive_candidate_positions_yx: tuple[tuple[float, float], ...]
    background_rms_grids: BackgroundRmsGrids
    boundary_label_count: int
    reconciliation_round_count: int
    separate_candidate_scan: bool

discover_adaptive_candidates(source, manifest, coarse_grids, config, executor)

Find deterministic global peaks for strict adaptive-RMS candidates.

Source code in src/hebog/stages/detection.py

def discover_adaptive_candidates(
    source: WindowReadable,
    manifest: PartitionManifest,
    coarse_grids: BackgroundRmsGrids,
    config: BackgroundRmsConfig,
    executor: Executor,
) -> tuple[tuple[float, float], ...]:
    """Find deterministic global peaks for strict adaptive-RMS candidates."""
    adaptive = config.adaptive
    if adaptive is None:
        return ()
    if coarse_grids.adaptive_regions:
        raise ValueError("candidate discovery requires a coarse-only cache")
    requests = tuple(
        prepare_background_rms_tile_request(tile, coarse_grids, config)
        for tile in manifest.tiles
    )
    scan = partial(
        _scan_candidate_cells,
        source=source,
        threshold_sigma=adaptive.candidate_threshold_sigma,
        image_shape_yx=manifest.image_shape_yx,
    )
    summaries = tuple(
        summary
        for block in executor.map_batches(
            scan,
            _cell_blocks(requests),
            requirement=read_requirement(
                _CANDIDATE_SCAN_BYTES_PER_READ_PIXEL,
                (request.partition.read_bounds for request in requests),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
        for summary in block
    )
    candidates = reconcile_candidate_tiles(manifest, summaries)
    return tuple(
        (float(island.peak_position_yx[0]), float(island.peak_position_yx[1]))
        for island in candidates.islands
    )

run_detection_from_coarse_grids(source, manifest, coarse_grids, *, config, executor, sink, multiscale_protection=None, protect_coarse_source_support=False, refine_local_noise=False)

Run Phase 3 from one immutable Phase 2 coarse-grid result.

This boundary lets component benchmarks and future pipeline execution reuse an already-computed coarse background/RMS grid without changing the automatic adaptive-candidate or final product semantics.

Source code in src/hebog/stages/detection.py

def run_detection_from_coarse_grids(  # noqa: PLR0913
    source: WindowReadable,
    manifest: PartitionManifest,
    coarse_grids: BackgroundRmsGrids,
    *,
    config: DetectionStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
    multiscale_protection: MultiscaleSourceProtection | None = None,
    protect_coarse_source_support: bool = False,
    refine_local_noise: bool = False,
) -> DetectionStageResult:
    """Run Phase 3 from one immutable Phase 2 coarse-grid result.

    This boundary lets component benchmarks and future pipeline execution
    reuse an already-computed coarse background/RMS grid without changing the
    automatic adaptive-candidate or final product semantics.
    """
    if sink.manifest != manifest:
        raise ValueError(
            "detection sink must use the stage partition manifest"
        )
    if coarse_grids.adaptive_regions or coarse_grids.local_noise is not None:
        raise ValueError("detection requires a coarse-only background cache")
    if coarse_grids.coarse.geometry.image_shape_yx != manifest.image_shape_yx:
        raise ValueError(
            "coarse background cache must match the detection image shape"
        )
    candidate_positions = discover_adaptive_candidates(
        source,
        manifest,
        coarse_grids,
        config.background_rms,
        executor,
    )
    grids = refine_background_rms_grids(
        source,
        coarse_grids,
        config.background_rms,
        executor,
        bright_candidate_positions_yx=candidate_positions,
        source_protection_island_threshold_sigma=(
            config.source_finder.island_threshold_sigma
            if config.source_finder.profile == "continuum"
            else None
        ),
        multiscale_protection=multiscale_protection,
        protect_coarse_source_support=protect_coarse_source_support,
        refine_local_noise=refine_local_noise,
    )
    for product_name, dtype in (
        ("background", np.dtype("<f8")),
        ("rms", np.dtype("<f8")),
        ("source-filtering-mask", np.dtype(np.bool_)),
    ):
        sink.initialize_product(product_name=product_name, dtype=dtype)

    requests = tuple(
        prepare_background_rms_tile_request(
            partition,
            grids,
            config.background_rms,
        )
        for partition in manifest.tiles
    )
    detect = partial(
        _detect_and_write_cells,
        source=source,
        sink=sink,
        config=config.source_finder,
        image_shape_yx=manifest.image_shape_yx,
    )
    tile_products = tuple(
        product
        for block in executor.map_batches(
            detect,
            _cell_blocks(requests),
            requirement=read_requirement(
                _DETECTION_BYTES_PER_READ_PIXEL,
                (request.partition.read_bounds for request in requests),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
        for product in block
    )
    summaries = tuple(result.summary for result in tile_products)
    reconciliation = reconcile_island_tiles(
        manifest,
        summaries,
        config.source_finder,
    )
    mask_requests = tuple(
        _MaskTileRequest(background_rms=request, label_mapping=mapping)
        for request, mapping in zip(
            requests,
            reconciliation.tile_mappings,
            strict=True,
        )
    )
    write_mask = partial(
        _write_source_filtering_masks,
        source=source,
        sink=sink,
        config=config.source_finder,
        image_shape_yx=manifest.image_shape_yx,
    )
    mask_chunks = tuple(
        chunk
        for block in executor.map_batches(
            write_mask,
            _cell_blocks(mask_requests),
            requirement=read_requirement(
                _MASK_BYTES_PER_READ_PIXEL,
                (request.partition.read_bounds for request in requests),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
        for chunk in block
    )
    product_chunks = (
        tuple(
            chunk
            for result in tile_products
            for chunk in result.product_chunks
        )
        + mask_chunks
    )
    generation = sink.publish_generation(
        product_names=("background", "rms", "source-filtering-mask"),
        chunks=product_chunks,
    )
    boundary_label_count = sum(
        summary.boundary_labels.top.size
        + summary.boundary_labels.bottom.size
        + summary.boundary_labels.left.size
        + summary.boundary_labels.right.size
        for summary in summaries
    )
    return DetectionStageResult(
        generation=generation,
        islands=reconciliation.islands,
        adaptive_candidate_positions_yx=candidate_positions,
        background_rms_grids=grids,
        boundary_label_count=boundary_label_count,
        reconciliation_round_count=reconciliation.reduction_round_count,
        separate_candidate_scan=config.background_rms.adaptive is not None,
    )

run_detection_stage(source, manifest, config, executor, sink, *, multiscale_protection=None, protect_coarse_source_support=False, refine_local_noise=False)

Run bounded compact detection and publish one complete generation.

The separate high-significance scan and final mask pass deliberately trade one extra bounded image read each for simple retry semantics and compact scheduler results. No normalized, label, background, RMS, or mask plane is gathered by the executor.

Source code in src/hebog/stages/detection.py

def run_detection_stage(  # noqa: PLR0913
    source: WindowReadable,
    manifest: PartitionManifest,
    config: DetectionStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
    *,
    multiscale_protection: MultiscaleSourceProtection | None = None,
    protect_coarse_source_support: bool = False,
    refine_local_noise: bool = False,
) -> DetectionStageResult:
    """Run bounded compact detection and publish one complete generation.

    The separate high-significance scan and final mask pass deliberately trade
    one extra bounded image read each for simple retry semantics and compact
    scheduler results. No normalized, label, background, RMS, or mask plane is
    gathered by the executor.
    """
    if sink.manifest != manifest:
        raise ValueError(
            "detection sink must use the stage partition manifest"
        )
    coarse_grids = estimate_background_rms_grids(
        source,
        manifest.image_shape_yx,
        config.background_rms,
        executor,
        bright_candidate_positions_yx=(),
    )
    return run_detection_from_coarse_grids(
        source,
        manifest,
        coarse_grids,
        config=config,
        executor=executor,
        sink=sink,
        multiscale_protection=multiscale_protection,
        protect_coarse_source_support=protect_coarse_source_support,
        refine_local_noise=refine_local_noise,
    )

Residual multiscale detection

hebog.algorithms.multiscale

Bounded one-tile serial kernels for Phase 5 multiscale science.

BeamShapePixels dataclass

Elliptical restoring beam or noise correlation in pixel units.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class BeamShapePixels:
    """Elliptical restoring beam or noise correlation in pixel units."""

    major_fwhm_pixels: float
    minor_fwhm_pixels: float
    position_angle_degrees: float

    def __post_init__(self) -> None:
        """Require finite, positive, ordered beam axes."""
        values = (
            self.major_fwhm_pixels,
            self.minor_fwhm_pixels,
            self.position_angle_degrees,
        )
        if not all(isfinite(value) for value in values):
            raise ValueError("beam shape values must be finite")
        if self.major_fwhm_pixels <= 0 or self.minor_fwhm_pixels <= 0:
            raise ValueError("beam shape axes must be positive")
        if self.minor_fwhm_pixels > self.major_fwhm_pixels:
            raise ValueError("beam shape minor axis cannot exceed major")

__post_init__()

Require finite, positive, ordered beam axes.

Source code in src/hebog/algorithms/multiscale.py

def __post_init__(self) -> None:
    """Require finite, positive, ordered beam axes."""
    values = (
        self.major_fwhm_pixels,
        self.minor_fwhm_pixels,
        self.position_angle_degrees,
    )
    if not all(isfinite(value) for value in values):
        raise ValueError("beam shape values must be finite")
    if self.major_fwhm_pixels <= 0 or self.minor_fwhm_pixels <= 0:
        raise ValueError("beam shape axes must be positive")
    if self.minor_fwhm_pixels > self.major_fwhm_pixels:
        raise ValueError("beam shape minor axis cannot exceed major")

PreparedScaleInputs dataclass

Validated physical residual and reused Phase 2 RMS products.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class PreparedScaleInputs:
    """Validated physical residual and reused Phase 2 RMS products."""

    residual_jy_per_beam: npt.NDArray[np.float64]
    rms_jy_per_beam: npt.NDArray[np.float64]
    scientifically_valid: npt.NDArray[np.bool_]

ResidualAtrousPlan dataclass

Bounded three-scale residual B3-spline à trous construction.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ResidualAtrousPlan:
    """Bounded three-scale residual B3-spline à trous construction."""

    beam: BeamShapePixels
    noise_correlation: BeamShapePixels | None
    base_kernel: npt.NDArray[np.float64]
    scales: tuple[ResidualAtrousScale, ...]
    dtype: Literal["float64"] = "float64"

    @property
    def scale_dilations_pixels(self) -> tuple[int, ...]:
        """Return the frozen dyadic holes between B3 coefficients."""
        return tuple(item.dilation_pixels for item in self.scales)

    @property
    def scale_halo_pixels(self) -> tuple[int, ...]:
        """Return cumulative finite halos at each adjacent smoothing."""
        return tuple(item.cumulative_halo_pixels for item in self.scales)

    @property
    def maximum_halo_pixels(self) -> int:
        """Return the widest cumulative halo required by the transform."""
        return self.scales[-1].cumulative_halo_pixels

    @property
    def convolution_count_per_evaluation(self) -> int:
        """Count sparse one-dimensional signal and support convolutions."""
        return 4 * len(self.scales)

    @property
    def temporary_plane_count(self) -> int:
        """Return the reviewed peak full-size scratch-plane count."""
        return 7

convolution_count_per_evaluation property

Count sparse one-dimensional signal and support convolutions.

maximum_halo_pixels property

Return the widest cumulative halo required by the transform.

scale_dilations_pixels property

Return the frozen dyadic holes between B3 coefficients.

scale_halo_pixels property

Return cumulative finite halos at each adjacent smoothing.

temporary_plane_count property

Return the reviewed peak full-size scratch-plane count.

ResidualAtrousResult dataclass

One-tile corrective transform with transient scale provenance.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ResidualAtrousResult:
    """One-tile corrective transform with transient scale provenance."""

    family: Literal["residual-b3-atrous"]
    responses: tuple[ScaleFilterResponse, ...]
    reconstructed_signal_jy_per_beam: npt.NDArray[np.float64]
    coarse_smoothing_jy_per_beam: npt.NDArray[np.float64]
    scientifically_valid: npt.NDArray[np.bool_]
    convolution_count: int
    temporary_plane_count: int
    maximum_workspace_bytes: int

ResidualAtrousScale dataclass

One cumulative B3-spline scale and calibrated coefficient noise.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ResidualAtrousScale:
    """One cumulative B3-spline scale and calibrated coefficient noise."""

    scale_order: int
    dilation_pixels: int
    cumulative_halo_pixels: int
    response_kernel: npt.NDArray[np.float64]
    independent_noise_gain: float
    correlated_noise_gain: float

ResidualMultiscaleIslandDetection dataclass

Seeded original-residual islands under the promoted scale policy.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ResidualMultiscaleIslandDetection:
    """Seeded original-residual islands under the promoted scale policy."""

    combined_snr: npt.NDArray[np.float64]
    retained_mask: npt.NDArray[np.bool_]
    component_labels: npt.NDArray[np.int32]
    component_count: int
    minimum_island_pixels: int
    reconstruction: SignificantAtrousReconstruction

ScaleFilter dataclass

One calibrated scale response built from shared smoothing kernels.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ScaleFilter:
    """One calibrated scale response built from shared smoothing kernels."""

    scale_order: int
    nominal_scale_beam_fwhm: float
    family: FilterFamily
    components: tuple[tuple[float, ScaleSmoothingKernel], ...]
    response_calibration: float
    response_kernel: npt.NDArray[np.float64]
    halo_pixels: int
    independent_noise_gain: float
    correlated_noise_gain: float
    truncation_fraction: float

ScaleFilterBank dataclass

Canonical filter plans and bounded structural costs for one family.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ScaleFilterBank:
    """Canonical filter plans and bounded structural costs for one family."""

    family: FilterFamily
    beam: BeamShapePixels
    noise_correlation: BeamShapePixels | None
    truncation_sigma: float
    filters: tuple[ScaleFilter, ...]
    dtype: Literal["float64"] = "float64"

    @property
    def maximum_halo_pixels(self) -> int:
        """Return the widest read-only halo required by any scale."""
        return max(item.halo_pixels for item in self.filters)

    @property
    def convolution_count_per_evaluation(self) -> int:
        """Return convolutions after shared wavelet smoothing reuse."""
        unique_smoothing_components = {
            component.width_beams
            for item in self.filters
            for _, component in item.components
        }
        return 2 * len(unique_smoothing_components) + len(self.filters)

    @property
    def temporary_plane_count(self) -> int:
        """Return the maximum simultaneous full-size temporary planes."""
        return 7 if self.family == "beam-aware-matched-filter" else 9

convolution_count_per_evaluation property

Return convolutions after shared wavelet smoothing reuse.

maximum_halo_pixels property

Return the widest read-only halo required by any scale.

temporary_plane_count property

Return the maximum simultaneous full-size temporary planes.

ScaleFilterBankResult dataclass

Complete one-tile serial response with explicit bounded costs.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ScaleFilterBankResult:
    """Complete one-tile serial response with explicit bounded costs."""

    family: FilterFamily
    responses: tuple[ScaleFilterResponse, ...]
    convolution_count: int
    temporary_plane_count: int
    maximum_workspace_bytes: int

ScaleFilterResponse dataclass

One immutable physical response, local noise, and validity plane.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ScaleFilterResponse:
    """One immutable physical response, local noise, and validity plane."""

    scale_order: int
    nominal_scale_beam_fwhm: float
    response_jy_per_beam: npt.NDArray[np.float64]
    effective_rms_jy_per_beam: npt.NDArray[np.float64]
    valid_support_fraction: npt.NDArray[np.float64]
    scientifically_valid: npt.NDArray[np.bool_]

ScaleSmoothingKernel dataclass

One shared normalized Gaussian smoothing component.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class ScaleSmoothingKernel:
    """One shared normalized Gaussian smoothing component."""

    width_beams: float
    halo_pixels: int
    values: npt.NDArray[np.float64]

SignificantAtrousReconstruction dataclass

Adjacent-scale positive signal and auditable scale detections.

Source code in src/hebog/algorithms/multiscale.py

@dataclass(frozen=True, slots=True)
class SignificantAtrousReconstruction:
    """Adjacent-scale positive signal and auditable scale detections."""

    signal_jy_per_beam: npt.NDArray[np.float64]
    support_mask: npt.NDArray[np.bool_]
    significant_scale_masks: tuple[npt.NDArray[np.bool_], ...]

build_residual_atrous_plan(beam, *, noise_correlation=None)

Build the frozen three-scale separable B3-spline à trous plan.

Source code in src/hebog/algorithms/multiscale.py

def build_residual_atrous_plan(
    beam: BeamShapePixels,
    *,
    noise_correlation: BeamShapePixels | None = None,
) -> ResidualAtrousPlan:
    """Build the frozen three-scale separable B3-spline à trous plan."""
    base_kernel = np.asarray([1.0, 4.0, 6.0, 4.0, 1.0], dtype=np.float64)
    base_kernel /= 16.0
    previous_smoothing = np.ones((1, 1), dtype=np.float64)
    scales: list[ResidualAtrousScale] = []
    for scale_order, (dilation_pixels, cumulative_halo) in enumerate(
        zip(
            (1, 2, 4),
            residual_atrous_scale_halos_pixels(),
            strict=True,
        ),
        start=1,
    ):
        sparse = _dilated_b3_kernel(base_kernel, dilation_pixels)
        stage_kernel = np.outer(sparse, sparse)
        current_smoothing = fftconvolve(
            previous_smoothing,
            stage_kernel,
            mode="full",
        )
        previous_padded = _centre_pad(
            previous_smoothing,
            halo_pixels=cumulative_halo,
        )
        response_kernel = np.asarray(
            previous_padded - current_smoothing,
            dtype=np.float64,
        )
        scales.append(
            ResidualAtrousScale(
                scale_order=scale_order,
                dilation_pixels=dilation_pixels,
                cumulative_halo_pixels=cumulative_halo,
                response_kernel=_read_only(response_kernel),
                independent_noise_gain=_noise_gain(response_kernel, None),
                correlated_noise_gain=_noise_gain(
                    response_kernel, noise_correlation
                ),
            )
        )
        previous_smoothing = current_smoothing
    return ResidualAtrousPlan(
        beam=beam,
        noise_correlation=noise_correlation,
        base_kernel=_read_only(base_kernel),
        scales=tuple(scales),
    )

build_scale_filter_bank(beam, *, family, scales, truncation_sigma=4.0, noise_correlation=None)

Build calibrated matched or undecimated-wavelet filter plans.

Source code in src/hebog/algorithms/multiscale.py

def build_scale_filter_bank(
    beam: BeamShapePixels,
    *,
    family: FilterFamily,
    scales: tuple[tuple[int, float], ...],
    truncation_sigma: float = 4.0,
    noise_correlation: BeamShapePixels | None = None,
) -> ScaleFilterBank:
    """Build calibrated matched or undecimated-wavelet filter plans."""
    if family not in {
        "beam-aware-matched-filter",
        "undecimated-wavelet",
    }:
        raise ValueError("unsupported scale filter family")
    _validate_scales(scales)
    if not isfinite(truncation_sigma) or not (
        _MINIMUM_TRUNCATION_SIGMA
        <= truncation_sigma
        <= _MAXIMUM_TRUNCATION_SIGMA
    ):
        raise ValueError("truncation_sigma must be finite and within [3, 8]")

    smoothing_cache: dict[float, ScaleSmoothingKernel] = {}

    def smoothing(width_beams: float) -> ScaleSmoothingKernel:
        cache_key = round(width_beams, 12)
        cached = smoothing_cache.get(cache_key)
        if cached is None:
            cached = _build_smoothing_kernel(
                beam,
                width_beams=width_beams,
                truncation_sigma=truncation_sigma,
            )
            smoothing_cache[cache_key] = cached
        return cached

    filters: list[ScaleFilter] = []
    for scale_order, nominal_width in scales:
        if family == "beam-aware-matched-filter":
            components = ((1.0, smoothing(nominal_width)),)
        else:
            components = (
                (1.0, smoothing(nominal_width / _WAVELET_WIDTH_RATIO)),
                (-1.0, smoothing(nominal_width * _WAVELET_WIDTH_RATIO)),
            )
        halo_pixels = max(item.halo_pixels for _, item in components)
        raw_response_kernel = np.zeros(
            (2 * halo_pixels + 1, 2 * halo_pixels + 1),
            dtype=np.float64,
        )
        for coefficient, component in components:
            raw_response_kernel += coefficient * _centre_pad(
                component.values,
                halo_pixels=halo_pixels,
            )
        template = _unit_integrated_flux_template(
            beam,
            nominal_scale_beam_fwhm=nominal_width,
            halo_pixels=halo_pixels,
        )
        unit_response = float(
            np.sum(raw_response_kernel * template, dtype=np.float64)
        )
        if not isfinite(unit_response) or unit_response <= 0:
            raise ValueError("scale filter has no positive nominal response")
        response_calibration = 1.0 / unit_response
        response_kernel = np.asarray(
            response_calibration * raw_response_kernel,
            dtype=np.float64,
        )
        independent_noise_gain = _noise_gain(response_kernel, None)
        correlated_noise_gain = _noise_gain(
            response_kernel,
            noise_correlation,
        )
        template_boundary = np.concatenate(
            (
                template[0, :],
                template[-1, :],
                template[:, 0],
                template[:, -1],
            )
        )
        relative_kernel_tail = float(
            np.max(np.abs(template_boundary))
            / template[halo_pixels, halo_pixels]
        )
        filters.append(
            ScaleFilter(
                scale_order=scale_order,
                nominal_scale_beam_fwhm=nominal_width,
                family=family,
                components=components,
                response_calibration=response_calibration,
                response_kernel=_read_only(response_kernel),
                halo_pixels=halo_pixels,
                independent_noise_gain=independent_noise_gain,
                correlated_noise_gain=correlated_noise_gain,
                truncation_fraction=relative_kernel_tail,
            )
        )
    return ScaleFilterBank(
        family=family,
        beam=beam,
        noise_correlation=noise_correlation,
        truncation_sigma=truncation_sigma,
        filters=tuple(filters),
    )

calibrated_scale_snrs(responses, *, minimum_support_fraction)

Return immutable calibrated SNR planes for aligned scale responses.

Source code in src/hebog/algorithms/multiscale.py

def calibrated_scale_snrs(
    responses: tuple[ScaleFilterResponse, ...],
    *,
    minimum_support_fraction: float,
) -> tuple[npt.NDArray[np.float64], ...]:
    """Return immutable calibrated SNR planes for aligned scale responses."""
    if not responses:
        raise ValueError("scale responses must not be empty")
    if (
        not isfinite(minimum_support_fraction)
        or not 0 < minimum_support_fraction <= 1
    ):
        raise ValueError("minimum support fraction must be within (0, 1]")
    shape = responses[0].response_jy_per_beam.shape
    scale_snrs: list[npt.NDArray[np.float64]] = []
    for response in responses:
        arrays = (
            response.response_jy_per_beam,
            response.effective_rms_jy_per_beam,
            response.valid_support_fraction,
            response.scientifically_valid,
        )
        if any(array.shape != shape for array in arrays):
            raise ValueError("scale responses must have the same shape")
        scale_validity = (
            response.scientifically_valid
            & (response.valid_support_fraction >= minimum_support_fraction)
            & np.isfinite(response.response_jy_per_beam)
            & np.isfinite(response.effective_rms_jy_per_beam)
            & (response.effective_rms_jy_per_beam > 0)
        )
        scale_snr = np.full(shape, -np.inf, dtype=np.float64)
        np.divide(
            response.response_jy_per_beam,
            response.effective_rms_jy_per_beam,
            out=scale_snr,
            where=scale_validity,
        )
        scale_snrs.append(_read_only(scale_snr))
    return tuple(scale_snrs)

detect_residual_multiscale_islands(prepared_inputs, matched_filter, atrous_result, beam, config)

Seed and grow promoted islands on original residual pixels.

Source code in src/hebog/algorithms/multiscale.py

def detect_residual_multiscale_islands(
    prepared_inputs: PreparedScaleInputs,
    matched_filter: ScaleFilterBankResult,
    atrous_result: ResidualAtrousResult,
    beam: BeamShapePixels,
    config: ResidualMultiscaleDetectionConfig,
) -> ResidualMultiscaleIslandDetection:
    """Seed and grow promoted islands on original residual pixels."""
    if matched_filter.family != "beam-aware-matched-filter":
        raise ValueError("multiscale seed aid must be the matched filter")
    if atrous_result.family != "residual-b3-atrous":
        raise ValueError("multiscale representation must be residual B3")
    shape = prepared_inputs.residual_jy_per_beam.shape
    if (
        prepared_inputs.rms_jy_per_beam.shape != shape
        or prepared_inputs.scientifically_valid.shape != shape
    ):
        raise ValueError("prepared multiscale inputs must have the same shape")
    reconstruction = reconstruct_significant_atrous(
        atrous_result,
        detection_sigma=config.detection_threshold_sigma,
        island_sigma=config.island_threshold_sigma,
        minimum_support_fraction=config.minimum_scale_support_fraction,
    )
    combined_snr = _maximum_calibrated_scale_snr(
        shape,
        matched_filter.responses,
        minimum_support_fraction=config.minimum_scale_support_fraction,
    )
    atrous_snr = _maximum_calibrated_scale_snr(
        shape,
        atrous_result.responses,
        minimum_support_fraction=config.minimum_scale_support_fraction,
    )
    atrous_snr[~reconstruction.support_mask] = -np.inf
    np.maximum(combined_snr, atrous_snr, out=combined_snr)

    direct_snr = np.full(shape, -np.inf, dtype=np.float64)
    direct_validity = (
        prepared_inputs.scientifically_valid
        & np.isfinite(prepared_inputs.residual_jy_per_beam)
        & np.isfinite(prepared_inputs.rms_jy_per_beam)
        & (prepared_inputs.rms_jy_per_beam > 0)
    )
    np.divide(
        prepared_inputs.residual_jy_per_beam,
        prepared_inputs.rms_jy_per_beam,
        out=direct_snr,
        where=direct_validity,
    )
    np.maximum(combined_snr, direct_snr, out=combined_snr)
    combined_snr[~direct_validity] = -np.inf

    original_support = direct_validity & (
        direct_snr >= config.island_threshold_sigma
    )
    raw_labels, _ = cast(
        tuple[npt.NDArray[np.int32], int],
        label(original_support, structure=np.ones((3, 3), dtype=np.int8)),
    )
    seed_labels = np.unique(
        raw_labels[combined_snr >= config.detection_threshold_sigma]
    )
    seed_labels = seed_labels[seed_labels > 0]
    retained = np.isin(raw_labels, seed_labels)

    minimum_island_pixels = minimum_residual_island_pixels(beam, config)
    label_count = int(np.max(raw_labels)) + 1
    retained_counts = np.bincount(raw_labels[retained], minlength=label_count)
    accepted = retained_counts >= minimum_island_pixels
    direct_maxima = np.full(label_count, -np.inf, dtype=np.float64)
    np.maximum.at(direct_maxima, raw_labels.ravel(), direct_snr.ravel())
    accepted |= direct_maxima >= config.detection_threshold_sigma
    accepted[0] = False
    accepted_labels = np.flatnonzero(accepted)
    retained = np.isin(raw_labels, accepted_labels)
    component_labels = np.where(retained, raw_labels, 0).astype(
        np.int32,
        copy=False,
    )
    return ResidualMultiscaleIslandDetection(
        combined_snr=_read_only(combined_snr),
        retained_mask=_read_only(np.asarray(retained, dtype=np.bool_)),
        component_labels=_read_only(component_labels),
        component_count=int(np.count_nonzero(np.unique(component_labels) > 0)),
        minimum_island_pixels=minimum_island_pixels,
        reconstruction=reconstruction,
    )

evaluate_residual_atrous(prepared_inputs, plan, *, minimum_support_fraction, accepted_compact_model=None, accepted_compact_mask=None)

Evaluate normalized adjacent B3 smoothings of a compact-clean residual.

Accepted compact emission may be subtracted through a model or excluded through a mask. The returned coefficients are detection provenance; final masks and measurements must continue to use the original residual pixels.

Source code in src/hebog/algorithms/multiscale.py

def evaluate_residual_atrous(
    prepared_inputs: PreparedScaleInputs,
    plan: ResidualAtrousPlan,
    *,
    minimum_support_fraction: float,
    accepted_compact_model: npt.ArrayLike | None = None,
    accepted_compact_mask: npt.ArrayLike | None = None,
) -> ResidualAtrousResult:
    """Evaluate normalized adjacent B3 smoothings of a compact-clean residual.

    Accepted compact emission may be subtracted through a model or excluded
    through a mask. The returned coefficients are detection provenance; final
    masks and measurements must continue to use the original residual pixels.
    """
    if (
        not isfinite(minimum_support_fraction)
        or not 0 < minimum_support_fraction <= 1
    ):
        raise ValueError("minimum_support_fraction must be within (0, 1]")
    if (
        accepted_compact_model is not None
        and accepted_compact_mask is not None
    ):
        raise ValueError("compact emission must be subtracted or excluded")

    residual = np.array(
        prepared_inputs.residual_jy_per_beam,
        dtype=np.float64,
        copy=True,
    )
    input_validity = np.array(
        prepared_inputs.scientifically_valid,
        dtype=np.bool_,
        copy=True,
    )
    if accepted_compact_model is not None:
        compact_model = _as_float_plane(
            accepted_compact_model, name="accepted_compact_model"
        )
        if compact_model.shape != residual.shape:
            raise ValueError(
                "accepted compact model must match residual shape"
            )
        if not np.isfinite(compact_model[input_validity]).all():
            raise ValueError(
                "accepted compact model must be finite where valid"
            )
        np.subtract(
            residual,
            compact_model,
            out=residual,
            where=input_validity,
        )
    if accepted_compact_mask is not None:
        compact_mask = np.asarray(accepted_compact_mask)
        if compact_mask.shape != residual.shape:
            raise ValueError("accepted compact mask must match residual shape")
        if not np.issubdtype(compact_mask.dtype, np.bool_):
            raise TypeError("accepted compact mask must be boolean")
        input_validity &= ~compact_mask
        residual[~input_validity] = 0.0

    current_smoothing = residual
    current_support = np.asarray(input_validity, dtype=np.float64)
    responses: list[ScaleFilterResponse] = []
    for scale in plan.scales:
        next_support = _separable_sparse_smoothing(
            current_support,
            plan.base_kernel,
            dilation_pixels=scale.dilation_pixels,
        )
        numerator = _separable_sparse_smoothing(
            current_smoothing * current_support,
            plan.base_kernel,
            dilation_pixels=scale.dilation_pixels,
        )
        next_smoothing = np.zeros(residual.shape, dtype=np.float64)
        np.divide(
            numerator,
            next_support,
            out=next_smoothing,
            where=next_support > 0,
        )
        scale_validity = input_validity & (
            next_support >= minimum_support_fraction
        )
        coefficient = np.full(residual.shape, np.nan, dtype=np.float64)
        np.subtract(
            current_smoothing,
            next_smoothing,
            out=coefficient,
            where=scale_validity,
        )
        effective_rms = np.full(residual.shape, np.nan, dtype=np.float64)
        propagated_rms = np.zeros(residual.shape, dtype=np.float64)
        np.divide(
            prepared_inputs.rms_jy_per_beam * scale.correlated_noise_gain,
            next_support,
            out=propagated_rms,
            where=next_support > 0,
        )
        np.copyto(effective_rms, propagated_rms, where=scale_validity)
        responses.append(
            ScaleFilterResponse(
                scale_order=scale.scale_order,
                nominal_scale_beam_fwhm=float(2 ** (scale.scale_order - 1)),
                response_jy_per_beam=_read_only(coefficient),
                effective_rms_jy_per_beam=_read_only(effective_rms),
                valid_support_fraction=_read_only(
                    np.asarray(next_support, dtype=np.float64)
                ),
                scientifically_valid=_read_only(
                    np.asarray(scale_validity, dtype=np.bool_)
                ),
            )
        )
        current_smoothing = next_smoothing
        current_support = next_support

    final_validity = input_validity & (
        current_support >= minimum_support_fraction
    )
    reconstructed = np.full(residual.shape, np.nan, dtype=np.float64)
    np.subtract(
        residual,
        current_smoothing,
        out=reconstructed,
        where=final_validity,
    )
    coarse_smoothing = np.full(residual.shape, np.nan, dtype=np.float64)
    np.copyto(coarse_smoothing, current_smoothing, where=final_validity)
    plane_bytes = residual.size * np.dtype(np.float64).itemsize
    retained_plane_count = 4 * len(responses) + 3
    kernel_bytes = plan.base_kernel.nbytes + sum(
        item.response_kernel.nbytes for item in plan.scales
    )
    return ResidualAtrousResult(
        family="residual-b3-atrous",
        responses=tuple(responses),
        reconstructed_signal_jy_per_beam=_read_only(reconstructed),
        coarse_smoothing_jy_per_beam=_read_only(coarse_smoothing),
        scientifically_valid=_read_only(
            np.asarray(final_validity, dtype=np.bool_)
        ),
        convolution_count=plan.convolution_count_per_evaluation,
        temporary_plane_count=plan.temporary_plane_count,
        maximum_workspace_bytes=(
            (plan.temporary_plane_count + retained_plane_count) * plane_bytes
            + kernel_bytes
        ),
    )

evaluate_scale_filter_bank(prepared_inputs, filter_bank, *, minimum_support_fraction)

Evaluate every configured scale from already prepared Phase 2 planes.

Source code in src/hebog/algorithms/multiscale.py

def evaluate_scale_filter_bank(
    prepared_inputs: PreparedScaleInputs,
    filter_bank: ScaleFilterBank,
    *,
    minimum_support_fraction: float,
) -> ScaleFilterBankResult:
    """Evaluate every configured scale from already prepared Phase 2 planes."""
    if (
        not isfinite(minimum_support_fraction)
        or not 0 < minimum_support_fraction <= 1
    ):
        raise ValueError("minimum_support_fraction must be within (0, 1]")
    residual = prepared_inputs.residual_jy_per_beam
    usable_rms = prepared_inputs.rms_jy_per_beam
    input_validity = prepared_inputs.scientifically_valid
    valid_float = np.asarray(input_validity, dtype=np.float64)
    spatial = _requires_spatial_convolution(prepared_inputs)
    # Preserve ordinary variance arithmetic; rescale only when precision or
    # the representable range of squared RMS requires it. This is not a floor.
    maximum_rms = float(np.max(usable_rms, initial=0.0))
    limits = np.finfo(np.float64)
    rescale_variance = spatial or (
        0 < maximum_rms < sqrt(limits.tiny) or maximum_rms > sqrt(limits.max)
    )
    rms_scale = maximum_rms if rescale_variance else 1.0
    smoothing_cache: dict[
        float,
        tuple[npt.NDArray[np.float64], npt.NDArray[np.float64]],
    ] = {}
    responses: list[ScaleFilterResponse] = []

    for scale_filter in filter_bank.filters:
        component_responses: list[npt.NDArray[np.float64]] = []
        component_supports: list[npt.NDArray[np.float64]] = []
        for _, component in scale_filter.components:
            cached = smoothing_cache.get(component.width_beams)
            if cached is None:
                numerator = _convolve_scale_plane(
                    residual,
                    component.values,
                    spatial=spatial,
                )
                support = _convolve_scale_plane(
                    valid_float,
                    component.values,
                    spatial=spatial,
                )
                smoothed = np.zeros(residual.shape, dtype=np.float64)
                np.divide(
                    numerator,
                    support,
                    out=smoothed,
                    where=support > 0,
                )
                cached = (
                    np.asarray(smoothed, dtype=np.float64),
                    np.asarray(support, dtype=np.float64),
                )
                smoothing_cache[component.width_beams] = cached
            component_responses.append(cached[0])
            component_supports.append(cached[1])

        raw_response = np.zeros(residual.shape, dtype=np.float64)
        for (coefficient, _), component_response in zip(
            scale_filter.components,
            component_responses,
            strict=True,
        ):
            raw_response += coefficient * component_response
        calibrated_response = scale_filter.response_calibration * raw_response
        support_fraction = np.minimum.reduce(component_supports)
        scientifically_valid = input_validity & (
            support_fraction >= minimum_support_fraction
        )

        response = np.full(residual.shape, np.nan, dtype=np.float64)
        np.copyto(
            response,
            calibrated_response,
            where=scientifically_valid,
        )
        variance = _convolve_scale_plane(
            np.square(usable_rms / rms_scale) * valid_float,
            np.square(scale_filter.response_kernel),
            spatial=spatial,
        )
        np.maximum(variance, 0.0, out=variance)
        effective_rms = np.full(residual.shape, np.nan, dtype=np.float64)
        noise_ratio = (
            scale_filter.correlated_noise_gain
            / scale_filter.independent_noise_gain
        )
        propagated = np.zeros(residual.shape, dtype=np.float64)
        np.divide(
            np.sqrt(variance) * noise_ratio * rms_scale,
            support_fraction,
            out=propagated,
            where=support_fraction > 0,
        )
        np.copyto(
            effective_rms,
            propagated,
            where=scientifically_valid,
        )
        responses.append(
            ScaleFilterResponse(
                scale_order=scale_filter.scale_order,
                nominal_scale_beam_fwhm=(scale_filter.nominal_scale_beam_fwhm),
                response_jy_per_beam=_read_only(response),
                effective_rms_jy_per_beam=_read_only(effective_rms),
                valid_support_fraction=_read_only(
                    np.asarray(support_fraction, dtype=np.float64)
                ),
                scientifically_valid=_read_only(
                    np.asarray(scientifically_valid, dtype=np.bool_)
                ),
            )
        )

    plane_bytes = residual.size * np.dtype(np.float64).itemsize
    retained_plane_count = 4 * len(responses)
    kernel_bytes = sum(
        item.response_kernel.nbytes
        + sum(component.values.nbytes for _, component in item.components)
        for item in filter_bank.filters
    )
    return ScaleFilterBankResult(
        family=filter_bank.family,
        responses=tuple(responses),
        convolution_count=filter_bank.convolution_count_per_evaluation,
        temporary_plane_count=filter_bank.temporary_plane_count,
        maximum_workspace_bytes=(
            (filter_bank.temporary_plane_count + retained_plane_count)
            * plane_bytes
            + kernel_bytes
        ),
    )

minimum_residual_island_pixels(beam, config)

Return the promoted one-beam residual-island area floor.

Source code in src/hebog/algorithms/multiscale.py

def minimum_residual_island_pixels(
    beam: BeamShapePixels,
    config: ResidualMultiscaleDetectionConfig,
) -> int:
    """Return the promoted one-beam residual-island area floor."""
    return max(
        1,
        ceil(
            config.minimum_island_area_beams
            * _GAUSSIAN_BEAM_AREA_FACTOR
            * beam.major_fwhm_pixels
            * beam.minor_fwhm_pixels
        ),
    )

prepare_scale_filter_inputs(image, valid_pixels, background, rms)

Validate prepared Phase 2 products and form one physical residual.

Source code in src/hebog/algorithms/multiscale.py

def prepare_scale_filter_inputs(
    image: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    background: npt.ArrayLike,
    rms: npt.ArrayLike,
) -> PreparedScaleInputs:
    """Validate prepared Phase 2 products and form one physical residual."""
    image_array = _as_float_plane(image, name="image")
    background_array = _as_float_plane(background, name="background")
    rms_array = _as_float_plane(rms, name="rms")
    validity = np.asarray(valid_pixels)
    if validity.ndim != _IMAGE_DIMENSIONS:
        raise ValueError("valid_pixels must be two-dimensional")
    if not np.issubdtype(validity.dtype, np.bool_):
        raise TypeError("valid_pixels must be a boolean array")
    if not (
        image_array.shape
        == background_array.shape
        == rms_array.shape
        == validity.shape
    ):
        raise ValueError("scale-filter arrays must have the same shape")
    scientifically_valid = (
        np.asarray(validity, dtype=np.bool_)
        & np.isfinite(image_array)
        & np.isfinite(background_array)
        & np.isfinite(rms_array)
        & (rms_array > 0)
    )
    residual = np.zeros(image_array.shape, dtype=np.float64)
    np.subtract(
        image_array,
        background_array,
        out=residual,
        where=scientifically_valid,
    )
    usable_rms = np.zeros(image_array.shape, dtype=np.float64)
    np.copyto(usable_rms, rms_array, where=scientifically_valid)
    return PreparedScaleInputs(
        residual_jy_per_beam=_read_only(residual),
        rms_jy_per_beam=_read_only(usable_rms),
        scientifically_valid=_read_only(scientifically_valid),
    )

reconstruct_denoised_atrous(result, *, significance_sigma)

Reconstruct brightness from coarse emission and signed coefficients.

This estimates image brightness, not a discovery significance or flux. Both coefficient signs are retained symmetrically. The coarse plane is essential: dropping it removes broad emission, while adding the complete detail sum to the original image sharpens noise instead of denoising it. Callers retain the original RMS and discovery thresholds for publication.

Source code in src/hebog/algorithms/multiscale.py

def reconstruct_denoised_atrous(
    result: ResidualAtrousResult,
    *,
    significance_sigma: float,
) -> npt.NDArray[np.float64]:
    """Reconstruct brightness from coarse emission and signed coefficients.

    This estimates image brightness, not a discovery significance or flux.
    Both coefficient signs are retained symmetrically. The coarse plane is
    essential: dropping it removes broad emission, while adding the complete
    detail sum to the original image sharpens noise instead of denoising it.
    Callers retain the original RMS and discovery thresholds for publication.
    """
    if not isfinite(significance_sigma) or significance_sigma < 0.0:
        raise ValueError(
            "shrinkage significance must be finite and non-negative"
        )
    reconstructed = np.array(
        result.coarse_smoothing_jy_per_beam, dtype=np.float64, copy=True
    )
    for response in result.responses:
        coefficient = response.response_jy_per_beam
        significant = (
            response.scientifically_valid
            & np.isfinite(coefficient)
            & (
                np.abs(coefficient)
                >= significance_sigma * response.effective_rms_jy_per_beam
            )
        )
        np.add(
            reconstructed, coefficient, out=reconstructed, where=significant
        )
    return _read_only(reconstructed)

reconstruct_significant_atrous(result, *, detection_sigma, island_sigma, minimum_support_fraction)

Reconstruct positive coefficients persistent at adjacent scales.

Source code in src/hebog/algorithms/multiscale.py

def reconstruct_significant_atrous(
    result: ResidualAtrousResult,
    *,
    detection_sigma: float,
    island_sigma: float,
    minimum_support_fraction: float,
) -> SignificantAtrousReconstruction:
    """Reconstruct positive coefficients persistent at adjacent scales."""
    if not (isfinite(detection_sigma) and isfinite(island_sigma)):
        raise ValueError("à trous thresholds must be finite")
    if detection_sigma <= island_sigma or island_sigma <= 0:
        raise ValueError(
            "thresholds require detection_sigma > island_sigma > 0"
        )
    if (
        not isfinite(minimum_support_fraction)
        or not 0 < minimum_support_fraction <= 1
    ):
        raise ValueError("minimum support fraction must be within (0, 1]")
    if len(result.responses) < _MINIMUM_ATROUS_SCALE_COUNT:
        raise ValueError("à trous reconstruction requires adjacent scales")
    scale_orders = tuple(response.scale_order for response in result.responses)
    if scale_orders != tuple(range(1, len(result.responses) + 1)):
        raise ValueError(
            "à trous responses require canonical adjacent scale orders"
        )
    scale_snrs = calibrated_scale_snrs(
        result.responses,
        minimum_support_fraction=minimum_support_fraction,
    )
    shape = scale_snrs[0].shape
    significant = tuple(
        _read_only(np.asarray(item >= island_sigma, dtype=np.bool_))
        for item in scale_snrs
    )
    adjacent_support = np.logical_or.reduce(
        tuple(
            current & following for current, following in pairwise(significant)
        )
    )
    component_labels, _ = cast(
        tuple[npt.NDArray[np.int32], int],
        label(
            adjacent_support,
            structure=np.ones((3, 3), dtype=np.int8),
        ),
    )
    maximum_snr = np.maximum.reduce(scale_snrs)
    seed_labels = np.unique(component_labels[maximum_snr >= detection_sigma])
    seed_labels = seed_labels[seed_labels > 0]
    retained_support = np.isin(component_labels, seed_labels)
    reconstructed = np.zeros(shape, dtype=np.float64)
    for response, scale_significant in zip(
        result.responses, significant, strict=True
    ):
        add_support = retained_support & scale_significant
        np.add(
            reconstructed,
            response.response_jy_per_beam,
            out=reconstructed,
            where=add_support,
        )
    return SignificantAtrousReconstruction(
        signal_jy_per_beam=_read_only(reconstructed),
        support_mask=_read_only(np.asarray(retained_support, dtype=np.bool_)),
        significant_scale_masks=significant,
    )

residual_atrous_scale_halos_pixels()

Return cumulative radii for the frozen dyadic B3 sequence.

Source code in src/hebog/algorithms/multiscale.py

def residual_atrous_scale_halos_pixels() -> tuple[int, ...]:
    """Return cumulative radii for the frozen dyadic B3 sequence."""
    return tuple(accumulate(2 * dilation for dilation in (1, 2, 4)))

scale_smoothing_halo_pixels(beam, *, width_beams, truncation_sigma)

Return one Gaussian filter radius without allocating its kernel.

Source code in src/hebog/algorithms/multiscale.py

def scale_smoothing_halo_pixels(
    beam: BeamShapePixels,
    *,
    width_beams: float,
    truncation_sigma: float,
) -> int:
    """Return one Gaussian filter radius without allocating its kernel."""
    if not isfinite(width_beams) or width_beams <= 0:
        raise ValueError("scale width must be finite and positive")
    if not isfinite(truncation_sigma) or not (
        _MINIMUM_TRUNCATION_SIGMA
        <= truncation_sigma
        <= _MAXIMUM_TRUNCATION_SIGMA
    ):
        raise ValueError("truncation_sigma must be finite and within [3, 8]")
    major_sigma = beam.major_fwhm_pixels * width_beams * _FWHM_TO_SIGMA
    return ceil(truncation_sigma * major_sigma)

hebog.algorithms.multiscale_tiles

Reviewed stage halos and bounded multiscale tile evaluation.

MultiscaleDetectionTileEvidence dataclass

Core-local fields that precede global topology reconciliation.

Source code in src/hebog/algorithms/multiscale_tiles.py

@dataclass(frozen=True, slots=True)
class MultiscaleDetectionTileEvidence:
    """Core-local fields that precede global topology reconciliation."""

    partition: TilePartition
    direct_snr: npt.NDArray[np.float64]
    matched_maximum_snr: npt.NDArray[np.float64]
    atrous_maximum_snr: npt.NDArray[np.float64]
    atrous_scale_snrs: tuple[npt.NDArray[np.float64], ...]
    significant_scale_masks: tuple[npt.NDArray[np.bool_], ...]
    reconstruction_membership: npt.NDArray[np.bool_]
    reconstruction_seeds: npt.NDArray[np.bool_]
    detection_membership: npt.NDArray[np.bool_]
    detection_seeds: npt.NDArray[np.bool_]

    @property
    def retained_array_bytes(self) -> int:
        """Return exact owned ndarray payload retained for local topology."""
        return sum(
            array.nbytes
            for array in (
                self.direct_snr,
                self.matched_maximum_snr,
                self.atrous_maximum_snr,
                *self.atrous_scale_snrs,
                *self.significant_scale_masks,
                self.reconstruction_membership,
                self.reconstruction_seeds,
                self.detection_membership,
                self.detection_seeds,
            )
        )

retained_array_bytes property

Return exact owned ndarray payload retained for local topology.

MultiscaleFilterTileResult dataclass

Core-only multiscale filter evidence from one bounded halo read.

Workspace evidence describes the complete read evaluation. Every array in the returned scientific records is an owned immutable core copy, so no result retains an image-sized or halo-sized input buffer.

Source code in src/hebog/algorithms/multiscale_tiles.py

@dataclass(frozen=True, slots=True)
class MultiscaleFilterTileResult:
    """Core-only multiscale filter evidence from one bounded halo read.

    Workspace evidence describes the complete read evaluation. Every array in
    the returned scientific records is an owned immutable core copy, so no
    result retains an image-sized or halo-sized input buffer.
    """

    partition: TilePartition
    prepared_inputs: PreparedScaleInputs
    matched_filter: ScaleFilterBankResult
    atrous_result: ResidualAtrousResult
    read_pixel_count: int
    maximum_filter_evaluation_bytes: int

    @property
    def retained_array_bytes(self) -> int:
        """Return exact owned ndarray payload retained by this core result."""
        return (
            _prepared_array_bytes(self.prepared_inputs)
            + _scale_response_array_bytes(self.matched_filter.responses)
            + _residual_atrous_array_bytes(self.atrous_result)
        )

retained_array_bytes property

Return exact owned ndarray payload retained by this core result.

derive_multiscale_detection_tile_evidence(result, config)

Derive local masks and seeds before bounded global reconciliation.

No connected component is accepted here because support may cross any number of tile boundaries. Adjacent-scale membership and seed values are point-local; the stage reconciles their compact side/corner summaries.

Source code in src/hebog/algorithms/multiscale_tiles.py

def derive_multiscale_detection_tile_evidence(
    result: MultiscaleFilterTileResult,
    config: ResidualMultiscaleDetectionConfig,
) -> MultiscaleDetectionTileEvidence:
    """Derive local masks and seeds before bounded global reconciliation.

    No connected component is accepted here because support may cross any
    number of tile boundaries. Adjacent-scale membership and seed values are
    point-local; the stage reconciles their compact side/corner summaries.
    """
    prepared = result.prepared_inputs
    shape = prepared.residual_jy_per_beam.shape
    minimum_support = config.minimum_scale_support_fraction
    matched_scale_snrs = calibrated_scale_snrs(
        result.matched_filter.responses,
        minimum_support_fraction=minimum_support,
    )
    atrous_scale_snrs = calibrated_scale_snrs(
        result.atrous_result.responses,
        minimum_support_fraction=minimum_support,
    )
    matched_maximum = np.maximum.reduce(matched_scale_snrs)
    atrous_maximum = np.maximum.reduce(atrous_scale_snrs)
    direct_snr = np.full(shape, -np.inf, dtype=np.float64)
    np.divide(
        prepared.residual_jy_per_beam,
        prepared.rms_jy_per_beam,
        out=direct_snr,
        where=prepared.scientifically_valid,
    )
    significant = tuple(
        np.asarray(
            scale_snr >= config.island_threshold_sigma,
            dtype=np.bool_,
        )
        for scale_snr in atrous_scale_snrs
    )
    reconstruction_membership = np.logical_or.reduce(
        tuple(
            current & following for current, following in pairwise(significant)
        )
    )
    reconstruction_seeds = reconstruction_membership & (
        atrous_maximum >= config.detection_threshold_sigma
    )
    detection_membership = prepared.scientifically_valid & (
        direct_snr >= config.island_threshold_sigma
    )
    combined_seed_snr = np.maximum(matched_maximum, direct_snr)
    np.maximum(
        combined_seed_snr,
        atrous_maximum,
        out=combined_seed_snr,
        where=reconstruction_membership,
    )
    detection_seeds = detection_membership & (
        combined_seed_snr >= config.detection_threshold_sigma
    )
    return MultiscaleDetectionTileEvidence(
        partition=result.partition,
        direct_snr=_immutable_array(direct_snr),
        matched_maximum_snr=_immutable_array(matched_maximum),
        atrous_maximum_snr=_immutable_array(atrous_maximum),
        atrous_scale_snrs=atrous_scale_snrs,
        significant_scale_masks=tuple(
            _immutable_array(mask) for mask in significant
        ),
        reconstruction_membership=_immutable_array(reconstruction_membership),
        reconstruction_seeds=_immutable_array(reconstruction_seeds),
        detection_membership=_immutable_array(detection_membership),
        detection_seeds=_immutable_array(detection_seeds),
    )

evaluate_multiscale_filter_tile(prepared_read, *, partition, image_shape_yx, beam, minimum_support_fraction)

Evaluate the promoted filters on one read and return its owned core.

This is the bounded local-neighbourhood seam for multiscale detection. Connected support is deliberately not labelled here: callers reconcile core labels through the existing bounded edge and corner summaries.

Source code in src/hebog/algorithms/multiscale_tiles.py

def evaluate_multiscale_filter_tile(
    prepared_read: PreparedScaleInputs,
    *,
    partition: TilePartition,
    image_shape_yx: tuple[int, int],
    beam: BeamShapePixels,
    minimum_support_fraction: float,
) -> MultiscaleFilterTileResult:
    """Evaluate the promoted filters on one read and return its owned core.

    This is the bounded local-neighbourhood seam for multiscale detection.
    Connected support is deliberately not labelled here: callers reconcile core
    labels through the existing bounded edge and corner summaries.
    """
    read_shape_yx = prepared_read.residual_jy_per_beam.shape
    if not (
        prepared_read.rms_jy_per_beam.shape == read_shape_yx
        and prepared_read.scientifically_valid.shape == read_shape_yx
    ):
        raise ValueError(
            "prepared filter-read arrays must have the same shape"
        )
    required_halo = scale_filter_halo_pixels(beam)
    _validate_filter_read(
        partition,
        image_shape_yx=image_shape_yx,
        required_halo_pixels=required_halo,
        read_shape_yx=read_shape_yx,
    )
    prepared_core = PreparedScaleInputs(
        residual_jy_per_beam=_core_copy(
            prepared_read.residual_jy_per_beam,
            partition,
        ),
        rms_jy_per_beam=_core_copy(
            prepared_read.rms_jy_per_beam,
            partition,
        ),
        scientifically_valid=_core_copy(
            prepared_read.scientifically_valid,
            partition,
        ),
    )
    prepared_read_bytes = _prepared_array_bytes(prepared_read)
    prepared_core_bytes = _prepared_array_bytes(prepared_core)
    matched_read = evaluate_scale_filter_bank(
        prepared_read,
        build_scale_filter_bank(
            beam,
            family="beam-aware-matched-filter",
            scales=_FROZEN_SCALES,
            truncation_sigma=_FROZEN_FILTER_TRUNCATION_SIGMA,
            noise_correlation=beam,
        ),
        minimum_support_fraction=minimum_support_fraction,
    )
    maximum_filter_evaluation_bytes = (
        prepared_read_bytes
        + prepared_core_bytes
        + matched_read.maximum_workspace_bytes
    )
    matched_core = ScaleFilterBankResult(
        family=matched_read.family,
        responses=tuple(
            _core_response(response, partition)
            for response in matched_read.responses
        ),
        convolution_count=matched_read.convolution_count,
        temporary_plane_count=matched_read.temporary_plane_count,
        maximum_workspace_bytes=matched_read.maximum_workspace_bytes,
    )
    maximum_filter_evaluation_bytes = max(
        maximum_filter_evaluation_bytes,
        prepared_read_bytes
        + prepared_core_bytes
        + _scale_response_array_bytes(matched_read.responses)
        + _scale_response_array_bytes(matched_core.responses),
    )
    del matched_read
    atrous_read = evaluate_residual_atrous(
        prepared_read,
        build_residual_atrous_plan(beam, noise_correlation=beam),
        minimum_support_fraction=minimum_support_fraction,
    )
    matched_core_bytes = _scale_response_array_bytes(matched_core.responses)
    maximum_filter_evaluation_bytes = max(
        maximum_filter_evaluation_bytes,
        prepared_read_bytes
        + prepared_core_bytes
        + matched_core_bytes
        + atrous_read.maximum_workspace_bytes,
    )
    del prepared_read
    atrous_core = ResidualAtrousResult(
        family="residual-b3-atrous",
        responses=tuple(
            _core_response(response, partition)
            for response in atrous_read.responses
        ),
        reconstructed_signal_jy_per_beam=_core_copy(
            atrous_read.reconstructed_signal_jy_per_beam,
            partition,
        ),
        coarse_smoothing_jy_per_beam=_core_copy(
            atrous_read.coarse_smoothing_jy_per_beam,
            partition,
        ),
        scientifically_valid=_core_copy(
            atrous_read.scientifically_valid,
            partition,
        ),
        convolution_count=atrous_read.convolution_count,
        temporary_plane_count=atrous_read.temporary_plane_count,
        maximum_workspace_bytes=atrous_read.maximum_workspace_bytes,
    )
    maximum_filter_evaluation_bytes = max(
        maximum_filter_evaluation_bytes,
        prepared_core_bytes
        + matched_core_bytes
        + _residual_atrous_array_bytes(atrous_read)
        + _residual_atrous_array_bytes(atrous_core),
    )
    return MultiscaleFilterTileResult(
        partition=partition,
        prepared_inputs=prepared_core,
        matched_filter=matched_core,
        atrous_result=atrous_core,
        read_pixel_count=partition.read_bounds.shape_yx[0]
        * partition.read_bounds.shape_yx[1],
        maximum_filter_evaluation_bytes=maximum_filter_evaluation_bytes,
    )

scale_filter_halo_pixels(beam)

Return the widest frozen multiscale filter halo without allocation.

Source code in src/hebog/algorithms/multiscale_tiles.py

def scale_filter_halo_pixels(beam: BeamShapePixels) -> int:
    """Return the widest frozen multiscale filter halo without allocation."""
    return max(
        *_scale_filter_halos_pixels(beam),
        *residual_atrous_scale_halos_pixels(),
    )

segment_association_halo_pixels(beam)

Return the frozen residual/reconstruction grouping radius.

Source code in src/hebog/algorithms/multiscale_tiles.py

def segment_association_halo_pixels(beam: BeamShapePixels) -> int:
    """Return the frozen residual/reconstruction grouping radius."""
    return ceil(_FROZEN_SEGMENT_ASSOCIATION_BEAMS * beam.major_fwhm_pixels)

hebog.stages.multiscale

Bounded multiscale execution and atomic product publication.

MultiscaleStageConfig dataclass

Reviewed residual science and coarse executor-batch limit.

Source code in src/hebog/stages/multiscale.py

@dataclass(frozen=True, slots=True)
class MultiscaleStageConfig:
    """Reviewed residual science and coarse executor-batch limit."""

    beam: BeamShapePixels
    detection: ResidualMultiscaleDetectionConfig
    maximum_tiles_per_batch: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before stage products are initialized."""
        if (
            isinstance(self.maximum_tiles_per_batch, bool)
            or not isinstance(self.maximum_tiles_per_batch, Integral)
            or self.maximum_tiles_per_batch < 1
        ):
            raise ValueError(
                "maximum_tiles_per_batch must be a positive integer"
            )

__post_init__()

Reject an unbounded task before stage products are initialized.

Source code in src/hebog/stages/multiscale.py

def __post_init__(self) -> None:
    """Reject an unbounded task before stage products are initialized."""
    if (
        isinstance(self.maximum_tiles_per_batch, bool)
        or not isinstance(self.maximum_tiles_per_batch, Integral)
        or self.maximum_tiles_per_batch < 1
    ):
        raise ValueError(
            "maximum_tiles_per_batch must be a positive integer"
        )

MultiscaleStageResult dataclass

Published products, stable topology, and scalar execution evidence.

Source code in src/hebog/stages/multiscale.py

@dataclass(frozen=True, slots=True)
class MultiscaleStageResult:
    """Published products, stable topology, and scalar execution evidence."""

    generation: ProductGenerationManifest
    detection_islands: tuple[DetectedIsland, ...]
    reconstruction_islands: tuple[DetectedIsland, ...]
    scale_islands_by_order: tuple[tuple[DetectedIsland, ...], ...]
    scale_nominal_beam_fwhms: tuple[float, ...]
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_batch_partition_count: int
    maximum_read_pixel_count: int
    maximum_workspace_bytes: int
    maximum_retained_array_bytes: int
    maximum_worker_bytes: int
    topology_summary_count: int
    scale_summary_count: int
    boundary_summary_array_bytes: int
    maximum_task_summary_array_bytes: int
    published_product_shard_count: int
    maximum_task_product_shard_count: int
    reconciliation_round_count: int

multiscale_product_names()

Return the canonical accepted multiscale product set.

Source code in src/hebog/stages/multiscale.py

def multiscale_product_names() -> tuple[str, ...]:
    """Return the canonical accepted multiscale product set."""
    return _MULTISCALE_PRODUCT_NAMES

run_multiscale_stage(source, background_rms_source, manifest, *, config, executor, sink)

Reconcile bounded multiscale topology and publish accepted products.

The first pass returns only compact side/corner summaries. After global reconciliation, workers recompute the same bounded filters, apply stable label mappings, and write accepted core products. This deliberate second read avoids an image-sized persisted filter-response bank. No scientific array is returned through the executor.

Source code in src/hebog/stages/multiscale.py

def run_multiscale_stage(  # noqa: PLR0913
    source: WindowReadable,
    background_rms_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: MultiscaleStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
) -> MultiscaleStageResult:
    """Reconcile bounded multiscale topology and publish accepted products.

    The first pass returns only compact side/corner summaries. After global
    reconciliation, workers recompute the same bounded filters, apply stable
    label mappings, and write accepted core products. This deliberate second
    read avoids an image-sized persisted filter-response bank. No scientific
    array is returned through the executor.
    """
    _validate_stage_inputs(
        background_rms_source,
        manifest,
        config,
        sink,
    )
    partition_batches = _partition_batches(
        manifest,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    inputs = _TileInputs(
        source=source,
        background_rms_source=background_rms_source,
        image_shape_yx=manifest.image_shape_yx,
        beam=config.beam,
        detection=config.detection,
    )
    scan = partial(_scan_topology_batch, inputs=inputs)
    tile_reads = tuple(partition.read_bounds for partition in manifest.tiles)
    topology_results = tuple(
        executor.map_batches(
            scan,
            partition_batches,
            requirement=read_requirement(
                _FILTER_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not topology_results:
        raise ValueError("executor returned no multiscale topology results")
    reconstruction = reconcile_candidate_tiles(
        manifest,
        tuple(
            summary
            for result in topology_results
            for summary in result.reconstruction_summaries
        ),
    )
    detection_candidates = reconcile_candidate_tiles(
        manifest,
        tuple(
            summary
            for result in topology_results
            for summary in result.detection_summaries
        ),
    )
    minimum_pixels = minimum_residual_island_pixels(
        config.beam,
        config.detection,
    )
    detection_islands = tuple(
        island
        for island in detection_candidates.islands
        if island.pixel_count >= minimum_pixels
        or island.peak_signal_to_noise
        >= config.detection.detection_threshold_sigma
    )
    accepted_detection_labels = frozenset(
        island.global_label for island in detection_islands
    )
    publication_requests = tuple(
        _PublicationTileRequest(
            partition=partition,
            reconstruction_mapping=reconstruction.mapping_for_tile(
                partition.tile_id
            ),
            detection_mapping=_retain_mapping_labels(
                detection_candidates.mapping_for_tile(partition.tile_id),
                accepted_detection_labels,
            ),
        )
        for partition in manifest.tiles
    )
    for product_name in _MULTISCALE_PRODUCT_NAMES:
        sink.initialize_product(
            product_name=product_name,
            dtype=_product_dtype(product_name),
        )
    publication_batches = _publication_batches(
        publication_requests,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    publish = partial(_publish_batch, inputs=inputs, sink=sink)
    publication_results = tuple(
        executor.map_batches(
            publish,
            publication_batches,
            requirement=read_requirement(
                _FILTER_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not publication_results:
        raise ValueError("executor returned no multiscale publication results")
    reconciled_scales = tuple(
        reconcile_candidate_tiles(
            manifest,
            tuple(
                summary
                for result in publication_results
                for summary in result.scale_summaries_by_order[scale_index]
            ),
        )
        for scale_index in range(len(_SCALE_ORDERS))
    )
    scale_islands = tuple(
        reconciled.islands for reconciled in reconciled_scales
    )
    significant_chunks_by_tile: dict[str, dict[str, ProductChunk]] = {}
    for result in publication_results:
        for chunk in result.product_chunks:
            if chunk.product_name.endswith("-significant"):
                significant_chunks_by_tile.setdefault(chunk.tile_id, {})[
                    chunk.product_name
                ] = chunk
    scale_label_batches = _scale_label_batches(
        tuple(
            _ScaleLabelRequest(
                partition=partition,
                significant_chunks=tuple(
                    significant_chunks_by_tile[partition.tile_id][
                        f"scale-{order}-significant"
                    ]
                    for order in _SCALE_ORDERS
                ),
                global_by_local_label_by_order=tuple(
                    tuple(
                        zip(
                            mapping.local_labels,
                            mapping.global_labels,
                            strict=True,
                        )
                    )
                    for mapping in (
                        reconciled.mapping_for_tile(partition.tile_id)
                        for reconciled in reconciled_scales
                    )
                ),
            )
            for partition in manifest.tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    scale_label_results = tuple(
        executor.map_batches(
            partial(
                _publish_scale_labels,
                sink=sink,
                image_shape_yx=manifest.image_shape_yx,
            ),
            scale_label_batches,
            requirement=read_requirement(
                _SCALE_LABEL_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scale_label_results:
        raise ValueError("executor returned no scale label results")
    generation = sink.publish_generation(
        product_names=_MULTISCALE_PRODUCT_NAMES,
        chunks=(
            chunk
            for result in (*publication_results, *scale_label_results)
            for chunk in result.product_chunks
        ),
    )
    all_results = (*topology_results, *publication_results)
    batch_counts = (
        len(partition_batches),
        len(publication_batches),
        len(scale_label_batches),
    )
    return MultiscaleStageResult(
        generation=generation,
        detection_islands=detection_islands,
        reconstruction_islands=reconstruction.islands,
        scale_islands_by_order=scale_islands,
        scale_nominal_beam_fwhms=publication_results[
            0
        ].scale_nominal_beam_fwhms,
        partition_count=len(manifest.tiles),
        executor_task_count=sum(batch_counts),
        maximum_graph_width=max(batch_counts),
        maximum_batch_partition_count=max(
            len(batch.partitions) for batch in partition_batches
        ),
        maximum_read_pixel_count=max(
            result.maximum_read_pixel_count for result in all_results
        ),
        maximum_workspace_bytes=max(
            result.maximum_workspace_bytes for result in all_results
        ),
        maximum_retained_array_bytes=max(
            result.maximum_retained_array_bytes for result in all_results
        ),
        maximum_worker_bytes=max(
            result.maximum_worker_bytes for result in all_results
        ),
        topology_summary_count=2 * len(manifest.tiles),
        scale_summary_count=len(_SCALE_ORDERS) * len(manifest.tiles),
        boundary_summary_array_bytes=sum(
            result.summary_array_bytes for result in all_results
        ),
        maximum_task_summary_array_bytes=max(
            result.summary_array_bytes for result in all_results
        ),
        published_product_shard_count=len(generation.chunks),
        maximum_task_product_shard_count=max(
            len(result.product_chunks) for result in publication_results
        ),
        reconciliation_round_count=max(
            reconstruction.reduction_round_count,
            detection_candidates.reduction_round_count,
        ),
    )

hebog.stages.support

The global support reduction the tile-native support pass depends on.

ADR-008's pass C needs the adjacent-scale persistence of the multiscale features, which decides which recovered support may stay published, and no bounded halo can supply it. It is reconciled here from compact per-core summaries and written back as owned cores, so the composition reads it by window instead of computing it over whole planes.

SupportTopologyStageConfig dataclass

Reviewed scale orders and the coarse executor-batch limit.

Source code in src/hebog/stages/support.py

@dataclass(frozen=True, slots=True)
class SupportTopologyStageConfig:
    """Reviewed scale orders and the coarse executor-batch limit."""

    scale_orders: tuple[int, ...]
    maximum_tiles_per_batch: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before stage products are initialized."""
        if (
            isinstance(self.maximum_tiles_per_batch, bool)
            or not isinstance(self.maximum_tiles_per_batch, Integral)
            or self.maximum_tiles_per_batch < 1
        ):
            raise ValueError(
                "maximum_tiles_per_batch must be a positive integer"
            )
        if len(self.scale_orders) < _MINIMUM_PERSISTENT_SCALE_COUNT:
            raise ValueError(
                "adjacent-scale persistence needs at least two scale orders"
            )
        if tuple(sorted(set(self.scale_orders))) != tuple(self.scale_orders):
            raise ValueError("scale orders must be unique and ascending")

__post_init__()

Reject an unbounded task before stage products are initialized.

Source code in src/hebog/stages/support.py

def __post_init__(self) -> None:
    """Reject an unbounded task before stage products are initialized."""
    if (
        isinstance(self.maximum_tiles_per_batch, bool)
        or not isinstance(self.maximum_tiles_per_batch, Integral)
        or self.maximum_tiles_per_batch < 1
    ):
        raise ValueError(
            "maximum_tiles_per_batch must be a positive integer"
        )
    if len(self.scale_orders) < _MINIMUM_PERSISTENT_SCALE_COUNT:
        raise ValueError(
            "adjacent-scale persistence needs at least two scale orders"
        )
    if tuple(sorted(set(self.scale_orders))) != tuple(self.scale_orders):
        raise ValueError("scale orders must be unique and ascending")

SupportTopologyStageResult dataclass

The published persistence plane and scalar execution evidence.

Source code in src/hebog/stages/support.py

@dataclass(frozen=True, slots=True)
class SupportTopologyStageResult:
    """The published persistence plane and scalar execution evidence."""

    generation: ProductGenerationManifest
    persistent_detection_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_read_pixel_count: int
    boundary_summary_array_bytes: int
    reconciliation_round_count: int

run_support_topology_stage(detection_source, manifest, *, config, executor, sink)

Reconcile adjacent-scale persistence and publish its plane.

The first round returns only compact per-scale core summaries and the adjacent-scale label overlaps each core observes. After reconciliation the second round labels the same core masks again and writes the scale features corroborated at an adjacent scale. No scientific array is returned through the executor.

Source code in src/hebog/stages/support.py

def run_support_topology_stage(
    detection_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: SupportTopologyStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
) -> SupportTopologyStageResult:
    """Reconcile adjacent-scale persistence and publish its plane.

    The first round returns only compact per-scale core summaries and the
    adjacent-scale label overlaps each core observes. After reconciliation the
    second round labels the same core masks again and writes the scale
    features corroborated at an adjacent scale. No scientific array is
    returned through the executor.
    """
    _validate_stage_inputs(detection_source, manifest, config, sink)
    scan = partial(
        _scan_topology_batch,
        detection_source=detection_source,
        image_shape_yx=manifest.image_shape_yx,
        scale_orders=config.scale_orders,
    )
    partition_batches = _partition_batches(
        manifest,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    tile_reads = tuple(partition.read_bounds for partition in manifest.tiles)
    topology_results = tuple(
        executor.map_batches(
            scan,
            partition_batches,
            requirement=read_requirement(
                _TOPOLOGY_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not topology_results:
        raise ValueError("executor returned no support topology results")
    tiles = tuple(tile for result in topology_results for tile in result.tiles)
    scale_reconciliations = tuple(
        reconcile_candidate_tiles(
            manifest,
            tuple(tile.scale_summaries[index] for tile in tiles),
        )
        for index in range(len(config.scale_orders))
    )
    persistent_nodes = _persistent_global_nodes(tiles, scale_reconciliations)
    sink.initialize_product(
        product_name="persistent-support",
        dtype=np.dtype(np.bool_),
    )
    publication_batches = _publication_batches(
        _publication_requests(
            tiles,
            scale_reconciliations,
            persistent_nodes,
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    publish = partial(
        _publish_batch,
        detection_source=detection_source,
        sink=sink,
        image_shape_yx=manifest.image_shape_yx,
        scale_orders=config.scale_orders,
    )
    publication_results = tuple(
        executor.map_batches(
            publish,
            publication_batches,
            requirement=read_requirement(
                _PUBLICATION_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not publication_results:
        raise ValueError("executor returned no support publication results")
    generation = sink.publish_generation(
        product_names=_SUPPORT_PRODUCT_NAMES,
        chunks=(
            chunk
            for result in publication_results
            for chunk in result.product_chunks
        ),
    )
    return SupportTopologyStageResult(
        generation=generation,
        persistent_detection_count=len(persistent_nodes),
        partition_count=len(manifest.tiles),
        executor_task_count=len(partition_batches) + len(publication_batches),
        maximum_graph_width=max(
            len(partition_batches),
            len(publication_batches),
        ),
        maximum_read_pixel_count=max(
            (
                *(
                    result.maximum_read_pixel_count
                    for result in topology_results
                ),
                *(
                    result.maximum_read_pixel_count
                    for result in publication_results
                ),
            )
        ),
        boundary_summary_array_bytes=sum(
            result.summary_array_bytes for result in topology_results
        ),
        reconciliation_round_count=max(
            reconciliation.reduction_round_count
            for reconciliation in scale_reconciliations
        ),
    )

support_product_names()

Return the canonical published support product set.

Source code in src/hebog/stages/support.py

def support_product_names() -> tuple[str, ...]:
    """Return the canonical published support product set."""
    return _SUPPORT_PRODUCT_NAMES

Published support

hebog.algorithms.extended_measurement

Pure measurement kernels for irregular extended emission.

DetectedSegmentPosition dataclass

Flux centroid and peak tied to one accepted detection segment.

Source code in src/hebog/algorithms/extended_measurement.py

@dataclass(frozen=True, slots=True)
class DetectedSegmentPosition:
    """Flux centroid and peak tied to one accepted detection segment."""

    available: bool
    centroid_xy: tuple[float, float] | None
    peak_position_xy: tuple[int, int] | None
    support_pixel_count: int
    integrated_weight: float
    unavailable_reason: SegmentPositionUnavailableReason | None
    weighting: Literal["signed-original", "denoised"] = "signed-original"

SegmentWindow dataclass

Where one measured window sits inside its image plane.

Measuring a segment over the whole plane costs image size for every segment. A window lets a caller pass only the pixels around one segment while keeping every reported position in the plane's pixel frame.

Source code in src/hebog/algorithms/extended_measurement.py

@dataclass(frozen=True, slots=True)
class SegmentWindow:
    """Where one measured window sits inside its image plane.

    Measuring a segment over the whole plane costs image size for every
    segment. A window lets a caller pass only the pixels around one segment
    while keeping every reported position in the plane's pixel frame.
    """

    origin_yx: tuple[int, int]
    plane_shape_yx: tuple[int, int]

    def require_holds(self, shape_yx: tuple[int, int]) -> None:
        """Reject a window that does not fit its plane.

        Raises:
            ValueError: If the origin is negative or the window leaves the
                plane.
        """
        if min(self.origin_yx) < 0:
            raise ValueError("segment window origin must be non-negative")
        if any(
            origin + extent > plane
            for origin, extent, plane in zip(
                self.origin_yx, shape_yx, self.plane_shape_yx, strict=True
            )
        ):
            raise ValueError("segment window must stay inside its plane")

require_holds(shape_yx)

Reject a window that does not fit its plane.

Raises:

Type	Description
ValueError	If the origin is negative or the window leaves the plane.

Source code in src/hebog/algorithms/extended_measurement.py

def require_holds(self, shape_yx: tuple[int, int]) -> None:
    """Reject a window that does not fit its plane.

    Raises:
        ValueError: If the origin is negative or the window leaves the
            plane.
    """
    if min(self.origin_yx) < 0:
        raise ValueError("segment window origin must be non-negative")
    if any(
        origin + extent > plane
        for origin, extent, plane in zip(
            self.origin_yx, shape_yx, self.plane_shape_yx, strict=True
        )
    ):
        raise ValueError("segment window must stay inside its plane")

apply_owner_restores(original_labels, refined_labels, restored_owners)

Restore the original support of every owner cleanup splits or removes.

The decision is one boolean per owner, so a tile applies it to its own core without seeing the owner's whole window.

Source code in src/hebog/algorithms/extended_measurement.py

def apply_owner_restores(
    original_labels: npt.ArrayLike,
    refined_labels: npt.ArrayLike,
    restored_owners: Collection[int],
) -> npt.NDArray[np.int32]:
    """Restore the original support of every owner cleanup splits or removes.

    The decision is one boolean per owner, so a tile applies it to its own
    core without seeing the owner's whole window.
    """
    original = np.asarray(original_labels)
    refined = np.asarray(refined_labels, dtype=np.int32).copy()
    if not restored_owners:
        return refined
    restored = np.isin(
        original,
        np.asarray(sorted(restored_owners), dtype=original.dtype),
    )
    np.copyto(refined, original.astype(np.int32, copy=False), where=restored)
    return refined

assign_connected_source_support(source_labels, persistent, component)

Assign one connected support component to its nearest source seeds.

Every array covers the window that holds this component. Candidates are the component's persistent pixels that no source seeds; the nearest immutable source pixel owns each, and an exact distance tie goes to the smaller source label, whose order follows the canonical source IDs. source_labels is written in place and returned.

Source code in src/hebog/algorithms/extended_measurement.py

def assign_connected_source_support(
    source_labels: npt.NDArray[np.int32],
    persistent: npt.NDArray[np.bool_],
    component: npt.NDArray[np.bool_],
) -> npt.NDArray[np.int32]:
    """Assign one connected support component to its nearest source seeds.

    Every array covers the window that holds this component. Candidates are
    the component's persistent pixels that no source seeds; the nearest
    immutable source pixel owns each, and an exact distance tie goes to the
    smaller source label, whose order follows the canonical source IDs.
    ``source_labels`` is written in place and returned.
    """
    seed_points = np.column_stack(np.nonzero(component & (source_labels > 0)))
    candidate_points = np.column_stack(
        np.nonzero(component & persistent & (source_labels == 0))
    )
    if not seed_points.size or not candidate_points.size:
        return source_labels
    points = np.asarray(candidate_points, dtype=np.int64)
    owners = nearest_source_seed_labels(
        np.asarray(seed_points, dtype=np.int64),
        np.asarray(
            source_labels[seed_points[:, 0], seed_points[:, 1]],
            dtype=np.int64,
        ),
        points,
    )
    source_labels[points[:, 0], points[:, 1]] = owners.astype(
        np.int32, copy=False
    )
    return source_labels

assign_nearest_seed_support(seed_labels, significant_support, valid_pixels, *, maximum_distance_pixels)

Give significant support the nearest seed pixel of its own component.

Positive input labels are the seeds and keep their pixels. A valid significant pixel takes the label of the nearest seed pixel, by straight-line distance, among the seeds of its own eight-connected component of seeds and support, when that seed lies within maximum_distance_pixels. Equal distances use the label whose first row-major seed pixel comes first. The components are labelled here, so the planes must hold every pixel of the support being partitioned.

Source code in src/hebog/algorithms/extended_measurement.py

def assign_nearest_seed_support(
    seed_labels: npt.ArrayLike,
    significant_support: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    *,
    maximum_distance_pixels: float,
) -> npt.NDArray[np.int32]:
    """Give significant support the nearest seed pixel of its own component.

    Positive input labels are the seeds and keep their pixels. A valid
    significant pixel takes the label of the nearest seed pixel, by
    straight-line distance, among the seeds of its own eight-connected
    component of seeds and support, when that seed lies within
    ``maximum_distance_pixels``. Equal distances use the label whose first
    row-major seed pixel comes first. The components are labelled here, so
    the planes must hold every pixel of the support being partitioned.
    """
    labels, significant, valid = _validated_seed_planes(
        seed_labels,
        significant_support,
        valid_pixels,
    )
    if (
        isinstance(maximum_distance_pixels, bool)
        or not isfinite(maximum_distance_pixels)
        or maximum_distance_pixels < 0.0
    ):
        raise ValueError(
            "maximum seed distance must be finite and non-negative"
        )
    output = np.asarray(labels, dtype=np.int32).copy()
    seed_points = np.column_stack(np.nonzero(labels > 0))
    candidate_points = np.column_stack(
        np.nonzero(significant & valid & (labels == 0))
    )
    if not seed_points.size or not candidate_points.size:
        return output
    connected_labels, _ = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(
            ((labels > 0) | significant) & valid,
            structure=np.ones((3, 3), dtype=np.int8),
        ),
    )
    seed_ranks, labels_by_rank = _canonical_seed_ranks(labels, None)
    seed_components = connected_labels[seed_points[:, 0], seed_points[:, 1]]
    candidate_components = connected_labels[
        candidate_points[:, 0], candidate_points[:, 1]
    ]
    for component in sorted({int(value) for value in seed_components}):
        local_seed = seed_components == component
        local_candidate = candidate_components == component
        if not np.any(local_candidate):
            continue
        points = np.asarray(candidate_points[local_candidate], dtype=np.int64)
        distances, owner_ranks = _nearest_canonical_seed_ranks(
            cast(_NearestSeedTree, cKDTree(seed_points[local_seed])),
            points,
            seed_ranks[local_seed],
        )
        eligible = distances <= maximum_distance_pixels
        eligible_points = points[eligible]
        output[eligible_points[:, 0], eligible_points[:, 1]] = labels_by_rank[
            owner_ranks[eligible]
        ].astype(np.int32, copy=False)
    return output

assign_persistent_source_support(source_labels, persistent_multiscale_support, valid_pixels)

Assign connected persistent support to canonical source owners once.

Source labels are the already accepted catalogue-source partition. Exact persistent support connected to at least one source seed extends that source's measurement seed. Where support is connected to multiple sources, the nearest immutable source pixel owns it; exact distance ties use the smaller source label, whose ordering is derived from canonical source IDs. Disconnected, invalid, and non-persistent pixels cannot enter the result.

Source code in src/hebog/algorithms/extended_measurement.py

def assign_persistent_source_support(
    source_labels: npt.ArrayLike,
    persistent_multiscale_support: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
) -> npt.NDArray[np.int32]:
    """Assign connected persistent support to canonical source owners once.

    Source labels are the already accepted catalogue-source partition. Exact
    persistent support connected to at least one source seed extends that
    source's measurement seed. Where support is connected to multiple sources,
    the nearest immutable source pixel owns it; exact distance ties use the
    smaller source label, whose ordering is derived from canonical source IDs.
    Disconnected, invalid, and non-persistent pixels cannot enter the result.
    """
    labels = _segment_label_plane(source_labels)
    persistent = np.asarray(persistent_multiscale_support)
    valid = np.asarray(valid_pixels)
    if (
        persistent.ndim != _IMAGE_DIMENSIONS
        or persistent.shape != labels.shape
        or persistent.dtype != np.bool_
    ):
        raise ValueError(
            "source labels and persistent multiscale support must be aligned "
            "boolean two-dimensional planes"
        )
    if (
        valid.ndim != _IMAGE_DIMENSIONS
        or valid.shape != labels.shape
        or valid.dtype != np.bool_
    ):
        raise ValueError(
            "source labels and valid pixels must be aligned boolean "
            "two-dimensional planes"
        )
    if np.any((labels > 0) & ~valid):
        raise ValueError("source seed pixels must be scientifically valid")
    if np.any(persistent & ~valid):
        raise ValueError(
            "persistent source support must be scientifically valid"
        )
    output = np.asarray(labels, dtype=np.int32).copy()
    if not np.any(labels > 0) or not np.any(persistent & (labels == 0)):
        return output
    connected_labels, count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(
            (labels > 0) | persistent,
            structure=np.ones((3, 3), dtype=np.int8),
        ),
    )
    for component in range(1, count + 1):
        assign_connected_source_support(
            output,
            persistent,
            np.asarray(connected_labels == component),
        )
    return output

assign_seeded_multiscale_support(component_labels, significant_multiscale_support, valid_pixels, *, beam_major_fwhm_pixels, recovery_radius_beams=_MULTISCALE_RECOVERY_RADIUS_BEAMS, canonical_seed_references_yx=None)

Attach bounded multiscale support to its owner along its own pixels.

Positive input labels are authoritative direct-residual source identities and keep their pixels. Each other valid significant pixel takes the owner of the seed pixel nearest to it along an eight-connected path through valid significant pixels, in axial steps of one pixel and diagonal steps of the square root of two, when that path is no longer than the recovery radius. Equal lengths use the owner whose globally row-major seed reference appears first, independently of task-local label integers or completion order. Every pixel on such a shortest path takes the same owner, so each owner's support is connected through its own pixels, and a pixel its owner reaches only across unsupported or invalid pixels, or through another owner's, is not attached to it. Within four pixels, path length orders the offsets as straight-line distance does, so without such obstacles a pixel takes its straight-line nearest owner; only the reach can differ, a path off the axes and diagonals being the longer (2.41 pixels to an offset of one by two, against 2.24).

A pixel's owner depends only on the planes within the radius, so a tiled caller needs that halo and the global reference pixel of every owner present in its read.

Source code in src/hebog/algorithms/extended_measurement.py

def assign_seeded_multiscale_support(  # noqa: PLR0913
    component_labels: npt.ArrayLike,
    significant_multiscale_support: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    *,
    beam_major_fwhm_pixels: float,
    recovery_radius_beams: float = _MULTISCALE_RECOVERY_RADIUS_BEAMS,
    canonical_seed_references_yx: (
        Mapping[int, tuple[int, int]] | None
    ) = None,
) -> npt.NDArray[np.int32]:
    """Attach bounded multiscale support to its owner along its own pixels.

    Positive input labels are authoritative direct-residual source identities
    and keep their pixels. Each other valid significant pixel takes the owner
    of the seed pixel nearest to it along an eight-connected path through
    valid significant pixels, in axial steps of one pixel and diagonal steps
    of the square root of two, when that path is no longer than the recovery
    radius. Equal lengths use the owner whose globally row-major seed
    reference appears first, independently of task-local label integers or
    completion order. Every pixel on such a shortest path takes the same
    owner, so each owner's support is connected through its own pixels, and
    a pixel its owner reaches only across unsupported or invalid pixels, or
    through another owner's, is not attached to it. Within four pixels, path
    length orders the offsets as straight-line distance does, so without
    such obstacles a pixel takes its straight-line nearest owner; only the
    reach can differ, a path off the axes and diagonals being the longer
    (2.41 pixels to an offset of one by two, against 2.24).

    A pixel's owner depends only on the planes within the radius, so a
    tiled caller needs that halo and the global reference pixel of every
    owner present in its read.
    """
    labels, significant, valid = _validated_seed_planes(
        component_labels,
        significant_multiscale_support,
        valid_pixels,
    )
    if (
        isinstance(beam_major_fwhm_pixels, bool)
        or not isfinite(beam_major_fwhm_pixels)
        or beam_major_fwhm_pixels <= 0.0
    ):
        raise ValueError("beam major FWHM must be finite and positive")
    if (
        isinstance(recovery_radius_beams, bool)
        or not isfinite(recovery_radius_beams)
        or recovery_radius_beams < 0.0
    ):
        raise ValueError("recovery radius must be finite and non-negative")
    output = np.asarray(labels, dtype=np.int32).copy()
    candidates = significant & valid & (labels == 0)
    if not np.any(labels > 0) or not np.any(candidates):
        return output
    seed_ranks, labels_by_rank = _canonical_seed_ranks(
        labels,
        canonical_seed_references_yx,
    )
    rank_plane = np.full(labels.shape, -1, dtype=np.int32)
    rank_plane[labels > 0] = seed_ranks
    nearest = _nearest_seed_ranks_along_paths(
        rank_plane,
        candidates,
        maximum_length=recovery_radius_beams * beam_major_fwhm_pixels,
    )
    attached = candidates & (nearest >= 0)
    output[attached] = labels_by_rank[nearest[attached]].astype(
        np.int32, copy=False
    )
    return output

clean_detected_segment_labels(component_labels)

Remove sub-beam protrusions while preserving segment identities.

A three-by-three binary opening is deliberately smaller than the sampled restoring beams supported by the source-finder contracts. It suppresses single-pixel flood-threshold excursions without growing, merging, or relabelling accepted emission.

Source code in src/hebog/algorithms/extended_measurement.py

def clean_detected_segment_labels(
    component_labels: npt.ArrayLike,
) -> npt.NDArray[np.int32]:
    """Remove sub-beam protrusions while preserving segment identities.

    A three-by-three binary opening is deliberately smaller than the sampled
    restoring beams supported by the source-finder contracts. It suppresses
    single-pixel flood-threshold excursions without growing, merging, or
    relabelling accepted emission.
    """
    labels = _segment_label_plane(component_labels)
    retained = binary_opening(
        labels > 0,
        structure=np.ones(
            (
                _SUB_BEAM_OPENING_WIDTH_PIXELS,
                _SUB_BEAM_OPENING_WIDTH_PIXELS,
            ),
            dtype=np.bool_,
        ),
    )
    return np.where(retained, labels, 0).astype(np.int32, copy=False)

expand_detected_segment_labels(component_labels, valid_pixels, *, radius_pixels)

Build unique nearest-segment apertures on observable pixels.

Expansion recovers original-pixel source wings omitted by the detection threshold. The input is one bounded measurement plane or tile. Where apertures overlap, each pixel belongs to its nearest accepted support, so close segments cannot double-count flux.

Source code in src/hebog/algorithms/extended_measurement.py

def expand_detected_segment_labels(
    component_labels: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    *,
    radius_pixels: int,
) -> npt.NDArray[np.int32]:
    """Build unique nearest-segment apertures on observable pixels.

    Expansion recovers original-pixel source wings omitted by the detection
    threshold. The input is one bounded measurement plane or tile. Where
    apertures overlap, each pixel belongs to its nearest accepted support, so
    close segments cannot double-count flux.
    """
    labels = _segment_label_plane(component_labels)
    valid = np.asarray(valid_pixels)
    if (
        valid.ndim != _IMAGE_DIMENSIONS
        or valid.shape != labels.shape
        or valid.dtype != np.bool_
    ):
        raise ValueError(
            "component labels and valid pixels must be aligned "
            "two-dimensional planes"
        )
    if isinstance(radius_pixels, bool) or radius_pixels < 0:
        raise ValueError("radius_pixels must be a non-negative integer")
    if not np.any(labels > 0):
        return np.zeros(labels.shape, dtype=np.int32)
    distances, nearest_indices = cast(
        tuple[
            npt.NDArray[np.float64],
            npt.NDArray[np.int32],
        ],
        distance_transform_edt(
            labels == 0,
            return_distances=True,
            return_indices=True,
        ),
    )
    nearest_labels = labels[tuple(nearest_indices)]
    expanded = np.where(
        (distances <= radius_pixels) & valid,
        nearest_labels,
        0,
    )
    return np.asarray(expanded, dtype=np.int32)

expand_source_measurement_labels(source_labels, valid_pixels, *, radius_pixels)

Expand source apertures with canonical source-identity tie breaking.

Source code in src/hebog/algorithms/extended_measurement.py

def expand_source_measurement_labels(
    source_labels: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    *,
    radius_pixels: int,
) -> npt.NDArray[np.int32]:
    """Expand source apertures with canonical source-identity tie breaking."""
    labels = _segment_label_plane(source_labels)
    valid = np.asarray(valid_pixels)
    if (
        valid.ndim != _IMAGE_DIMENSIONS
        or valid.shape != labels.shape
        or valid.dtype != np.bool_
    ):
        raise ValueError(
            "source labels and valid pixels must be aligned boolean "
            "two-dimensional planes"
        )
    if isinstance(radius_pixels, bool) or radius_pixels < 0:
        raise ValueError("radius_pixels must be a non-negative integer")
    output = np.asarray(labels, dtype=np.int32).copy()
    seed_points = np.column_stack(np.nonzero(labels > 0))
    if not seed_points.size or radius_pixels == 0:
        return output
    # Only a pixel within the radius of some seed can survive the distance
    # test below, so the reachable band is the candidate set. Querying every
    # unlabelled pixel instead costs 128 bytes of nearest-neighbour state
    # each, which is the plane's area rather than the sources' extent.
    reachable = maximum_filter(
        labels > 0,
        size=2 * radius_pixels + 1,
        mode="constant",
        cval=False,
    )
    candidate_points = np.column_stack(
        np.nonzero(valid & (labels == 0) & reachable)
    )
    if not candidate_points.size:
        return output
    canonical_labels = np.asarray(
        sorted(int(value) for value in np.unique(labels) if value > 0),
        dtype=np.int32,
    )
    ranks_by_label = {
        int(label): rank for rank, label in enumerate(canonical_labels)
    }
    seed_ranks = np.asarray(
        [ranks_by_label[int(labels[tuple(point)])] for point in seed_points],
        dtype=np.int64,
    )
    distances, owner_ranks = _nearest_canonical_seed_ranks(
        cast(_NearestSeedTree, cKDTree(seed_points)),
        np.asarray(candidate_points, dtype=np.int64),
        seed_ranks,
    )
    eligible = distances <= radius_pixels
    points = candidate_points[eligible]
    output[points[:, 0], points[:, 1]] = canonical_labels[
        owner_ranks[eligible]
    ]
    return output

measure_segment_position_pixels(y_pixels, x_pixels, signal_jy_per_beam, *, plane_shape_yx)

Measure a signed-flux centroid and peak from support pixels alone.

The arrays hold one segment's support pixels in raster order, with coordinates in the plane's pixel frame. A segment too wide to read at once is measured from the pixels its cores return, and the result is the one a window holding the whole segment would give, bit for bit: every sum and the first maximum visit the same pixels in the same order.

Source code in src/hebog/algorithms/extended_measurement.py

def measure_segment_position_pixels(
    y_pixels: npt.NDArray[np.int64],
    x_pixels: npt.NDArray[np.int64],
    signal_jy_per_beam: npt.NDArray[np.float64],
    *,
    plane_shape_yx: tuple[int, int],
) -> DetectedSegmentPosition:
    """Measure a signed-flux centroid and peak from support pixels alone.

    The arrays hold one segment's support pixels in raster order, with
    coordinates in the plane's pixel frame. A segment too wide to read at
    once is measured from the pixels its cores return, and the result is
    the one a window holding the whole segment would give, bit for bit:
    every sum and the first maximum visit the same pixels in the same order.
    """
    finite = np.isfinite(signal_jy_per_beam)
    support_pixel_count = int(np.count_nonzero(finite))
    if support_pixel_count == 0:
        return _unavailable_position(
            "empty-finite-support",
            support_pixel_count=0,
            integrated_weight=0.0,
        )
    weights = signal_jy_per_beam[finite]
    integrated_weight = float(np.sum(weights, dtype=np.float64))
    if not np.isfinite(integrated_weight) or integrated_weight <= 0:
        return _unavailable_position(
            "nonpositive-segment-flux",
            support_pixel_count=support_pixel_count,
            integrated_weight=integrated_weight,
        )
    conditioning = float(np.sum(np.abs(weights))) / integrated_weight
    epsilon = np.finfo(np.float64).eps
    if not np.isfinite(conditioning) or conditioning * epsilon > np.sqrt(
        epsilon
    ):
        return _unavailable_position(
            "ill-conditioned-segment-position",
            support_pixel_count=support_pixel_count,
            integrated_weight=integrated_weight,
        )
    # The coordinates are already in the plane's pixel frame, so which
    # window, if any, the pixels came from changes nothing in the sums.
    y_support = y_pixels[finite]
    x_support = x_pixels[finite]
    centroid_xy = (
        float(
            np.sum(x_support * weights, dtype=np.float64) / integrated_weight
        ),
        float(
            np.sum(y_support * weights, dtype=np.float64) / integrated_weight
        ),
    )
    # Signed cancellation can give a finite but physically unusable centroid.
    # Test the support rectangle, not mask membership: a shell's centre can
    # legitimately lie in its hole. Do not clamp the result to a bright pixel.
    roundoff = (
        epsilon * support_pixel_count * conditioning * max(plane_shape_yx)
    )
    if not (
        x_support.min() - roundoff
        <= centroid_xy[0]
        <= x_support.max() + roundoff
        and y_support.min() - roundoff
        <= centroid_xy[1]
        <= y_support.max() + roundoff
    ):
        return _unavailable_position(
            "ill-conditioned-segment-position",
            support_pixel_count=support_pixel_count,
            integrated_weight=integrated_weight,
        )
    # Raster order makes the first maximum the row-major first one.
    peak = int(np.argmax(weights))
    return DetectedSegmentPosition(
        available=True,
        centroid_xy=centroid_xy,
        peak_position_xy=(int(x_support[peak]), int(y_support[peak])),
        support_pixel_count=support_pixel_count,
        integrated_weight=integrated_weight,
        unavailable_reason=None,
    )

multiscale_recovery_radius_pixels(beam_major_fwhm_pixels, *, recovery_radius_beams=_MULTISCALE_RECOVERY_RADIUS_BEAMS)

Return the reviewed coherent-support recovery radius in pixels.

Source code in src/hebog/algorithms/extended_measurement.py

def multiscale_recovery_radius_pixels(
    beam_major_fwhm_pixels: float,
    *,
    recovery_radius_beams: float = _MULTISCALE_RECOVERY_RADIUS_BEAMS,
) -> int:
    """Return the reviewed coherent-support recovery radius in pixels."""
    if (
        isinstance(beam_major_fwhm_pixels, bool)
        or not isfinite(beam_major_fwhm_pixels)
        or beam_major_fwhm_pixels <= 0
    ):
        raise ValueError("beam major FWHM must be finite and positive")
    if (
        isinstance(recovery_radius_beams, bool)
        or not isfinite(recovery_radius_beams)
        or recovery_radius_beams < 0
    ):
        raise ValueError("recovery radius must be finite and non-negative")
    return ceil(recovery_radius_beams * beam_major_fwhm_pixels)

nearest_source_seed_labels(seed_points_yx, seed_labels, candidate_points_yx)

Return the label of each candidate pixel's nearest source seed.

The points are (y, x) pixel coordinates in any one frame, and an exact distance tie goes to the smaller source label, whose order follows the canonical source IDs. One candidate's owner depends only on the seeds, so a component too wide to read at once is assigned exactly by each core that holds its candidates, from the seeds that can own them.

Raises:

Type	Description
ValueError	If there is no seed, or the arrays disagree.

Source code in src/hebog/algorithms/extended_measurement.py

def nearest_source_seed_labels(
    seed_points_yx: npt.NDArray[np.int64],
    seed_labels: npt.NDArray[np.int64],
    candidate_points_yx: npt.NDArray[np.int64],
) -> npt.NDArray[np.int64]:
    """Return the label of each candidate pixel's nearest source seed.

    The points are ``(y, x)`` pixel coordinates in any one frame, and an
    exact distance tie goes to the smaller source label, whose order follows
    the canonical source IDs. One candidate's owner depends only on the
    seeds, so a component too wide to read at once is assigned exactly by
    each core that holds its candidates, from the seeds that can own them.

    Raises:
        ValueError: If there is no seed, or the arrays disagree.
    """
    if (
        seed_points_yx.ndim != _IMAGE_DIMENSIONS
        or seed_points_yx.shape[1:] != (_IMAGE_DIMENSIONS,)
        or not seed_points_yx.shape[0]
        or seed_labels.shape != seed_points_yx.shape[:1]
        or candidate_points_yx.ndim != _IMAGE_DIMENSIONS
        or candidate_points_yx.shape[1:] != (_IMAGE_DIMENSIONS,)
    ):
        raise ValueError(
            "source seeds must be non-empty, aligned (y, x) points"
        )
    _, owners = _nearest_canonical_seed_ranks(
        cast(_NearestSeedTree, cKDTree(seed_points_yx)),
        candidate_points_yx,
        seed_labels,
    )
    return owners

owner_support_needs_restore(published_window, kept_window, *, label_value)

Return whether cleanup removed one owner or split its publication.

Refinement may trim an owner's flood, but never remove it or split it. An owner keeps its original support when refinement keeps none of its pixels on its own evidence (kept_window, from :func:refinement_kept_labels), such as a compact detection between the detection threshold and the boundary floor whose footprint holds no full opening element: pixels recovered only because another owner's opened support lies near do not keep it. It keeps its original support too when the pixels publication would give it (published_window, from :func:published_owner_labels) are not one connected part.

The window must hold the owner's original, refined and published support completely; connectivity cannot be decided from a part of it. Callers that evaluate one owner per task use this decision, and the plane-wide :func:restore_segment_owners applies it to every owner.

Source code in src/hebog/algorithms/extended_measurement.py

def owner_support_needs_restore(
    published_window: npt.NDArray[np.int32],
    kept_window: npt.NDArray[np.int32],
    *,
    label_value: int,
) -> bool:
    """Return whether cleanup removed one owner or split its publication.

    Refinement may trim an owner's flood, but never remove it or split it.
    An owner keeps its original support when refinement keeps none of its
    pixels on its own evidence (``kept_window``, from
    :func:`refinement_kept_labels`), such as a compact detection between the
    detection threshold and the boundary floor whose footprint holds no full
    opening element: pixels recovered only because another owner's opened
    support lies near do not keep it. It keeps its original support too when
    the pixels publication would give it (``published_window``, from
    :func:`published_owner_labels`) are not one connected part.

    The window must hold the owner's original, refined and published support
    completely; connectivity cannot be decided from a part of it. Callers
    that evaluate one owner per task use this decision, and the plane-wide
    :func:`restore_segment_owners` applies it to every owner.
    """
    if not np.any(np.asarray(kept_window) == label_value):
        return True
    _, component_count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(
            np.asarray(published_window) == label_value,
            structure=np.ones((3, 3), dtype=np.int8),
        ),
    )
    return component_count != 1

preserve_owner_publication_bridges(previous_window, refined_window, *, label_value)

Keep only the previous regions that connect one owner's support.

The window must hold the owner completely; connectivity cannot be decided from a part of it. A bridge is a previously published region touching two retained parts of the same owner. When the owner still falls apart, its whole previous support is restored, and a part that remains disconnected from the previous support is dropped rather than published separately. When refinement retained nothing of the owner, its whole previous support is restored too, so no published owner loses all of its support.

Source code in src/hebog/algorithms/extended_measurement.py

def preserve_owner_publication_bridges(
    previous_window: npt.ArrayLike,
    refined_window: npt.ArrayLike,
    *,
    label_value: int,
) -> npt.NDArray[np.int32]:
    """Keep only the previous regions that connect one owner's support.

    The window must hold the owner completely; connectivity cannot be decided
    from a part of it. A bridge is a previously published region touching two
    retained parts of the same owner. When the owner still falls apart, its
    whole previous support is restored, and a part that remains disconnected
    from the previous support is dropped rather than published separately.
    When refinement retained nothing of the owner, its whole previous support
    is restored too, so no published owner loses all of its support.
    """
    structure = np.ones((3, 3), dtype=np.int8)
    previous = np.asarray(previous_window) == label_value
    local = np.asarray(refined_window, dtype=np.int32).copy()
    base_components, base_count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(local == label_value, structure=structure),
    )
    if base_count == 0:
        _, previous_count = cast(
            tuple[npt.NDArray[np.int32], int],
            connected_component_labels(previous, structure=structure),
        )
        if previous_count > 1:
            raise ValueError(
                "previous publication ownership must be connected"
            )
        local[previous] = label_value
        return local
    candidates, candidate_count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(
            previous & (local == 0),
            structure=structure,
        ),
    )
    for candidate_value in range(1, candidate_count + 1):
        candidate = candidates == candidate_value
        dilated_candidate = np.asarray(
            binary_dilation(candidate, structure=structure),
            dtype=np.bool_,
        )
        touching = np.unique(base_components[dilated_candidate])
        if np.count_nonzero(touching > 0) >= _MINIMUM_BRIDGE_TOUCH_COUNT:
            local[candidate] = label_value
    final_components, final_count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(local == label_value, structure=structure),
    )
    if final_count > 1:
        local[previous] = label_value
        final_components, final_count = cast(
            tuple[npt.NDArray[np.int32], int],
            connected_component_labels(
                local == label_value,
                structure=structure,
            ),
        )
    if final_count > 1:
        previous_components = np.unique(final_components[previous])
        previous_components = previous_components[previous_components > 0]
        if previous_components.size != 1:
            raise ValueError(
                "previous publication ownership must be connected"
            )
        local[
            (final_components > 0)
            & (final_components != previous_components[0])
        ] = 0
    return local

publication_owner_windows(owner_labels)

Return each owner's label and the window holding its support.

Source code in src/hebog/algorithms/extended_measurement.py

def publication_owner_windows(
    owner_labels: npt.NDArray[np.int64],
) -> tuple[tuple[int, tuple[slice, slice]], ...]:
    """Return each owner's label and the window holding its support."""
    ranked, label_values = _dense_label_ranks(owner_labels)
    return tuple(
        (int(label_value), bounds)
        for label_value, bounds in zip(
            label_values,
            find_objects(ranked),
            strict=True,
        )
        if bounds is not None
    )

published_owner_labels(support_labels, measurement_labels, *, flood_labels)

Give each pixel of a support plane the owner measurement gives it.

A refined pixel of a flood is published for that flood's owner, which measurement gives it. A refined pixel outside every flood is published for its measurement owner only where it touches that owner's flood, one step from it, so an owner restored whole is published in one part. A pixel no owner measures is not published.

Recovery at the island threshold never keeps a pixel touching a flood outside it, as that pixel would be in the flood, so in the source finder no recovered pixel outside every flood is published; persistence alone can join one to its owner, with the pixels between.

Source code in src/hebog/algorithms/extended_measurement.py

def published_owner_labels(
    support_labels: npt.ArrayLike,
    measurement_labels: npt.ArrayLike,
    *,
    flood_labels: npt.ArrayLike,
) -> npt.NDArray[np.int32]:
    """Give each pixel of a support plane the owner measurement gives it.

    A refined pixel of a flood is published for that flood's owner, which
    measurement gives it. A refined pixel outside every flood is published
    for its measurement owner only where it touches that owner's flood, one
    step from it, so an owner restored whole is published in one part. A
    pixel no owner measures is not published.

    Recovery at the island threshold never keeps a pixel touching a flood
    outside it, as that pixel would be in the flood, so in the source finder
    no recovered pixel outside every flood is published; persistence alone
    can join one to its owner, with the pixels between.
    """
    support = np.asarray(support_labels)
    measurement = np.asarray(measurement_labels)
    floods = np.asarray(flood_labels)
    if (
        measurement.ndim != _IMAGE_DIMENSIONS
        or support.shape != measurement.shape
        or floods.shape != measurement.shape
    ):
        raise ValueError(
            "support, measurement and flood labels must be aligned "
            "two-dimensional planes"
        )
    attached = (floods > 0) | _touches_own_label(floods, measurement)
    return np.where(
        (support > 0) & (measurement > 0) & attached, measurement, 0
    ).astype(np.int32, copy=False)

refine_multiscale_segment_labels(component_labels, combined_snr, significant_multiscale_support, *, beam_major_fwhm_pixels, core_minimum_neighbors=_MULTISCALE_CORE_MINIMUM_NEIGHBORS, boundary_minimum_snr=_MULTISCALE_BOUNDARY_MINIMUM_SNR, recovery_radius_beams=_MULTISCALE_RECOVERY_RADIUS_BEAMS, recovered_minimum_snr=None)

Refine segment support, restoring owners cleanup removes or splits.

The pixels publication gives each owner come from the whole-plane measurement ownership, so significant support must lie on valid pixels, as the published planes guarantee.

Source code in src/hebog/algorithms/extended_measurement.py

def refine_multiscale_segment_labels(  # noqa: PLR0913
    component_labels: npt.ArrayLike,
    combined_snr: npt.ArrayLike,
    significant_multiscale_support: npt.ArrayLike,
    *,
    beam_major_fwhm_pixels: float,
    core_minimum_neighbors: int = _MULTISCALE_CORE_MINIMUM_NEIGHBORS,
    boundary_minimum_snr: float = _MULTISCALE_BOUNDARY_MINIMUM_SNR,
    recovery_radius_beams: float = _MULTISCALE_RECOVERY_RADIUS_BEAMS,
    recovered_minimum_snr: float | None = None,
) -> npt.NDArray[np.int32]:
    """Refine segment support, restoring owners cleanup removes or splits.

    The pixels publication gives each owner come from the whole-plane
    measurement ownership, so significant support must lie on valid pixels,
    as the published planes guarantee.
    """
    labels = _segment_label_plane(component_labels)
    refined = refine_multiscale_segment_support(
        component_labels,
        combined_snr,
        significant_multiscale_support,
        beam_major_fwhm_pixels=beam_major_fwhm_pixels,
        core_minimum_neighbors=core_minimum_neighbors,
        boundary_minimum_snr=boundary_minimum_snr,
        recovery_radius_beams=recovery_radius_beams,
        recovered_minimum_snr=recovered_minimum_snr,
    )
    measurement = assign_seeded_multiscale_support(
        labels,
        np.asarray(significant_multiscale_support),
        np.ones(labels.shape, dtype=np.bool_),
        beam_major_fwhm_pixels=beam_major_fwhm_pixels,
        recovery_radius_beams=recovery_radius_beams,
    )
    return restore_segment_owners(
        labels,
        refined,
        published_labels=published_owner_labels(
            refined, measurement, flood_labels=labels
        ),
        kept_labels=refinement_kept_labels(
            labels,
            combined_snr,
            boundary_minimum_snr=boundary_minimum_snr,
        ),
    )

refine_multiscale_segment_support(component_labels, combined_snr, significant_multiscale_support, *, beam_major_fwhm_pixels, core_minimum_neighbors=_MULTISCALE_CORE_MINIMUM_NEIGHBORS, boundary_minimum_snr=_MULTISCALE_BOUNDARY_MINIMUM_SNR, recovery_radius_beams=_MULTISCALE_RECOVERY_RADIUS_BEAMS, recovered_minimum_snr=None)

Refine noisy flood boundaries with calibrated multiscale evidence.

Dense opened support remains without an additional significance test. Sparse boundary pixels remain only at high combined S/N, while adjacent significant à trous support may recover coherent emission omitted by the original-pixel flood. When recovered_minimum_snr is supplied, recovered support must also meet that original-pixel S/N floor. A recovered pixel of an owner's own flood keeps that owner, as publication does; one outside every flood inherits the nearest opened segment identity. This preserves deterministic ownership without merging or relabelling sources.

Every decision here is bounded by the opening and recovery radii, so a tile evaluates its own core exactly. Whether cleanup removed or split an owner is not: :func:restore_segment_owners decides that per owner, and :func:refine_multiscale_segment_labels composes the two.

Source code in src/hebog/algorithms/extended_measurement.py

def refine_multiscale_segment_support(  # noqa: PLR0913
    component_labels: npt.ArrayLike,
    combined_snr: npt.ArrayLike,
    significant_multiscale_support: npt.ArrayLike,
    *,
    beam_major_fwhm_pixels: float,
    core_minimum_neighbors: int = _MULTISCALE_CORE_MINIMUM_NEIGHBORS,
    boundary_minimum_snr: float = _MULTISCALE_BOUNDARY_MINIMUM_SNR,
    recovery_radius_beams: float = _MULTISCALE_RECOVERY_RADIUS_BEAMS,
    recovered_minimum_snr: float | None = None,
) -> npt.NDArray[np.int32]:
    """Refine noisy flood boundaries with calibrated multiscale evidence.

    Dense opened support remains without an additional significance test.
    Sparse boundary pixels remain only at high combined S/N, while adjacent
    significant à trous support may recover coherent emission omitted by the
    original-pixel flood. When ``recovered_minimum_snr`` is supplied, recovered
    support must also meet that original-pixel S/N floor. A recovered pixel
    of an owner's own flood keeps that owner, as publication does; one
    outside every flood inherits the nearest opened segment identity. This
    preserves deterministic ownership without merging or relabelling sources.

    Every decision here is bounded by the opening and recovery radii, so a
    tile evaluates its own core exactly. Whether cleanup removed or split an
    owner is not: :func:`restore_segment_owners` decides that per owner, and
    :func:`refine_multiscale_segment_labels` composes the two.
    """
    labels = _segment_label_plane(component_labels)
    snr = np.asarray(combined_snr)
    multiscale_support = np.asarray(significant_multiscale_support)
    if (
        snr.ndim != _IMAGE_DIMENSIONS
        or snr.shape != labels.shape
        or not np.issubdtype(snr.dtype, np.number)
        or np.issubdtype(snr.dtype, np.complexfloating)
    ):
        raise ValueError(
            "component labels and combined SNR must be aligned real "
            "two-dimensional planes"
        )
    if (
        multiscale_support.ndim != _IMAGE_DIMENSIONS
        or multiscale_support.shape != labels.shape
    ):
        raise ValueError(
            "component labels and multiscale support must be aligned "
            "two-dimensional planes"
        )
    if multiscale_support.dtype != np.bool_:
        raise ValueError("significant multiscale support must be boolean")
    if (
        isinstance(core_minimum_neighbors, bool)
        or not isinstance(core_minimum_neighbors, Integral)
        or not 1 <= core_minimum_neighbors <= _NEIGHBORHOOD_PIXEL_COUNT
    ):
        raise ValueError("core minimum neighbors must be an integer in [1, 9]")
    if not isfinite(boundary_minimum_snr) or boundary_minimum_snr <= 0:
        raise ValueError("boundary minimum SNR must be finite and positive")
    if recovered_minimum_snr is not None and (
        isinstance(recovered_minimum_snr, bool)
        or not isfinite(recovered_minimum_snr)
        or recovered_minimum_snr <= 0
    ):
        raise ValueError(
            "recovered minimum SNR must be finite and positive when supplied"
        )
    recovery_radius_pixels = multiscale_recovery_radius_pixels(
        beam_major_fwhm_pixels,
        recovery_radius_beams=recovery_radius_beams,
    )
    original_support = labels > 0
    high_confidence_boundary = original_support & (
        np.asarray(snr, dtype=np.float64) >= boundary_minimum_snr
    )
    cleaned = clean_detected_segment_labels(labels)
    cleaned_support = cleaned > 0
    if not np.any(cleaned_support):
        return np.where(high_confidence_boundary, labels, 0).astype(
            np.int32, copy=False
        )
    neighbor_count = convolve(
        cleaned_support.astype(np.int8),
        np.ones((3, 3), dtype=np.int8),
        mode="constant",
        cval=0,
    )
    dense_core = cleaned_support & (neighbor_count >= core_minimum_neighbors)
    nearby = (
        binary_dilation(cleaned_support, iterations=recovery_radius_pixels)
        if recovery_radius_pixels > 0
        else cleaned_support
    )
    recovered = multiscale_support & nearby
    if recovered_minimum_snr is not None:
        recovered &= np.asarray(snr, dtype=np.float64) >= recovered_minimum_snr
    _, nearest_indices = cast(
        tuple[npt.NDArray[np.float64], npt.NDArray[np.int32]],
        distance_transform_edt(
            ~cleaned_support,
            return_distances=True,
            return_indices=True,
        ),
    )
    owners = np.where(
        original_support, labels, cleaned[tuple(nearest_indices)]
    )
    retained = dense_core | recovered | high_confidence_boundary
    return np.where(retained, owners, 0).astype(np.int32, copy=False)

refine_persistent_publication_labels(component_labels, publication_labels, combined_snr, persistent_scale_support)

Publish connected owner support corroborated across adjacent scales.

Previously published low-confidence regions are retained only when they connect two retained parts of the same owner, or when nothing else of that owner is retained; no new threshold or ownership is introduced.

Source code in src/hebog/algorithms/extended_measurement.py

def refine_persistent_publication_labels(
    component_labels: npt.ArrayLike,
    publication_labels: npt.ArrayLike,
    combined_snr: npt.ArrayLike,
    persistent_scale_support: npt.ArrayLike,
) -> npt.NDArray[np.int32]:
    """Publish connected owner support corroborated across adjacent scales.

    Previously published low-confidence regions are retained only when they
    connect two retained parts of the same owner, or when nothing else of
    that owner is retained; no new threshold or ownership is introduced.
    """
    owners = _segment_label_plane(component_labels)
    return _preserve_publication_bridges(
        owners,
        _segment_label_plane(publication_labels),
        refine_persistent_publication_support(
            component_labels,
            publication_labels,
            combined_snr,
            persistent_scale_support,
        ),
    )

refine_persistent_publication_support(component_labels, publication_labels, combined_snr, persistent_scale_support, *, published_owner_values=None)

Retain owner support corroborated across adjacent scales, per pixel.

Dense opened support and independently strong original-image boundaries remain unchanged. Sparse recovered pixels remain only when their exact scale feature participates in an adjacent-scale association.

Whether an owner is published anywhere decides whether its persistent support is restored, which no bounded neighbourhood can answer. Tiled callers pass published_owner_values, the owners published over the whole image; a complete-plane call derives them.

Source code in src/hebog/algorithms/extended_measurement.py

def refine_persistent_publication_support(
    component_labels: npt.ArrayLike,
    publication_labels: npt.ArrayLike,
    combined_snr: npt.ArrayLike,
    persistent_scale_support: npt.ArrayLike,
    *,
    published_owner_values: npt.ArrayLike | None = None,
) -> npt.NDArray[np.int32]:
    """Retain owner support corroborated across adjacent scales, per pixel.

    Dense opened support and independently strong original-image boundaries
    remain unchanged. Sparse recovered pixels remain only when their exact
    scale feature participates in an adjacent-scale association.

    Whether an owner is published *anywhere* decides whether its persistent
    support is restored, which no bounded neighbourhood can answer. Tiled
    callers pass ``published_owner_values``, the owners published over the
    whole image; a complete-plane call derives them.
    """
    owners = _segment_label_plane(component_labels)
    previous = _segment_label_plane(publication_labels)
    snr = np.asarray(combined_snr)
    persistent = np.asarray(persistent_scale_support)
    if previous.shape != owners.shape:
        raise ValueError(
            "component and publication labels must be aligned label planes"
        )
    if np.any((previous > 0) & (previous != owners)):
        raise ValueError(
            "publication ownership must agree with component ownership"
        )
    if (
        snr.ndim != _IMAGE_DIMENSIONS
        or snr.shape != owners.shape
        or not np.issubdtype(snr.dtype, np.number)
        or np.iscomplexobj(snr)
    ):
        raise ValueError(
            "component labels and combined SNR must be aligned real "
            "two-dimensional planes"
        )
    if persistent.shape != owners.shape or persistent.dtype != np.bool_:
        raise ValueError(
            "persistent scale support must be one aligned boolean plane"
        )
    cleaned_support = clean_detected_segment_labels(owners) > 0
    neighbor_count = convolve(
        cleaned_support.astype(np.int8),
        np.ones((3, 3), dtype=np.int8),
        mode="constant",
        cval=0,
    )
    dense_core = cleaned_support & (
        neighbor_count >= _MULTISCALE_CORE_MINIMUM_NEIGHBORS
    )
    high_confidence_boundary = (owners > 0) & (
        np.asarray(snr, dtype=np.float64) >= _MULTISCALE_BOUNDARY_MINIMUM_SNR
    )
    previously_published = previous > 0
    retained = previously_published & (
        dense_core | high_confidence_boundary | persistent
    )
    published_owners = (
        np.unique(previous[previous > 0])
        if published_owner_values is None
        else np.asarray(published_owner_values)
    )
    restored_persistent = persistent & np.isin(owners, published_owners)
    return np.where(retained | restored_persistent, owners, 0).astype(
        np.int32,
        copy=False,
    )

refinement_kept_labels(component_labels, combined_snr, *, boundary_minimum_snr=_MULTISCALE_BOUNDARY_MINIMUM_SNR)

Return each owner's pixels refinement keeps on its own evidence.

These are its opened support and its pixels at the boundary floor. Both are bounded by the opening, so a tile evaluates its own core exactly.

Source code in src/hebog/algorithms/extended_measurement.py

def refinement_kept_labels(
    component_labels: npt.ArrayLike,
    combined_snr: npt.ArrayLike,
    *,
    boundary_minimum_snr: float = _MULTISCALE_BOUNDARY_MINIMUM_SNR,
) -> npt.NDArray[np.int32]:
    """Return each owner's pixels refinement keeps on its own evidence.

    These are its opened support and its pixels at the boundary floor. Both
    are bounded by the opening, so a tile evaluates its own core exactly.
    """
    labels = _segment_label_plane(component_labels)
    kept = (clean_detected_segment_labels(labels) > 0) | (
        (labels > 0)
        & (np.asarray(combined_snr, dtype=np.float64) >= boundary_minimum_snr)
    )
    return np.where(kept, labels, 0).astype(np.int32, copy=False)

restore_segment_owners(original_labels, refined_labels, *, published_labels, kept_labels)

Restore a direct owner only when cleanup would remove or split it.

Each owner is examined in the window holding its original and published support, instead of over the whole plane. Refinement recovers multiscale emission, so an owner can reach pixels its original support never covered, and those pixels decide whether cleanup split it. Every decision is taken before any owner is restored.

Source code in src/hebog/algorithms/extended_measurement.py

def restore_segment_owners(
    original_labels: npt.NDArray[np.int64],
    refined_labels: npt.NDArray[np.int32],
    *,
    published_labels: npt.NDArray[np.int32],
    kept_labels: npt.NDArray[np.int32],
) -> npt.NDArray[np.int32]:
    """Restore a direct owner only when cleanup would remove or split it.

    Each owner is examined in the window holding its original and published
    support, instead of over the whole plane. Refinement recovers multiscale
    emission, so an owner can reach pixels its original support never
    covered, and those pixels decide whether cleanup split it. Every
    decision is taken before any owner is restored.
    """
    windows = _owner_windows(original_labels, published_labels)
    restored = [
        int(label_value)
        for label_value, crop in windows.items()
        if crop is not None
        and owner_support_needs_restore(
            published_labels[crop],
            kept_labels[crop],
            label_value=label_value,
        )
    ]
    return apply_owner_restores(original_labels, refined_labels, restored)

segment_refinement_halo_pixels(beam_major_fwhm_pixels, *, recovery_radius_beams=_MULTISCALE_RECOVERY_RADIUS_BEAMS)

Return the halo covering refinement and persistence of the support.

Two reaches set it, and in each the radii add rather than compete. Refinement recovers a pixel from opened support up to the recovery radius r away, which it rounds up, and that opened support depends on labels two pixels further: ceil(r) + 2. Persistence takes its dense core from a 3x3 opening and a 3x3 count over the measurement labels, three pixels, and measurement gives a pixel to a seed at most r away, so at most floor(r) pixels along either axis: floor(r) + 3. The two agree unless r is a whole number of pixels, as it is at the default half-beam radius for any beam an even number of pixels wide, where persistence needs one pixel more. The second also covers refinement's own dense-core count, three pixels, when the recovery radius is small.

Source code in src/hebog/algorithms/extended_measurement.py

def segment_refinement_halo_pixels(
    beam_major_fwhm_pixels: float,
    *,
    recovery_radius_beams: float = _MULTISCALE_RECOVERY_RADIUS_BEAMS,
) -> int:
    """Return the halo covering refinement and persistence of the support.

    Two reaches set it, and in each the radii add rather than compete.
    Refinement recovers a pixel from opened support up to the recovery
    radius ``r`` away, which it rounds up, and that opened support depends on
    labels two pixels further: ``ceil(r) + 2``. Persistence takes its dense
    core from a 3x3 opening and a 3x3 count over the measurement labels,
    three pixels, and measurement gives a pixel to a seed at most ``r``
    away, so at most ``floor(r)`` pixels along either axis:
    ``floor(r) + 3``. The two agree unless ``r`` is a whole number of
    pixels, as it is at the default half-beam radius for any beam an even
    number of pixels wide, where persistence needs one pixel more. The
    second also covers refinement's own dense-core count, three pixels, when
    the recovery radius is small.
    """
    recovery_pixels = multiscale_recovery_radius_pixels(
        beam_major_fwhm_pixels,
        recovery_radius_beams=recovery_radius_beams,
    )
    measurement_reach_pixels = floor(
        recovery_radius_beams * beam_major_fwhm_pixels
    )
    return max(
        recovery_pixels + _OPENING_INFLUENCE_PIXELS,
        measurement_reach_pixels + _DENSE_CORE_INFLUENCE_PIXELS,
    )

hebog.algorithms.owner_connectivity

Owner connectivity decided from tile cores, exactly.

The support pass asks two questions of every owner that no bounded neighbourhood answers: whether cleanup splits its published support or removes it, and which earlier published regions bridge the parts of its persistent support, or replace it when none is left. Both are about the connected components of one label's pixels, and a component of one label connects only through pixels of that label, so the island reconciliation, which joins any two touching mask pixels, cannot answer them for owners that touch each other.

These functions answer them for an owner too wide to read at once. Each core labels its own same-label components and keeps only their labels and the component on each core-edge pixel; components join where they meet across the edges of the cores that were observed; and the decisions are taken over the joined components, exactly as :func:~hebog.algorithms.extended_measurement.owner_support_needs_restore and :func:~hebog.algorithms.extended_measurement.preserve_owner_publication_bridges take them over a window holding the whole owner.

ComponentKey = tuple[str, int]

One core's tile identifier and a component number local to that core.

LabelComponentSummary dataclass

One core's components, reduced to their labels and edge pixels.

labels[k - 1] is the label component k carries, and edges holds the component on every pixel of the core's four edges, 0 where there is none. That is all a join across cores needs.

Source code in src/hebog/algorithms/owner_connectivity.py

@dataclass(frozen=True, slots=True)
class LabelComponentSummary:
    """One core's components, reduced to their labels and edge pixels.

    ``labels[k - 1]`` is the label component ``k`` carries, and ``edges``
    holds the component on every pixel of the core's four edges, 0 where
    there is none. That is all a join across cores needs.
    """

    partition: TilePartition
    labels: tuple[int, ...]
    edges: TileBoundaryLabels

    def component_keys(self) -> tuple[ComponentKey, ...]:
        """Name every component this core holds."""
        tile_id = self.partition.tile_id
        return tuple(
            (tile_id, local) for local in range(1, len(self.labels) + 1)
        )

component_keys()

Name every component this core holds.

Source code in src/hebog/algorithms/owner_connectivity.py

def component_keys(self) -> tuple[ComponentKey, ...]:
    """Name every component this core holds."""
    tile_id = self.partition.tile_id
    return tuple(
        (tile_id, local) for local in range(1, len(self.labels) + 1)
    )

LabelComponents dataclass

One core's same-label eight-connected components.

components numbers them from one: label by label in ascending order, and in raster order within a label, so a core always numbers the same plane the same way. labels[k - 1] is the label of component k.

Source code in src/hebog/algorithms/owner_connectivity.py

@dataclass(frozen=True, slots=True)
class LabelComponents:
    """One core's same-label eight-connected components.

    ``components`` numbers them from one: label by label in ascending order,
    and in raster order within a label, so a core always numbers the same
    plane the same way. ``labels[k - 1]`` is the label of component ``k``.
    """

    partition: TilePartition
    components: npt.NDArray[np.int32]
    labels: tuple[int, ...]

    def summary(self) -> LabelComponentSummary:
        """Return the labels and edge pixels, which is all a join needs."""
        components = self.components
        return LabelComponentSummary(
            partition=self.partition,
            labels=self.labels,
            edges=TileBoundaryLabels(
                top=components[0, :].copy(),
                bottom=components[-1, :].copy(),
                left=components[:, 0].copy(),
                right=components[:, -1].copy(),
            ),
        )

    def label_of(self) -> npt.NDArray[np.int64]:
        """Return a lookup from component number to label, 0 for none."""
        return np.asarray((0, *self.labels), dtype=np.int64)

label_of()

Return a lookup from component number to label, 0 for none.

Source code in src/hebog/algorithms/owner_connectivity.py

def label_of(self) -> npt.NDArray[np.int64]:
    """Return a lookup from component number to label, 0 for none."""
    return np.asarray((0, *self.labels), dtype=np.int64)

summary()

Return the labels and edge pixels, which is all a join needs.

Source code in src/hebog/algorithms/owner_connectivity.py

def summary(self) -> LabelComponentSummary:
    """Return the labels and edge pixels, which is all a join needs."""
    components = self.components
    return LabelComponentSummary(
        partition=self.partition,
        labels=self.labels,
        edges=TileBoundaryLabels(
            top=components[0, :].copy(),
            bottom=components[-1, :].copy(),
            left=components[:, 0].copy(),
            right=components[:, -1].copy(),
        ),
    )

OwnerBridgeCore dataclass

What one core observed of the owners' bridge question, safe to return.

touching pairs the candidate and base components that touch inside the core, and base_holding_previous names the base components holding a previously published pixel of their owner.

Source code in src/hebog/algorithms/owner_connectivity.py

@dataclass(frozen=True, slots=True)
class OwnerBridgeCore:
    """What one core observed of the owners' bridge question, safe to return.

    ``touching`` pairs the candidate and base components that touch inside
    the core, and ``base_holding_previous`` names the base components holding
    a previously published pixel of their owner.
    """

    base: LabelComponentSummary
    candidates: LabelComponentSummary
    touching: tuple[tuple[int, int], ...]
    base_holding_previous: tuple[int, ...]

OwnerBridgePlanes dataclass

One core's base and candidate components for the owners it holds.

The base is each owner's persistent support; the candidates are its previously published pixels outside that support. They stay on the task that labelled them.

Source code in src/hebog/algorithms/owner_connectivity.py

@dataclass(frozen=True, slots=True)
class OwnerBridgePlanes:
    """One core's base and candidate components for the owners it holds.

    The base is each owner's persistent support; the candidates are its
    previously published pixels outside that support. They stay on the
    task that labelled them.
    """

    base: LabelComponents
    candidates: LabelComponents

OwnerBridgeShare dataclass

The components one core adds to their owners or clears from them.

Source code in src/hebog/algorithms/owner_connectivity.py

@dataclass(frozen=True, slots=True)
class OwnerBridgeShare:
    """The components one core adds to their owners or clears from them."""

    added: tuple[int, ...]
    cleared: tuple[int, ...]

apply_owner_bridges(final_core, planes, share)

Write one core's share of the bridge decisions, in place.

planes must be the ones this core observed, labelled from the same planes with the same owners, so their component numbers are the ones the decision names.

Source code in src/hebog/algorithms/owner_connectivity.py

def apply_owner_bridges(
    final_core: npt.NDArray[np.int32],
    planes: OwnerBridgePlanes,
    share: OwnerBridgeShare,
) -> None:
    """Write one core's share of the bridge decisions, in place.

    ``planes`` must be the ones this core observed, labelled from the same
    planes with the same owners, so their component numbers are the ones the
    decision names.
    """
    candidates = planes.candidates.components
    added = np.isin(candidates, share.added)
    final_core[added] = planes.candidates.label_of()[candidates[added]]
    final_core[np.isin(planes.base.components, share.cleared)] = 0

decide_owner_bridges(cores)

Apply the owner bridge rule over joined components, owner by owner.

A candidate touching two parts of its owner's base bridges them. If the base and its bridges are connected they are published; otherwise the part holding the owner's earlier pixels is kept, with every candidate it holds, and every other part is cleared. An owner with no base keeps all of its candidates, which are then its whole earlier support. That is :func:~hebog.algorithms.extended_measurement.preserve_owner_publication_bridges, whose connected components here are the joined base and candidate components and the touching pairs between them: two candidates never touch, and neither do two base parts, or each would be one component.

Returns:

Type	Description
dict[str, OwnerBridgeShare]	Each core's share of the decisions, for the cores that change, and
frozenset[int]	the owners whose support changes.

Raises:

Type	Description
ValueError	If an owner's earlier pixels lie in several parts that nothing reconnects, or it has no base and its earlier pixels are not one part, which the window rule refuses too.

Source code in src/hebog/algorithms/owner_connectivity.py

def decide_owner_bridges(
    cores: Sequence[OwnerBridgeCore],
) -> tuple[dict[str, OwnerBridgeShare], frozenset[int]]:
    """Apply the owner bridge rule over joined components, owner by owner.

    A candidate touching two parts of its owner's base bridges them. If the
    base and its bridges are connected they are published; otherwise the
    part holding the owner's earlier pixels is kept, with every candidate it
    holds, and every other part is cleared. An owner with no base keeps all
    of its candidates, which are then its whole earlier support. That is
    :func:`~hebog.algorithms.extended_measurement.preserve_owner_publication_bridges`,
    whose connected components here are the joined base and candidate
    components and the touching pairs between them: two candidates never
    touch, and neither do two base parts, or each would be one component.

    Returns:
        Each core's share of the decisions, for the cores that change, and
        the owners whose support changes.

    Raises:
        ValueError: If an owner's earlier pixels lie in several parts that
            nothing reconnects, or it has no base and its earlier pixels are
            not one part, which the window rule refuses too.
    """
    bases = _joined(tuple(core.base for core in cores))
    candidates = _joined(tuple(core.candidates for core in cores))
    touched = _touched(cores, bases.roots, candidates.roots)
    holding = {
        bases.roots[(core.base.partition.tile_id, local)]
        for core in cores
        for local in core.base_holding_previous
    }
    added: set[ComponentKey] = set()
    cleared: set[ComponentKey] = set()
    changed: set[int] = set()
    for owner in sorted(bases.by_label.keys() | candidates.by_label.keys()):
        owner_bases = bases.by_label.get(owner, set())
        owner_candidates = candidates.by_label.get(owner, set())
        owner_added, owner_cleared = (
            _decide_owner(owner_bases, owner_candidates, touched, holding)
            if owner_bases
            else _restore_owner(owner_candidates)
        )
        added |= owner_added
        cleared |= owner_cleared
        if owner_added or owner_cleared:
            changed.add(owner)
    return (
        _shares(cores, bases.roots, candidates.roots, added, cleared),
        frozenset(changed),
    )

join_components(keys, touching)

Return every component's root once each touching pair is joined.

Two components share a root exactly when a chain of touching pairs links them; which member is the root does not matter, only that it is shared.

Source code in src/hebog/algorithms/owner_connectivity.py

def join_components[Key: Hashable](
    keys: Iterable[Key],
    touching: Iterable[tuple[Key, Key]],
) -> dict[Key, Key]:
    """Return every component's root once each touching pair is joined.

    Two components share a root exactly when a chain of touching pairs links
    them; which member is the root does not matter, only that it is shared.
    """
    parent: dict[Key, Key] = {key: key for key in keys}

    def root(key: Key) -> Key:
        """Follow one key to its root, compressing the path."""
        top = key
        while parent[top] != top:
            top = parent[top]
        while parent[key] != top:
            parent[key], key = top, parent[key]
        return top

    for first, second in touching:
        first_root, second_root = root(first), root(second)
        if first_root != second_root:
            parent[second_root] = first_root
    return {key: root(key) for key in parent}

label_components(label_core, partition)

Label every same-label eight-connected component of one core.

Each label is labelled inside its own bounds, so the cost follows the labels' extents rather than the core times the number of labels.

Raises:

Type	Description
ValueError	If the plane is not the partition's core shape.

Source code in src/hebog/algorithms/owner_connectivity.py

def label_components(
    label_core: npt.NDArray[np.int32],
    partition: TilePartition,
) -> LabelComponents:
    """Label every same-label eight-connected component of one core.

    Each label is labelled inside its own bounds, so the cost follows the
    labels' extents rather than the core times the number of labels.

    Raises:
        ValueError: If the plane is not the partition's core shape.
    """
    plane = np.asarray(label_core, dtype=np.int32)
    if plane.shape != partition.core_bounds.shape_yx:
        raise ValueError("owner labels must cover the partition's core")
    components = np.zeros(plane.shape, dtype=np.int32)
    labels: list[int] = []
    extents = label_extents(plane)
    for value, y_start, y_stop, x_start, x_stop in zip(
        extents.values,
        extents.y_start,
        extents.y_stop,
        extents.x_start,
        extents.x_stop,
        strict=True,
    ):
        window = (
            slice(int(y_start), int(y_stop)),
            slice(int(x_start), int(x_stop)),
        )
        local, count = cast(
            "tuple[npt.NDArray[np.int32], int]",
            connected_component_labels(
                plane[window] == value, structure=_EIGHT_CONNECTED
            ),
        )
        components[window] = np.where(
            local > 0, local + len(labels), components[window]
        )
        labels.extend([int(value)] * count)
    return LabelComponents(
        partition=partition,
        components=components,
        labels=tuple(labels),
    )

observe_owner_bridges(planes, previous_core)

Reduce one core's bridge components to what a decision needs.

Source code in src/hebog/algorithms/owner_connectivity.py

def observe_owner_bridges(
    planes: OwnerBridgePlanes,
    previous_core: npt.NDArray[np.int32],
) -> OwnerBridgeCore:
    """Reduce one core's bridge components to what a decision needs."""
    base = planes.base.components
    holding = (base > 0) & (
        np.asarray(previous_core) == planes.base.label_of()[base]
    )
    return OwnerBridgeCore(
        base=planes.base.summary(),
        candidates=planes.candidates.summary(),
        touching=tuple(
            sorted(touching_within_core(planes.candidates, planes.base))
        ),
        base_holding_previous=tuple(
            int(local) for local in np.unique(base[holding])
        ),
    )

owner_bridge_planes(persistent_core, previous_core, owners, partition)

Label one core's base and candidate components for chosen owners.

persistent_core is the persistent support the bridge rule refines and previous_core the earlier publication, both over one core. An owner's published pixel carries that owner or nothing in the persistent plane, so its candidates are its published pixels the plane leaves empty.

Source code in src/hebog/algorithms/owner_connectivity.py

def owner_bridge_planes(
    persistent_core: npt.NDArray[np.int32],
    previous_core: npt.NDArray[np.int32],
    owners: Iterable[int],
    partition: TilePartition,
) -> OwnerBridgePlanes:
    """Label one core's base and candidate components for chosen owners.

    ``persistent_core`` is the persistent support the bridge rule refines
    and ``previous_core`` the earlier publication, both over one core. An
    owner's published pixel carries that owner or nothing in the persistent
    plane, so its candidates are its published pixels the plane leaves empty.
    """
    chosen = np.asarray(sorted(owners), dtype=np.int64)
    persistent = np.asarray(persistent_core, dtype=np.int32)
    previous = np.asarray(previous_core, dtype=np.int32)
    return OwnerBridgePlanes(
        base=label_components(
            np.where(np.isin(persistent, chosen), persistent, 0).astype(
                np.int32, copy=False
            ),
            partition,
        ),
        candidates=label_components(
            np.where(
                np.isin(previous, chosen) & (persistent == 0), previous, 0
            ).astype(np.int32, copy=False),
            partition,
        ),
    )

owners_needing_restore(summaries, owners)

Return the owners whose pixels fall into several joined parts, or none.

The cores must hold every pixel of each owner asked about. An owner that cleanup removed leaves no component in any core, so the caller names the owners it asks about rather than the summaries naming them. This is :func:~hebog.algorithms.extended_measurement.owner_support_needs_restore for every owner at once.

Source code in src/hebog/algorithms/owner_connectivity.py

def owners_needing_restore(
    summaries: Sequence[LabelComponentSummary],
    owners: Iterable[int],
) -> frozenset[int]:
    """Return the owners whose pixels fall into several joined parts, or none.

    The cores must hold every pixel of each owner asked about. An owner that
    cleanup removed leaves no component in any core, so the caller names the
    owners it asks about rather than the summaries naming them. This is
    :func:`~hebog.algorithms.extended_measurement.owner_support_needs_restore`
    for every owner at once.
    """
    by_label = _joined(summaries).by_label
    return frozenset(
        owner for owner in owners if len(by_label.get(owner, ())) != 1
    )

touching_across_cores(first, second)

Return pairs of same-label components that touch across a core edge.

The first component of each pair comes from first and the second from a neighbouring core of second, in any direction, so the pairs between two planes are all found once and those within one plane twice. Only the cores given are consulted: a core that holds none of the labels need not be observed at all.

Source code in src/hebog/algorithms/owner_connectivity.py

def touching_across_cores(
    first: Sequence[LabelComponentSummary],
    second: Sequence[LabelComponentSummary],
) -> frozenset[tuple[ComponentKey, ComponentKey]]:
    """Return pairs of same-label components that touch across a core edge.

    The first component of each pair comes from ``first`` and the second
    from a neighbouring core of ``second``, in any direction, so the pairs
    between two planes are all found once and those within one plane twice.
    Only the cores given are consulted: a core that holds none of the labels
    need not be observed at all.
    """
    by_index = {
        (summary.partition.tile_y_index, summary.partition.tile_x_index): (
            summary
        )
        for summary in second
    }
    pairs: set[tuple[ComponentKey, ComponentKey]] = set()
    for summary in first:
        y_index = summary.partition.tile_y_index
        x_index = summary.partition.tile_x_index
        for (dy, dx), mine, theirs in _SIDES:
            neighbour = by_index.get((y_index + dy, x_index + dx))
            if neighbour is not None:
                pairs |= _facing_pairs(
                    summary,
                    _edge(summary.edges, mine),
                    neighbour,
                    _edge(neighbour.edges, theirs),
                    (-1, 0, 1),
                )
        for (dy, dx), (mine, own_corner), (theirs, facing) in _CORNERS:
            neighbour = by_index.get((y_index + dy, x_index + dx))
            if neighbour is not None:
                pairs |= _facing_pairs(
                    summary,
                    _edge(summary.edges, mine)[[own_corner]],
                    neighbour,
                    _edge(neighbour.edges, theirs)[[facing]],
                    (0,),
                )
    return frozenset(pairs)

touching_within_core(first, second)

Return pairs of same-label components of two planes that touch.

Both planes cover one core, and a pixel touches its eight neighbours.

Source code in src/hebog/algorithms/owner_connectivity.py

def touching_within_core(
    first: LabelComponents,
    second: LabelComponents,
) -> frozenset[tuple[int, int]]:
    """Return pairs of same-label components of two planes that touch.

    Both planes cover one core, and a pixel touches its eight neighbours.
    """
    height, width = first.components.shape
    first_labels = first.label_of()
    second_labels = second.label_of()
    padded = np.pad(second.components, 1)
    first_owners = first_labels[first.components]
    pairs: set[tuple[int, int]] = set()
    for dy in (-1, 0, 1):
        for dx in (-1, 0, 1):
            neighbour = padded[
                1 + dy : 1 + dy + height, 1 + dx : 1 + dx + width
            ]
            touching = (
                (first.components > 0)
                & (neighbour > 0)
                & (first_owners == second_labels[neighbour])
            )
            if not bool(np.any(touching)):
                continue
            pairs.update(
                (int(mine), int(theirs))
                for mine, theirs in np.unique(
                    np.column_stack(
                        (first.components[touching], neighbour[touching])
                    ),
                    axis=0,
                )
            )
    return frozenset(pairs)

hebog.stages.publication

Owner-scoped and core-scoped rounds of the tile-native support pass.

ADR-008 splits the support pass because two of its steps are scoped to an owner rather than to a bounded neighbourhood: restoring an owner that cleanup would remove or whose publication it would split, and preserving the previously published regions that bridge two retained parts of one owner, or all of them when no part is retained. Each is decided once per owner, from the window holding that owner, and returns a record; the cores then apply those records and write the final labels and mask.

No admission bounds an owner's area, so an owner's window can be wider than the read budget. Such an owner is never read whole: both of its questions are about the connected components of its own pixels, so each core that its window reaches labels its components, the components join across core edges, and the decisions are taken over them (:mod:hebog.algorithms.owner_connectivity), with exactly the answer the window would give.

Every pixel quantity here is recomputed in the round that needs it rather than persisted, exactly as the detection pass recomputes its filters instead of storing a response bank.

PublicationStageConfig dataclass

Reviewed support thresholds and the bounded task limits.

Source code in src/hebog/stages/publication.py

@dataclass(frozen=True, slots=True)
class PublicationStageConfig:
    """Reviewed support thresholds and the bounded task limits."""

    beam: BeamShapePixels
    island_threshold_sigma: float
    minimum_island_pixels: int
    maximum_island_pixels: int | None
    maximum_tiles_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before stage products are initialized."""
        if (
            isinstance(self.maximum_tiles_per_batch, bool)
            or not isinstance(self.maximum_tiles_per_batch, Integral)
            or self.maximum_tiles_per_batch < 1
        ):
            raise ValueError(
                "maximum_tiles_per_batch must be a positive integer"
            )
        if (
            isinstance(self.maximum_batch_read_pixels, bool)
            or not isinstance(self.maximum_batch_read_pixels, Integral)
            or self.maximum_batch_read_pixels < 1
        ):
            raise ValueError(
                "maximum_batch_read_pixels must be a positive integer"
            )
        if self.minimum_island_pixels < 1:
            raise ValueError("minimum island pixels must be positive")

    @property
    def halo_pixels(self) -> int:
        """Return the halo every pixel decision in this pass reads."""
        return segment_refinement_halo_pixels(self.beam.major_fwhm_pixels)

halo_pixels property

Return the halo every pixel decision in this pass reads.

__post_init__()

Reject an unbounded task before stage products are initialized.

Source code in src/hebog/stages/publication.py

def __post_init__(self) -> None:
    """Reject an unbounded task before stage products are initialized."""
    if (
        isinstance(self.maximum_tiles_per_batch, bool)
        or not isinstance(self.maximum_tiles_per_batch, Integral)
        or self.maximum_tiles_per_batch < 1
    ):
        raise ValueError(
            "maximum_tiles_per_batch must be a positive integer"
        )
    if (
        isinstance(self.maximum_batch_read_pixels, bool)
        or not isinstance(self.maximum_batch_read_pixels, Integral)
        or self.maximum_batch_read_pixels < 1
    ):
        raise ValueError(
            "maximum_batch_read_pixels must be a positive integer"
        )
    if self.minimum_island_pixels < 1:
        raise ValueError("minimum island pixels must be positive")

PublicationStageResult dataclass

Published support products and scalar execution evidence.

Source code in src/hebog/stages/publication.py

@dataclass(frozen=True, slots=True)
class PublicationStageResult:
    """Published support products and scalar execution evidence."""

    generation: ProductGenerationManifest
    accepted_island_count: int
    restored_owner_count: int
    bridged_owner_count: int
    published_owner_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_owner_read_pixels: int
    maximum_batch_read_pixels: int
    owner_batch_count: int
    wide_owner_count: int

publication_product_names()

Return the canonical published support product set.

Source code in src/hebog/stages/publication.py

def publication_product_names() -> tuple[str, ...]:
    """Return the canonical published support product set."""
    return _PUBLICATION_PRODUCT_NAMES

run_publication_stage(detection_source, support_source, manifest, *, detection_islands, config, executor, sink)

Decide owner connectivity, then publish the final support products.

Four rounds, in the order ADR-008 sets out: owners that cleanup would remove or whose publication it would split, the owners published anywhere, the label patches that bridge an owner's support or restore it when persistence keeps none, and the cores that apply all three with the caller's island admission. Only the last round writes.

An owner whose read exceeds maximum_batch_read_pixels is decided from its cores instead of its window: one round labels its published support in every core its window reaches, before the scan, and another labels its base and candidate support, after it. The write round applies each core's share of the decisions. wide_owner_count reports how many owners took that path.

Source code in src/hebog/stages/publication.py

def run_publication_stage(  # noqa: PLR0913
    detection_source: CompletedProductSource,
    support_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    detection_islands: tuple[DetectedIsland, ...],
    config: PublicationStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
) -> PublicationStageResult:
    """Decide owner connectivity, then publish the final support products.

    Four rounds, in the order ADR-008 sets out: owners that cleanup would
    remove or whose publication it would split, the owners published
    anywhere, the label patches that bridge an owner's support or restore it
    when persistence keeps none, and the cores that apply all three with the
    caller's island admission. Only the last round writes.

    An owner whose read exceeds ``maximum_batch_read_pixels`` is decided
    from its cores instead of its window: one round labels its published
    support in every core its window reaches, before the scan, and another
    labels its base and candidate support, after it. The write round applies
    each core's share of the decisions. ``wide_owner_count`` reports how many
    owners took that path.
    """
    _validate_stage_inputs(
        detection_source,
        support_source,
        manifest,
        config,
        sink,
    )
    image_shape_yx = manifest.image_shape_yx
    image_width = image_shape_yx[1]
    owner_requests = _owner_requests(
        detection_islands,
        image_shape_yx=image_shape_yx,
        recovery_pixels=multiscale_recovery_radius_pixels(
            config.beam.major_fwhm_pixels
        ),
        halo_pixels=config.halo_pixels,
    )
    budget = config.maximum_batch_read_pixels
    wide = tuple(
        request
        for request in owner_requests
        if read_pixels(request.read_bounds) > budget
    )
    owner_batches = _owner_batches(
        tuple(
            request
            for request in owner_requests
            if read_pixels(request.read_bounds) <= budget
        ),
        core_shape_yx=manifest.tile_core_shape_yx,
        maximum_batch_read_pixels=budget,
    )
    owner_batches = tuple(
        _OwnerBatch(
            requests=batch.requests,
            read_bounds=batch.read_bounds,
            seed_references_yx=_seed_shard(
                detection_islands, batch.read_bounds
            ),
        )
        for batch in owner_batches
    )
    restore_results = _map_owner_batches(
        executor,
        owner_batches,
        partial(
            _decide_restores,
            detection_source=detection_source,
            support_source=support_source,
            config=config,
        ),
        chunk_shape_yx=manifest.tile_core_shape_yx,
    )
    wide_cores = tuple(
        partition
        for partition in manifest.tiles
        if _wide_shard(wide, partition)
    )
    split_batches = _tile_batches(
        tuple(
            _TileRequest(
                partition=partition,
                seed_references_yx=_seed_shard(
                    detection_islands, partition.read_bounds
                ),
                restored_owners=(),
                wide_owners=_wide_shard(wide, partition),
            )
            for partition in wide_cores
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    split_results = map_round(
        executor,
        partial(
            _observe_wide_splits,
            detection_source=detection_source,
            support_source=support_source,
            config=config,
        ),
        split_batches,
        round_name="wide owner split",
        requirement=read_requirement(
            _TILE_BYTES_PER_READ_PIXEL,
            (partition.read_bounds for partition in wide_cores),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    split_summaries = tuple(
        summary for result in split_results for summary in result.summaries
    )
    _require_every_core(
        split_batches,
        (summary.partition.tile_id for summary in split_summaries),
        question="split",
    )
    wide_labels = frozenset(request.label_value for request in wide)
    kept_wide = frozenset(
        owner for result in split_results for owner in result.kept_owners
    )
    restored = (
        frozenset(
            owner
            for result in restore_results
            for owner in result.restored_owners
        )
        | owners_needing_restore(split_summaries, wide_labels)
        | (wide_labels - kept_wide)
    )

    def tile_request(
        partition: TilePartition,
        **shards: object,
    ) -> _TileRequest:
        """Shard the global decisions to one core's own read."""
        return _TileRequest(
            partition=partition,
            seed_references_yx=_seed_shard(
                detection_islands,
                partition.read_bounds,
            ),
            restored_owners=_owner_shard(
                detection_islands,
                restored,
                partition.read_bounds,
            ),
            **shards,  # type: ignore[arg-type]
        )

    scan_batches = _tile_batches(
        tuple(tile_request(partition) for partition in manifest.tiles),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_published_owners,
                detection_source=detection_source,
                support_source=support_source,
                config=config,
            ),
            scan_batches,
            requirement=read_requirement(
                _TILE_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no published-owner results")
    published = frozenset(
        owner for result in scan_results for owner in result.published_owners
    )
    bridge_results = _map_owner_batches(
        executor,
        tuple(
            _OwnerBatch(
                requests=batch.requests,
                read_bounds=batch.read_bounds,
                seed_references_yx=batch.seed_references_yx,
                restored_owners=_owner_shard(
                    detection_islands,
                    restored,
                    batch.read_bounds,
                ),
                published_owners=_owner_shard(
                    detection_islands,
                    published,
                    batch.read_bounds,
                ),
            )
            for batch in owner_batches
        ),
        partial(
            _decide_bridges,
            detection_source=detection_source,
            support_source=support_source,
            config=config,
            image_width=image_width,
        ),
        chunk_shape_yx=manifest.tile_core_shape_yx,
    )
    # Patches apply in canonical owner order, whichever batch decided them.
    patches = tuple(
        sorted(
            (patch for result in bridge_results for patch in result.patches),
            key=lambda patch: patch.label_value,
        )
    )
    wide_bridge_batches = _tile_batches(
        tuple(
            tile_request(
                partition,
                published_owners=_owner_shard(
                    detection_islands,
                    published,
                    partition.read_bounds,
                ),
                wide_owners=_wide_shard(wide, partition),
            )
            for partition in wide_cores
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    wide_bridge_results = map_round(
        executor,
        partial(
            _observe_wide_bridges,
            detection_source=detection_source,
            support_source=support_source,
            config=config,
        ),
        wide_bridge_batches,
        round_name="wide owner bridge",
        requirement=read_requirement(
            _TILE_BYTES_PER_READ_PIXEL,
            (partition.read_bounds for partition in wide_cores),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    bridge_cores = tuple(
        core for result in wide_bridge_results for core in result.cores
    )
    _require_every_core(
        wide_bridge_batches,
        (core.base.partition.tile_id for core in bridge_cores),
        question="bridges",
    )
    wide_shares, bridged_wide = decide_owner_bridges(bridge_cores)
    accepted = _accepted_islands(detection_islands, config)
    for product_name in _PUBLICATION_PRODUCT_NAMES:
        sink.initialize_product(
            product_name=product_name,
            dtype=_product_dtype(product_name),
        )
    publish_batches = _tile_batches(
        tuple(
            tile_request(
                partition,
                published_owners=_owner_shard(
                    detection_islands,
                    published,
                    partition.read_bounds,
                ),
                # Both shards are read-scoped, not core-scoped: an owner's
                # recovered support reaches the recovery radius beyond its
                # reconciled bounds, so an owner whose bounds stop just short
                # of this core can still hold pixels inside it. Naming only
                # the owners the core's bounds intersect would clear those
                # pixels, and the published support would move with the tiles.
                accepted_owners=_owner_shard(
                    detection_islands,
                    accepted,
                    partition.read_bounds,
                ),
                patches=_patch_shard(
                    patches,
                    partition,
                    image_width=image_width,
                ),
                wide_owners=_wide_shard(wide, partition),
                wide_share=wide_shares.get(partition.tile_id),
            )
            for partition in manifest.tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    publish_results = tuple(
        executor.map_batches(
            partial(
                _publish_batch,
                detection_source=detection_source,
                support_source=support_source,
                sink=sink,
                config=config,
                image_width=image_width,
            ),
            publish_batches,
            requirement=read_requirement(
                _TILE_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not publish_results:
        raise ValueError("executor returned no publication results")
    generation = sink.publish_generation(
        product_names=_PUBLICATION_PRODUCT_NAMES,
        chunks=(
            chunk
            for result in publish_results
            for chunk in result.product_chunks
        ),
    )
    owner_reads = tuple(
        result.maximum_owner_read_pixels
        for result in (
            *restore_results,
            *bridge_results,
            *split_results,
            *wide_bridge_results,
        )
    )
    return PublicationStageResult(
        generation=generation,
        accepted_island_count=len(accepted),
        restored_owner_count=len(restored),
        bridged_owner_count=len(patches) + len(bridged_wide),
        published_owner_count=len(published),
        partition_count=len(manifest.tiles),
        executor_task_count=(
            2 * len(owner_batches)
            + len(split_batches)
            + len(scan_batches)
            + len(wide_bridge_batches)
            + len(publish_batches)
        ),
        maximum_graph_width=max(
            len(owner_batches),
            len(split_batches),
            len(scan_batches),
            len(wide_bridge_batches),
            len(publish_batches),
        ),
        maximum_owner_read_pixels=max(owner_reads, default=0),
        maximum_batch_read_pixels=config.maximum_batch_read_pixels,
        owner_batch_count=len(owner_batches),
        wide_owner_count=len(wide),
    )

Components: deblending and fitting

hebog.algorithms.deblending

Deterministic bounded watershed deblending for compact islands.

CompactDeblendResult dataclass

Bounded region topology, not measured sources or Gaussian fits.

Source code in src/hebog/algorithms/deblending.py

@dataclass(frozen=True, slots=True)
class CompactDeblendResult:
    """Bounded region topology, not measured sources or Gaussian fits."""

    island_id: str
    status: Literal["single-region", "deblended"]
    regions: tuple[DeblendedRegion, ...]
    region_labels: npt.NDArray[np.int32]

    def compact_summary(self) -> CompactDeblendSummary:
        """Drop bounded region labels before returning through an executor."""
        return CompactDeblendSummary(
            island_id=self.island_id,
            status=self.status,
            regions=self.regions,
        )

compact_summary()

Drop bounded region labels before returning through an executor.

Source code in src/hebog/algorithms/deblending.py

def compact_summary(self) -> CompactDeblendSummary:
    """Drop bounded region labels before returning through an executor."""
    return CompactDeblendSummary(
        island_id=self.island_id,
        status=self.status,
        regions=self.regions,
    )

CompactDeblendSummary dataclass

Executor-safe compact region facts with no pixel label arrays.

Source code in src/hebog/algorithms/deblending.py

@dataclass(frozen=True, slots=True)
class CompactDeblendSummary:
    """Executor-safe compact region facts with no pixel label arrays."""

    island_id: str
    status: Literal["single-region", "deblended"]
    regions: tuple[DeblendedRegion, ...]

CompactIslandPixels dataclass

One admitted normalized island bounds region and exact membership.

Source code in src/hebog/algorithms/deblending.py

@dataclass(frozen=True, slots=True)
class CompactIslandPixels:
    """One admitted normalized island bounds region and exact membership."""

    island: DetectedIsland
    normalized_residual: npt.NDArray[np.float64]
    island_membership: npt.NDArray[np.bool_]

DeblendedRegion dataclass

One deterministic compact region for later scientific measurement.

Source code in src/hebog/algorithms/deblending.py

@dataclass(frozen=True, slots=True)
class DeblendedRegion:
    """One deterministic compact region for later scientific measurement."""

    region_id: str
    region_label: int
    island_id: str
    pixel_count: int
    bounds: ImageBounds
    peak_signal_to_noise: float
    peak_position_yx: tuple[int, int]
    first_pixel_yx: tuple[int, int]

deblend_compact_island(compact_island, config)

Split one admitted island into deterministic watershed regions.

The watershed floods the island's own intensity: every pixel belongs to the peak its steepest ascent reaches, and two peaks are judged at the pass between them, the highest level at which one connected part of the island holds both.

Source code in src/hebog/algorithms/deblending.py

def deblend_compact_island(
    compact_island: CompactIslandPixels,
    config: CompactDeblendConfig,
) -> CompactDeblendResult:
    """Split one admitted island into deterministic watershed regions.

    The watershed floods the island's own intensity: every pixel belongs to
    the peak its steepest ascent reaches, and two peaks are judged at the
    pass between them, the highest level at which one connected part of the
    island holds both.
    """
    normalized, membership = _validate_input(compact_island, config)
    bounds = compact_island.island.bounds
    peaks = _marker_positions(normalized, membership, bounds, config)
    if len(peaks) == 1:
        labels = np.where(membership, 1, 0).astype(np.int32)
    else:
        basins = _ascent_basins(normalized, membership, bounds, peaks)
        labels = _merge_shallow_regions(
            basins,
            normalized,
            bounds,
            peaks,
            config,
        )
    labels = _merge_undersized_regions(
        labels,
        normalized,
        minimum_region_pixels=config.minimum_region_pixels,
    )
    labels = np.asarray(labels, dtype=np.int32)
    labels.setflags(write=False)
    regions = _summarize_regions(
        compact_island.island,
        labels,
        normalized,
    )
    return CompactDeblendResult(
        island_id=compact_island.island.island_id,
        status="deblended" if len(regions) > 1 else "single-region",
        regions=regions,
        region_labels=labels,
    )

hebog.algorithms.component_topology

Bounded component topology within connected detection islands.

DeblendedComponentTopology dataclass

Component owners plus explicit bounded-deblending disposition.

Source code in src/hebog/algorithms/component_topology.py

@dataclass(frozen=True, slots=True)
class DeblendedComponentTopology:
    """Component owners plus explicit bounded-deblending disposition."""

    direct_component_labels: npt.NDArray[np.int32]
    measurement_component_labels: npt.NDArray[np.int32]
    deblended_parent_count: int
    deferred_parent_count: int

ParentComponentMembership dataclass

One parent's deblended components, in that parent's own windows.

The labels are local to the parent, numbered from one in the order the reviewed watershed produced them. A caller that deblends one parent per task offsets them by the components every earlier parent produced, which is what :func:deblend_component_topology does in one pass.

Source code in src/hebog/algorithms/component_topology.py

@dataclass(frozen=True, slots=True)
class ParentComponentMembership:
    """One parent's deblended components, in that parent's own windows.

    The labels are local to the parent, numbered from one in the order the
    reviewed watershed produced them. A caller that deblends one parent per
    task offsets them by the components every earlier parent produced, which
    is what :func:`deblend_component_topology` does in one pass.
    """

    component_count: int
    direct_labels: npt.NDArray[np.int32]
    measurement_labels: npt.NDArray[np.int32]
    deblended: bool
    deferred: bool

deblend_component_topology(normalized_residual, direct_component_labels, measurement_component_labels, valid_pixels, config)

Deblend admitted parents while preserving their complete support.

Direct connected islands remain the parent/support topology. Within each admitted parent, an intensity watershed defines Gaussian-component ownership: peaks split at the pass between them, and each component holds the pixels that rise to its own peak. Measurement pixels keep their original parent membership and are assigned to the nearest new direct seed with canonical tie-breaking. Parents above either hard compact-work bound remain one explicit deferred component; this bounded helper never drops their science.

Source code in src/hebog/algorithms/component_topology.py

def deblend_component_topology(
    normalized_residual: npt.ArrayLike,
    direct_component_labels: npt.ArrayLike,
    measurement_component_labels: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    config: CompactDeblendConfig,
) -> DeblendedComponentTopology:
    """Deblend admitted parents while preserving their complete support.

    Direct connected islands remain the parent/support topology. Within each
    admitted parent, an intensity watershed defines Gaussian-component
    ownership: peaks split at the pass between them, and each component holds
    the pixels that rise to its own peak. Measurement pixels keep their
    original parent membership and are assigned to the nearest new direct
    seed with canonical tie-breaking. Parents above either hard compact-work
    bound remain one explicit deferred component; this bounded helper never
    drops their science.
    """
    normalized, direct, measurement, valid = _validated_inputs(
        normalized_residual,
        direct_component_labels,
        measurement_component_labels,
        valid_pixels,
    )
    output_direct = np.zeros(direct.shape, dtype=np.int32)
    output_measurement = np.zeros(measurement.shape, dtype=np.int32)
    next_label = 1
    deblended_parent_count = 0
    deferred_parent_count = 0
    measurement_records = {
        label: (bounds, slices)
        for label, bounds, slices, _ in _parent_records(measurement)
    }
    for parent_label, bounds, slices, first_pixel in _parent_records(direct):
        measurement_bounds, measurement_slices = measurement_records[
            parent_label
        ]
        membership = deblend_parent_components(
            normalized[slices],
            direct[slices] == parent_label,
            measurement[measurement_slices] == parent_label,
            valid[measurement_slices],
            parent_label=parent_label,
            direct_bounds=bounds,
            measurement_bounds=measurement_bounds,
            image_shape_yx=direct.shape,
            first_pixel_yx=first_pixel,
            config=config,
        )
        output_direct[slices] += np.where(
            membership.direct_labels > 0,
            membership.direct_labels + next_label - 1,
            0,
        ).astype(np.int32, copy=False)
        output_measurement[measurement_slices] += np.where(
            membership.measurement_labels > 0,
            membership.measurement_labels + next_label - 1,
            0,
        ).astype(np.int32, copy=False)
        next_label += membership.component_count
        deblended_parent_count += int(membership.deblended)
        deferred_parent_count += int(membership.deferred)
    if not np.array_equal(output_direct > 0, direct > 0):
        raise ValueError("component deblending changed direct support")
    if not np.array_equal(output_measurement > 0, measurement > 0):
        raise ValueError("component deblending changed measurement support")
    output_identities = set(np.unique(output_direct[output_direct > 0]))
    if output_identities != set(
        np.unique(output_measurement[output_measurement > 0])
    ):
        raise ValueError("deblended component identities are inconsistent")
    output_direct.setflags(write=False)
    output_measurement.setflags(write=False)
    return DeblendedComponentTopology(
        direct_component_labels=output_direct,
        measurement_component_labels=output_measurement,
        deblended_parent_count=deblended_parent_count,
        deferred_parent_count=deferred_parent_count,
    )

deblend_parent_components(normalized_window, direct_membership, measurement_membership, valid_window, *, parent_label, direct_bounds, measurement_bounds, image_shape_yx, first_pixel_yx, config)

Deblend one admitted parent inside the windows that hold it.

Every decision needs the parent's complete support and nothing beyond it, so one task can evaluate one parent exactly. A parent above either hard compact-work bound stays one explicit deferred component rather than losing its science, and so does a parent whose peak never reaches the detection threshold.

Source code in src/hebog/algorithms/component_topology.py

def deblend_parent_components(  # noqa: PLR0913
    normalized_window: npt.NDArray[np.float64],
    direct_membership: npt.NDArray[np.bool_],
    measurement_membership: npt.NDArray[np.bool_],
    valid_window: npt.NDArray[np.bool_],
    *,
    parent_label: int,
    direct_bounds: ImageBounds,
    measurement_bounds: ImageBounds,
    image_shape_yx: tuple[int, int],
    first_pixel_yx: tuple[int, int],
    config: CompactDeblendConfig,
) -> ParentComponentMembership:
    """Deblend one admitted parent inside the windows that hold it.

    Every decision needs the parent's complete support and nothing beyond it,
    so one task can evaluate one parent exactly. A parent above either hard
    compact-work bound stays one explicit deferred component rather than
    losing its science, and so does a parent whose peak never reaches the
    detection threshold.
    """
    image_height, image_width = image_shape_yx
    measurement_labels = np.zeros(
        measurement_bounds.shape_yx,
        dtype=np.int32,
    )
    direct_pixels = int(np.count_nonzero(direct_membership))
    if parent_is_deferred(direct_bounds, direct_pixels, config):
        return _one_parent_component(
            direct_membership,
            measurement_membership,
            direct_bounds=direct_bounds,
            measurement_labels=measurement_labels,
            deferred=True,
        )
    peak_linear = int(
        np.argmax(np.where(direct_membership, normalized_window, -np.inf))
    )
    peak_local = np.unravel_index(peak_linear, direct_membership.shape)
    if (
        float(normalized_window[peak_local])
        <= config.minimum_peak_signal_to_noise
    ):
        return _one_parent_component(
            direct_membership,
            measurement_membership,
            direct_bounds=direct_bounds,
            measurement_labels=measurement_labels,
            deferred=False,
        )
    result = deblend_compact_island(
        CompactIslandPixels(
            island=DetectedIsland(
                island_id=f"component-parent-{parent_label:08d}",
                global_label=parent_label,
                pixel_count=direct_pixels,
                bounds=direct_bounds,
                peak_signal_to_noise=float(normalized_window[peak_local]),
                peak_position_yx=(
                    direct_bounds.y_start + int(peak_local[0]),
                    direct_bounds.x_start + int(peak_local[1]),
                ),
                first_pixel_yx=first_pixel_yx,
                touches_image_edge=(
                    direct_bounds.y_start == 0
                    or direct_bounds.x_start == 0
                    or direct_bounds.y_stop == image_height
                    or direct_bounds.x_stop == image_width
                ),
            ),
            normalized_residual=normalized_window,
            island_membership=direct_membership,
        ),
        config,
    )
    direct_labels = np.asarray(result.region_labels, dtype=np.int32)
    if len(result.regions) == 1:
        measurement_labels[measurement_membership] = 1
        return ParentComponentMembership(
            component_count=1,
            direct_labels=direct_labels,
            measurement_labels=measurement_labels,
            deblended=False,
            deferred=False,
        )
    seed_labels = np.zeros(measurement_bounds.shape_yx, dtype=np.int32)
    seed_labels[
        slice(
            direct_bounds.y_start - measurement_bounds.y_start,
            direct_bounds.y_stop - measurement_bounds.y_start,
        ),
        slice(
            direct_bounds.x_start - measurement_bounds.x_start,
            direct_bounds.x_stop - measurement_bounds.x_start,
        ),
    ] = direct_labels
    measurement_labels += _assign_parent_measurement_support(
        seed_labels,
        measurement_membership,
        valid_window,
    )
    return ParentComponentMembership(
        component_count=len(result.regions),
        direct_labels=direct_labels,
        measurement_labels=measurement_labels,
        deblended=True,
        deferred=False,
    )

parent_is_deferred(direct_bounds, direct_pixel_count, config)

Return whether a parent passes either hard compact-work bound.

Such a parent stays one explicit deferred component. That decision needs only the parent's bounds and size, and publishing it needs none of its pixels, so no task has to read a deferred parent's window.

Examples:

>>> bounds = CompactDeblendConfig(5.0, 2, 1.0, 7, 100, 250)
>>> parent_is_deferred(ImageBounds(0, 10, 0, 10), 60, bounds)
False
>>> parent_is_deferred(ImageBounds(0, 3, 0, 90), 60, bounds)
True
>>> parent_is_deferred(ImageBounds(0, 10, 0, 10), 101, bounds)
True

Source code in src/hebog/algorithms/component_topology.py

def parent_is_deferred(
    direct_bounds: ImageBounds,
    direct_pixel_count: int,
    config: CompactDeblendConfig,
) -> bool:
    """Return whether a parent passes either hard compact-work bound.

    Such a parent stays one explicit deferred component. That decision needs
    only the parent's bounds and size, and publishing it needs none of its
    pixels, so no task has to read a deferred parent's window.

    Examples:
        >>> bounds = CompactDeblendConfig(5.0, 2, 1.0, 7, 100, 250)
        >>> parent_is_deferred(ImageBounds(0, 10, 0, 10), 60, bounds)
        False
        >>> parent_is_deferred(ImageBounds(0, 3, 0, 90), 60, bounds)
        True
        >>> parent_is_deferred(ImageBounds(0, 10, 0, 10), 101, bounds)
        True
    """
    height, width = direct_bounds.shape_yx
    return (
        direct_pixel_count > config.maximum_compact_island_pixels
        or height * width > config.maximum_compact_bounds_pixels
    )

hebog.data_models.measurement

Compact scheduler-safe scientific measurement records.

CompactMeasurementGeometry dataclass

Reviewed local solid angles and noise correlation for compact fits.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class CompactMeasurementGeometry:
    """Reviewed local solid angles and noise correlation for compact fits."""

    pixel_solid_angle_steradians: float
    restoring_beam_solid_angle_steradians: float
    restoring_beam_covariance_pixels_squared: (
        tuple[float, float, float] | None
    ) = None
    noise_correlation_covariance_pixels_squared: (
        tuple[float, float, float] | None
    ) = None

    def __post_init__(self) -> None:
        """Require explicit finite positive pixel and beam solid angles."""
        if (
            not isfinite(self.pixel_solid_angle_steradians)
            or self.pixel_solid_angle_steradians <= 0
        ):
            raise ValueError("pixel solid angle must be finite and positive")
        if (
            not isfinite(self.restoring_beam_solid_angle_steradians)
            or self.restoring_beam_solid_angle_steradians <= 0
        ):
            raise ValueError("beam solid angle must be finite and positive")
        covariances = (
            (
                "restoring beam covariance",
                self.restoring_beam_covariance_pixels_squared,
            ),
            (
                "noise correlation covariance",
                self.noise_correlation_covariance_pixels_squared,
            ),
        )
        for name, covariance in covariances:
            if covariance is None:
                continue
            covariance_xx, covariance_xy, covariance_yy = covariance
            if not all(isfinite(value) for value in covariance):
                raise ValueError(f"{name} must be finite")
            if (
                covariance_xx <= 0
                or covariance_yy <= 0
                or covariance_xx * covariance_yy
                - covariance_xy * covariance_xy
                <= 0
            ):
                raise ValueError(f"{name} must be positive definite")

__post_init__()

Require explicit finite positive pixel and beam solid angles.

Source code in src/hebog/data_models/measurement.py

def __post_init__(self) -> None:
    """Require explicit finite positive pixel and beam solid angles."""
    if (
        not isfinite(self.pixel_solid_angle_steradians)
        or self.pixel_solid_angle_steradians <= 0
    ):
        raise ValueError("pixel solid angle must be finite and positive")
    if (
        not isfinite(self.restoring_beam_solid_angle_steradians)
        or self.restoring_beam_solid_angle_steradians <= 0
    ):
        raise ValueError("beam solid angle must be finite and positive")
    covariances = (
        (
            "restoring beam covariance",
            self.restoring_beam_covariance_pixels_squared,
        ),
        (
            "noise correlation covariance",
            self.noise_correlation_covariance_pixels_squared,
        ),
    )
    for name, covariance in covariances:
        if covariance is None:
            continue
        covariance_xx, covariance_xy, covariance_yy = covariance
        if not all(isfinite(value) for value in covariance):
            raise ValueError(f"{name} must be finite")
        if (
            covariance_xx <= 0
            or covariance_yy <= 0
            or covariance_xx * covariance_yy
            - covariance_xy * covariance_xy
            <= 0
        ):
            raise ValueError(f"{name} must be positive definite")

GaussianMomentInitializer dataclass

Brightness-weighted pixel-space initializer for nonlinear fitting.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class GaussianMomentInitializer:
    """Brightness-weighted pixel-space initializer for nonlinear fitting."""

    amplitude_jy_per_beam: float
    centroid_xy: tuple[float, float]
    covariance_xx_pixels_squared: float
    covariance_xy_pixels_squared: float
    covariance_yy_pixels_squared: float
    major_sigma_pixels: float
    minor_sigma_pixels: float
    major_axis_angle_degrees: float

MomentTarget dataclass

Canonical island or deblended-region measurement identity.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class MomentTarget:
    """Canonical island or deblended-region measurement identity."""

    object_kind: Literal["island", "deblended-region"]
    object_id: str
    island_id: str
    pixel_count: int

OwnedPixelPhotometry dataclass

Finite owned-pixel photometry distinct from a fitted Gaussian.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class OwnedPixelPhotometry:
    """Finite owned-pixel photometry distinct from a fitted Gaussian."""

    peak_brightness_jy_per_beam: float
    peak_position_xy: tuple[int, int]
    owned_pixel_integrated_flux_jy: float
    local_rms_jy_per_beam: float
    mean_brightness_jy_per_beam: float

ShapeUnavailableMomentMeasurement dataclass

Valid photometry whose pixel support cannot define a 2-D ellipse.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class ShapeUnavailableMomentMeasurement:
    """Valid photometry whose pixel support cannot define a 2-D ellipse."""

    target: MomentTarget
    photometry: OwnedPixelPhotometry
    reason: Literal["underdetermined-region", "singular-covariance"]
    status: Literal["shape-unavailable"] = "shape-unavailable"

UnavailableMomentMeasurement dataclass

A target for which valid photometry cannot be reported.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class UnavailableMomentMeasurement:
    """A target for which valid photometry cannot be reported."""

    target: MomentTarget
    reason: Literal[
        "non-finite-owned-pixels",
        "non-positive-measurement",
    ]
    status: Literal["unavailable"] = "unavailable"

ValidMomentMeasurement dataclass

Complete owned-pixel photometry and nonsingular shape initializer.

Source code in src/hebog/data_models/measurement.py

@dataclass(frozen=True, slots=True)
class ValidMomentMeasurement:
    """Complete owned-pixel photometry and nonsingular shape initializer."""

    target: MomentTarget
    photometry: OwnedPixelPhotometry
    initializer: GaussianMomentInitializer
    status: Literal["valid"] = "valid"

hebog.algorithms.measurement

Readable vectorized moment oracle for exact compact memberships.

CompactMomentInput

Bases: Protocol

Structural exact-label input kept inside one coarse worker task.

Source code in src/hebog/algorithms/measurement.py

class CompactMomentInput(Protocol):
    """Structural exact-label input kept inside one coarse worker task."""

    @property
    def island(self) -> DetectedIsland:
        """Return the reconciled parent island."""
        ...

    @property
    def regions(self) -> tuple[DeblendedRegion, ...]:
        """Return canonical deblended-region summaries."""
        ...

    @property
    def physical_residual(self) -> npt.NDArray[np.float64]:
        """Return the bounded physical residual plane."""
        ...

    @property
    def rms(self) -> npt.NDArray[np.float64]:
        """Return the aligned bounded RMS plane."""
        ...

    @property
    def valid_pixels(self) -> npt.NDArray[np.bool_]:
        """Return the aligned scientific-validity mask."""
        ...

    @property
    def region_labels(self) -> npt.NDArray[np.int32]:
        """Return exact zero-background deblended labels."""
        ...

island property

Return the reconciled parent island.

physical_residual property

Return the bounded physical residual plane.

region_labels property

Return exact zero-background deblended labels.

regions property

Return canonical deblended-region summaries.

rms property

Return the aligned bounded RMS plane.

valid_pixels property

Return the aligned scientific-validity mask.

fitted_gaussian_integrated_flux_jy(*, amplitude_jy_per_beam, major_sigma_pixels, minor_sigma_pixels, geometry)

Convert one fitted Gaussian's infinite-plane area to integrated Jy.

Source code in src/hebog/algorithms/measurement.py

def fitted_gaussian_integrated_flux_jy(
    *,
    amplitude_jy_per_beam: float,
    major_sigma_pixels: float,
    minor_sigma_pixels: float,
    geometry: CompactMeasurementGeometry,
) -> float:
    """Convert one fitted Gaussian's infinite-plane area to integrated Jy."""
    values = (
        amplitude_jy_per_beam,
        major_sigma_pixels,
        minor_sigma_pixels,
    )
    if any(not isfinite(value) or value <= 0 for value in values):
        raise ValueError("Gaussian amplitude and sigma axes must be positive")
    gaussian_area_pixels = 2.0 * pi * major_sigma_pixels * minor_sigma_pixels
    return (
        amplitude_jy_per_beam
        * gaussian_area_pixels
        * geometry.pixel_solid_angle_steradians
        / geometry.restoring_beam_solid_angle_steradians
    )

gaussian_beam_solid_angle_steradians(*, major_fwhm_degrees, minor_fwhm_degrees)

Return an elliptical Gaussian beam area from angular FWHM axes.

Source code in src/hebog/algorithms/measurement.py

def gaussian_beam_solid_angle_steradians(
    *,
    major_fwhm_degrees: float,
    minor_fwhm_degrees: float,
) -> float:
    """Return an elliptical Gaussian beam area from angular FWHM axes."""
    if (
        not isfinite(major_fwhm_degrees)
        or not isfinite(minor_fwhm_degrees)
        or major_fwhm_degrees <= 0
        or minor_fwhm_degrees <= 0
    ):
        raise ValueError("beam FWHM axes must be finite and positive")
    return (
        pi
        * radians(major_fwhm_degrees)
        * radians(minor_fwhm_degrees)
        / (4.0 * log(2.0))
    )

measure_compact_moments(compact, geometry, config)

Measure one island and its exact regions without pixel-wise loops.

Source code in src/hebog/algorithms/measurement.py

def measure_compact_moments(
    compact: CompactMomentInput,
    geometry: CompactMeasurementGeometry,
    config: CompactMomentConfig,
) -> tuple[CompactMomentMeasurement, ...]:
    """Measure one island and its exact regions without pixel-wise loops."""
    residual, rms, validity, labels = _validated_arrays(compact)
    targets = _targets_and_masks(compact, labels)
    context = _MomentContext(
        residual=residual,
        rms=rms,
        validity=validity,
        origin_yx=(
            getattr(compact, "array_bounds", compact.island.bounds).y_start,
            getattr(compact, "array_bounds", compact.island.bounds).x_start,
        ),
        geometry=geometry,
        config=config,
    )
    return tuple(
        _measure_target(
            target,
            membership,
            context,
        )
        for target, membership in targets
    )

hebog.data_models.fitting

Compact scheduler-safe Gaussian fitting records.

FailedCompactGaussianFit dataclass

An attempted fit that did not yield acceptable parameters.

Diagnostics are absent when numerical decomposition prevented a complete validated report; unknown work counts or residuals are not fabricated.

Source code in src/hebog/data_models/fitting.py

@dataclass(frozen=True, slots=True)
class FailedCompactGaussianFit:
    """An attempted fit that did not yield acceptable parameters.

    Diagnostics are absent when numerical decomposition prevented a complete
    validated report; unknown work counts or residuals are not fabricated.
    """

    moment: ValidMomentMeasurement
    reason: Literal[
        "fit-non-convergence",
        "fit-invalid-result",
        "fit-model-inadequate",
        "fit-linear-algebra-failure",
    ]
    diagnostics: GaussianFitDiagnostics | None
    quality_flags: tuple[str, ...]
    status: Literal["failed"] = "failed"

FittedGaussianPixelParameters dataclass

One fitted Gaussian in global pixel coordinates and physical units.

Source code in src/hebog/data_models/fitting.py

@dataclass(frozen=True, slots=True)
class FittedGaussianPixelParameters:
    """One fitted Gaussian in global pixel coordinates and physical units."""

    amplitude_jy_per_beam: float
    centroid_xy: tuple[float, float]
    major_sigma_pixels: float
    minor_sigma_pixels: float
    major_axis_angle_degrees: float
    integrated_flux_jy: float
    local_rms_jy_per_beam: float

GaussianFitDiagnostics dataclass

Bounded optimizer work and weighted-residual evidence.

Source code in src/hebog/data_models/fitting.py

@dataclass(frozen=True, slots=True)
class GaussianFitDiagnostics:
    """Bounded optimizer work and weighted-residual evidence."""

    converged: bool
    function_evaluations: int
    chi_squared: float
    degrees_of_freedom: int
    reduced_chi_squared: float | None
    parameters_at_bound: bool
    model_identity: Literal[
        "free-elliptical",
        "beam-constrained",
    ] = "free-elliptical"
    bound_parameters: tuple[str, ...] = ()
    relative_bound_distances: tuple[tuple[str, float], ...] = ()
    minimum_relative_bound_distance: float | None = None
    information_condition_number: float | None = None
    covariance_parameterization: Literal[
        "optimizer", "cartesian-precision"
    ] = "optimizer"
    visible_model_fraction: float | None = None
    retained_pixel_count: int = 0
    retained_bounds_yx: tuple[int, int, int, int] | None = None
    fallback_reason: (
        Literal[
            "free-model-bound-contact",
            "free-model-ill-conditioned",
            "free-model-not-significantly-extended",
            "free-model-non-convergence",
            "free-model-invalid-result",
        ]
        | None
    ) = None
    point_estimator: Literal["diagonal-weighted", "correlated-gls"] = (
        "diagonal-weighted"
    )
    point_estimator_fallback_reason: (
        Literal[
            "correlation-model-unavailable",
            "correlation-factorization-failed",
            "correlation-conditioning-failed",
            "correlation-ill-conditioned",
            "retained-region-exceeds-gls-limit",
        ]
        | None
    ) = None
    rejected_model_identity: (
        Literal[
            "free-elliptical",
            "beam-constrained",
        ]
        | None
    ) = None
    rejected_model_bound_parameters: tuple[str, ...] = ()

GaussianFitUncertainty dataclass

One-sigma position and flux covariance estimate for a fitted model.

Source code in src/hebog/data_models/fitting.py

@dataclass(frozen=True, slots=True)
class GaussianFitUncertainty:
    """One-sigma position and flux covariance estimate for a fitted model."""

    amplitude_error_jy_per_beam: float
    centroid_covariance_xx_pixels_squared: float
    centroid_covariance_xy_pixels_squared: float
    centroid_covariance_yy_pixels_squared: float
    integrated_flux_error_jy: float
    integrated_flux_bias_correction_sigma: float = 0.0
    amplitude_integrated_flux_covariance_jy_squared_per_beam: float | None = (
        None
    )
    shape_parameter_covariance: (
        tuple[float, float, float, float, float, float] | None
    ) = None
    """Upper triangle for ordered major/minor sigma and angle in radians."""

shape_parameter_covariance = None class-attribute instance-attribute

Upper triangle for ordered major/minor sigma and angle in radians.

UnavailableCompactGaussianFit dataclass

A region whose measurement cannot initialize the fit-all path.

Source code in src/hebog/data_models/fitting.py

@dataclass(frozen=True, slots=True)
class UnavailableCompactGaussianFit:
    """A region whose measurement cannot initialize the fit-all path."""

    moment: CompactMomentMeasurement
    reason: Literal[
        "non-finite-owned-pixels",
        "non-positive-measurement",
        "singular-covariance",
        "underdetermined-region",
        "joint-fit-work-limit",
        "joint-peer-unavailable",
    ]
    quality_flags: tuple[str, ...]
    status: Literal["unavailable"] = "unavailable"

ValidCompactGaussianFit dataclass

A converged fit retaining its independent moment oracle.

Source code in src/hebog/data_models/fitting.py

@dataclass(frozen=True, slots=True)
class ValidCompactGaussianFit:
    """A converged fit retaining its independent moment oracle."""

    moment: ValidMomentMeasurement
    parameters: FittedGaussianPixelParameters
    uncertainty: GaussianFitUncertainty | None
    diagnostics: GaussianFitDiagnostics
    quality_flags: tuple[str, ...]
    status: Literal["valid"] = "valid"

hebog.algorithms.fitting

Bounded joint compact Gaussian fitting with SciPy least squares.

MAXIMUM_JOINT_FIT_JACOBIAN_ELEMENTS = 1000000 module-attribute

Parameters times fitted pixels one joint fit admits.

MAXIMUM_JOINT_FIT_PARAMETERS = 96 module-attribute

Free parameters one joint fit admits: sixteen fixed-background ellipses.

fit_compact_gaussian_mixture(compact, moments, geometry, config, *, maximum_parameters=MAXIMUM_JOINT_FIT_PARAMETERS, maximum_jacobian_elements=MAXIMUM_JOINT_FIT_JACOBIAN_ELEMENTS)

Fit bounded neighbouring components jointly on original pixels.

This extends the existing free-ellipse solver, derivative, noise transform and covariance machinery. It does not add components or change detection. Callers supply one bounded parent plus context, excluding foreign owners. Work admission precedes allocation of the joint Jacobian. The background is fixed at the caller's independently estimated background map.

Source code in src/hebog/algorithms/fitting.py

def fit_compact_gaussian_mixture(  # noqa: PLR0913
    compact: CompactMomentInput,
    moments: tuple[CompactMomentMeasurement, ...],
    geometry: CompactMeasurementGeometry,
    config: CompactGaussianFitConfig,
    *,
    maximum_parameters: int = MAXIMUM_JOINT_FIT_PARAMETERS,
    maximum_jacobian_elements: int = MAXIMUM_JOINT_FIT_JACOBIAN_ELEMENTS,
) -> tuple[CompactGaussianFitResult, ...]:
    """Fit bounded neighbouring components jointly on original pixels.

    This extends the existing free-ellipse solver, derivative, noise transform
    and covariance machinery. It does not add components or change detection.
    Callers supply one bounded parent plus context, excluding foreign owners.
    Work admission precedes allocation of the joint Jacobian. The background
    is fixed at the caller's independently estimated background map.
    """
    if (
        type(maximum_parameters) is not int
        or type(maximum_jacobian_elements) is not int
        or maximum_parameters < len(_FREE_FIXED_BACKGROUND_PARAMETER_NAMES)
        or maximum_jacobian_elements < 1
    ):
        raise ValueError(
            "joint fit work limits must be positive and admit six parameters"
        )
    by_id = {moment.target.object_id: moment for moment in moments}
    if len(by_id) != len(moments) or set(by_id) != {
        region.region_id for region in compact.regions
    }:
        raise ValueError(
            "joint moments must identify every component exactly once"
        )
    ordered = tuple(by_id[region.region_id] for region in compact.regions)
    if not ordered:
        return ()
    available = tuple(_unavailable_fit(moment, config) for moment in ordered)
    if any(item is not None for item in available):
        return tuple(
            absent
            if absent is not None
            else UnavailableCompactGaussianFit(
                moment=moment,
                reason="joint-peer-unavailable",
                quality_flags=("fit-unavailable", "joint-peer-unavailable"),
            )
            for moment, absent in zip(ordered, available, strict=True)
        )
    valid = (
        np.asarray(compact.valid_pixels)
        & np.isfinite(compact.physical_residual)
        & np.isfinite(compact.rms)
        & (compact.rms > 0.0)
    )
    # Likelihood support is independent of the much larger adequacy halo.
    # All neighbours contribute their complete models at every retained
    # pixel; an ownership boundary never truncates an individual Gaussian.
    if config.pixel_support == "owned-region":
        valid &= np.isin(
            compact.region_labels,
            tuple(region.region_label for region in compact.regions),
        )
    parameter_count = len(_FREE_FIXED_BACKGROUND_PARAMETER_NAMES) * len(
        ordered
    )
    pixel_count = int(np.count_nonzero(valid))
    if not joint_fit_admits(
        len(ordered),
        pixel_count,
        maximum_parameters=maximum_parameters,
        maximum_jacobian_elements=maximum_jacobian_elements,
    ):
        return tuple(
            UnavailableCompactGaussianFit(
                moment=moment,
                reason="joint-fit-work-limit",
                quality_flags=("fit-unavailable", "joint-fit-work-limit"),
            )
            for moment in ordered
        )
    if pixel_count <= parameter_count:
        return tuple(
            UnavailableCompactGaussianFit(
                moment=moment,
                reason="underdetermined-region",
                quality_flags=("fit-unavailable",),
            )
            for moment in ordered
        )
    bounds = getattr(compact, "array_bounds", compact.island.bounds)
    samples = _fit_samples_from_mask(compact, valid, bounds, geometry, config)
    contexts = tuple(
        _FitPublicationContext(
            compact,
            region,
            cast(ValidMomentMeasurement, moment),
            geometry,
            config,
        )
        for region, moment in zip(compact.regions, ordered, strict=True)
    )
    initial_bounds = tuple(
        _mixture_initial_bounds(context.moment, context.region, samples)
        for context in contexts
    )
    return _solve_joint_components(samples, contexts, initial_bounds)

joint_fit_admits(component_count, pixel_count, *, maximum_parameters=MAXIMUM_JOINT_FIT_PARAMETERS, maximum_jacobian_elements=MAXIMUM_JOINT_FIT_JACOBIAN_ELEMENTS)

Return whether one joint fit admits this many components and pixels.

The work is a Jacobian of six parameters per component by the fitted pixels, refused before it is allocated. With owned-region pixel support the fitted pixels are the components' own, so a caller that counts those can apply this rule before any fit runs.

Source code in src/hebog/algorithms/fitting.py

def joint_fit_admits(
    component_count: int,
    pixel_count: int,
    *,
    maximum_parameters: int = MAXIMUM_JOINT_FIT_PARAMETERS,
    maximum_jacobian_elements: int = MAXIMUM_JOINT_FIT_JACOBIAN_ELEMENTS,
) -> bool:
    """Return whether one joint fit admits this many components and pixels.

    The work is a Jacobian of six parameters per component by the fitted
    pixels, refused before it is allocated. With ``owned-region`` pixel
    support the fitted pixels are the components' own, so a caller that
    counts those can apply this rule before any fit runs.
    """
    parameter_count = len(_FREE_FIXED_BACKGROUND_PARAMETER_NAMES) * (
        component_count
    )
    return (
        parameter_count <= maximum_parameters
        and parameter_count * pixel_count <= maximum_jacobian_elements
    )

hebog.algorithms.component_measurement

Bounded original-pixel Gaussian measurements for detected components.

ComponentGroupingEvidence dataclass

Worker-local merge evidence, linear in the bounded group population.

Source code in src/hebog/algorithms/component_measurement.py

@dataclass(frozen=True, slots=True)
class ComponentGroupingEvidence:
    """Worker-local merge evidence, linear in the bounded group population."""

    reason: AssociationEvidenceKind
    scale_ids: tuple[int, ...]
    component_labels: frozenset[int]
    protected_labels: frozenset[int] = frozenset()

ComponentMeasurements dataclass

Fits retain detection labels; compact groups constrain association.

Source code in src/hebog/algorithms/component_measurement.py

@dataclass(frozen=True, slots=True)
class ComponentMeasurements:
    """Fits retain detection labels; compact groups constrain association."""

    fits: tuple[tuple[int, CompactGaussianFitResult], ...]
    compact_groups: tuple[frozenset[int], ...]
    deferred_parent_count: int
    # The whole-plane oracle measures the support as it goes and returns it;
    # the tiled reconciliation reduces records only, and the cores published
    # the same plane for a later pass to read by window.
    measurement_support: np.ndarray | None = None
    extended_groups: tuple[frozenset[int], ...] = ()
    proposed_compact_groups: tuple[frozenset[int], ...] = ()
    grouping_evidence: tuple[ComponentGroupingEvidence, ...] = ()

FitParentMeasurement dataclass

One fit parent's bounded measurement outputs.

Every field is a record, so a caller that measures one fit parent per task returns nothing that grows with the parent's area. The persistent measurement support a measured parent contributes is not among them: :func:fit_parent_measurement_support derives it from the planes in support_bounds, so whoever writes the plane can derive it again.

Source code in src/hebog/algorithms/component_measurement.py

@dataclass(frozen=True, slots=True)
class FitParentMeasurement:
    """One fit parent's bounded measurement outputs.

    Every field is a record, so a caller that measures one fit parent per
    task returns nothing that grows with the parent's area. The persistent
    measurement support a measured parent contributes is not among them:
    :func:`fit_parent_measurement_support` derives it from the planes in
    ``support_bounds``, so whoever writes the plane can derive it again.
    """

    fits: tuple[tuple[int, CompactGaussianFitResult], ...] = ()
    compact_groups: tuple[frozenset[int], ...] = ()
    extended_groups: tuple[frozenset[int], ...] = ()
    evidence: tuple[ComponentGroupingEvidence, ...] = ()
    deferred: bool = False
    support_bounds: ImageBounds | None = None

SupportFeatureGroups dataclass

Extended groups and evidence one support feature contributed.

Source code in src/hebog/algorithms/component_measurement.py

@dataclass(frozen=True, slots=True)
class SupportFeatureGroups:
    """Extended groups and evidence one support feature contributed."""

    extended_groups: tuple[frozenset[int], ...] = ()
    evidence: tuple[ComponentGroupingEvidence, ...] = ()

compact_window_is_admitted(window, *, maximum_bounds_pixels)

Return whether the reviewed compact bound admits one window.

A fit parent or support feature whose window the bound refuses is ADR-008 T3 work: the parent is deferred rather than fitted on truncated pixels, and the feature is left ungrouped, so neither window is read.

Source code in src/hebog/algorithms/component_measurement.py

def compact_window_is_admitted(
    window: ImageBounds,
    *,
    maximum_bounds_pixels: int,
) -> bool:
    """Return whether the reviewed compact bound admits one window.

    A fit parent or support feature whose window the bound refuses is
    ADR-008 T3 work: the parent is deferred rather than fitted on truncated
    pixels, and the feature is left ungrouped, so neither window is read.
    """
    return int(np.prod(window.shape_yx)) <= maximum_bounds_pixels

fit_parent_margin_pixels(fit_config, atrous_plan)

Return the context a fit parent reads beyond its own support.

Source code in src/hebog/algorithms/component_measurement.py

def fit_parent_margin_pixels(
    fit_config: CompactGaussianFitConfig,
    atrous_plan: ResidualAtrousPlan,
) -> int:
    """Return the context a fit parent reads beyond its own support."""
    return max(
        ceil(fit_config.context_margin_pixels),
        atrous_plan.maximum_halo_pixels,
        _adequacy_filter_bank(atrous_plan).maximum_halo_pixels,
    )

fit_parent_measurement_support(residual_window, rms_window, valid_window, fit_parent_window, *, parent_index, atrous_plan, minimum_support_fraction, detection_sigma, island_sigma, minimum_pixels)

Return the persistent measurement support one measured parent adds.

The windows are the parent's own read, its support plus the reviewed context margin. The support depends on these planes and the reviewed thresholds only, never on the parent's fit, so a core that holds part of the window derives exactly what the measuring task would have returned. Callers combine parents with a boolean OR, and call this only for a parent that :func:measure_fit_parent_components did not defer.

Source code in src/hebog/algorithms/component_measurement.py

def fit_parent_measurement_support(  # noqa: PLR0913
    residual_window: np.ndarray,
    rms_window: np.ndarray,
    valid_window: np.ndarray,
    fit_parent_window: np.ndarray,
    *,
    parent_index: int,
    atrous_plan: ResidualAtrousPlan,
    minimum_support_fraction: float,
    detection_sigma: float,
    island_sigma: float,
    minimum_pixels: int,
) -> np.ndarray:
    """Return the persistent measurement support one measured parent adds.

    The windows are the parent's own read, its support plus the reviewed
    context margin. The support depends on these planes and the reviewed
    thresholds only, never on the parent's fit, so a core that holds part of
    the window derives exactly what the measuring task would have returned.
    Callers combine parents with a boolean OR, and call this only for a
    parent that :func:`measure_fit_parent_components` did not defer.
    """
    return _persistent_measurement_support(
        residual_window,
        rms_window,
        _fit_parent_local_valid(
            residual_window,
            rms_window,
            valid_window,
            fit_parent_window,
            parent_index,
        ),
        atrous_plan,
        minimum_support_fraction,
        detection_sigma,
        island_sigma,
        minimum_pixels,
    )

group_support_feature_components(residual_window, rms_window, valid_window, measurement_window, feature_window, fits, protected_labels, wcs, beam, atrous_plan, *, bounds, detection_sigma, island_sigma, minimum_pixels, minimum_support_fraction)

Group the components one connected support feature holds.

Both remaining cross-parent steps label the accumulated measurement support with the same connectivity, so they share this work unit and one read of its window. fits and protected_labels need only cover the components whose measurement labels reach this window.

Source code in src/hebog/algorithms/component_measurement.py

def group_support_feature_components(  # noqa: PLR0913, PLR0917
    residual_window: np.ndarray,
    rms_window: np.ndarray,
    valid_window: np.ndarray,
    measurement_window: np.ndarray,
    feature_window: np.ndarray,
    fits: tuple[tuple[int, CompactGaussianFitResult], ...],
    protected_labels: frozenset[int],
    wcs: WCS,
    beam: RestoringBeam,
    atrous_plan: ResidualAtrousPlan,
    *,
    bounds: ImageBounds,
    detection_sigma: float,
    island_sigma: float,
    minimum_pixels: int,
    minimum_support_fraction: float,
) -> SupportFeatureGroups:
    """Group the components one connected support feature holds.

    Both remaining cross-parent steps label the accumulated measurement
    support with the same connectivity, so they share this work unit and one
    read of its window. ``fits`` and ``protected_labels`` need only cover the
    components whose measurement labels reach this window.
    """
    evidence: list[ComponentGroupingEvidence] = []
    groups = (
        *_loop_groups_in_feature(
            residual_window,
            rms_window,
            valid_window,
            measurement_window,
            feature_window,
            fits,
            wcs,
            beam,
            atrous_plan,
            island_sigma,
            minimum_support_fraction,
            bounds=bounds,
            evidence=evidence,
        ),
        *_residual_groups_in_feature(
            residual_window,
            rms_window,
            valid_window,
            measurement_window,
            feature_window,
            fits,
            protected_labels,
            atrous_plan,
            minimum_support_fraction,
            detection_sigma,
            island_sigma,
            minimum_pixels,
            bounds=bounds,
            evidence=evidence,
        ),
    )
    return SupportFeatureGroups(groups, tuple(evidence))

measure_component_models(residual, rms, valid, direct_labels, measurement_labels, wcs, beam, moment_config, fit_config, *, detection_sigma, island_sigma, minimum_pixels, maximum_bounds_pixels, atrous_plan, minimum_support_fraction)

Fit each bounded connected measurement parent with original pixels.

No full-image fit matrix is constructed. Foreground owners outside a parent's window are excluded; negative background-context pixels remain in the fit. Positive exact seed pixels initialize the existing moment oracle only. Published ownership is never replaced by these fit masks.

Source code in src/hebog/algorithms/component_measurement.py

def measure_component_models(  # noqa: PLR0913, PLR0917
    residual: np.ndarray,
    rms: np.ndarray,
    valid: np.ndarray,
    direct_labels: np.ndarray,
    measurement_labels: np.ndarray,
    wcs: WCS,
    beam: RestoringBeam,
    moment_config: CompactMomentConfig,
    fit_config: CompactGaussianFitConfig,
    *,
    detection_sigma: float,
    island_sigma: float,
    minimum_pixels: int,
    maximum_bounds_pixels: int,
    atrous_plan: ResidualAtrousPlan,
    minimum_support_fraction: float,
) -> ComponentMeasurements:
    """Fit each bounded connected measurement parent with original pixels.

    No full-image fit matrix is constructed. Foreground owners outside a
    parent's window are excluded; negative background-context pixels remain
    in the fit. Positive exact seed pixels initialize the existing moment
    oracle only. Published ownership is never replaced by these fit masks.
    """
    margin = fit_parent_margin_pixels(fit_config, atrous_plan)
    parents = _bounded_fit_parents(
        measurement_labels,
        direct_labels,
        context_margin_pixels=fit_config.context_margin_pixels,
        read_margin_pixels=margin,
        maximum_bounds_pixels=maximum_bounds_pixels,
    )
    objects = find_objects(parents)
    measured_parents: list[FitParentMeasurement] = []
    measurement_support = np.zeros(residual.shape, dtype=np.bool_)
    for parent_index, slices in enumerate(objects, start=1):
        assert slices is not None, "fit-context labels must be dense"
        ys, xs = slices
        bounds = ImageBounds(
            max(0, ys.start - margin),
            min(residual.shape[0], ys.stop + margin),
            max(0, xs.start - margin),
            min(residual.shape[1], xs.stop + margin),
        )
        window = np.s_[
            bounds.y_start : bounds.y_stop, bounds.x_start : bounds.x_stop
        ]
        measured = measure_fit_parent_components(
            residual[window],
            rms[window],
            valid[window],
            parents[window],
            direct_labels[window],
            measurement_labels[window],
            compact_geometry_from_wcs(beam, wcs, bounds.center_xy),
            moment_config,
            fit_config,
            parent_index=parent_index,
            bounds=bounds,
            image_shape_yx=residual.shape,
            detection_sigma=detection_sigma,
            island_sigma=island_sigma,
            minimum_pixels=minimum_pixels,
            maximum_bounds_pixels=maximum_bounds_pixels,
            atrous_plan=atrous_plan,
            minimum_support_fraction=minimum_support_fraction,
        )
        measured_parents.append(measured)
        # Measurement-only persistent emission belongs to admitted owners
        # independently of whether a compact Gaussian describes them. A
        # bounded or unavailable fit must not truncate extended photometry.
        if not measured.deferred:
            measurement_support[window] |= fit_parent_measurement_support(
                residual[window],
                rms[window],
                valid[window],
                parents[window],
                parent_index=parent_index,
                atrous_plan=atrous_plan,
                minimum_support_fraction=minimum_support_fraction,
                detection_sigma=detection_sigma,
                island_sigma=island_sigma,
                minimum_pixels=minimum_pixels,
            )
    fits = tuple(
        item for measured in measured_parents for item in measured.fits
    )
    protected_labels = frozenset(
        index
        for measured in measured_parents
        for group in measured.compact_groups
        for index in group
    )
    measurement_support.setflags(write=False)
    return replace(
        reconcile_component_measurements(
            parents=tuple(measured_parents),
            features=_whole_plane_feature_groups(
                residual,
                rms,
                valid,
                measurement_labels,
                measurement_support,
                fits,
                protected_labels,
                wcs,
                beam,
                atrous_plan,
                detection_sigma=detection_sigma,
                island_sigma=island_sigma,
                minimum_pixels=minimum_pixels,
                maximum_bounds_pixels=maximum_bounds_pixels,
                minimum_support_fraction=minimum_support_fraction,
            ),
        ),
        measurement_support=measurement_support,
    )

measure_fit_parent_components(residual_window, rms_window, valid_window, fit_parent_window, direct_window, measurement_window, geometry, moment_config, fit_config, *, parent_index, bounds, image_shape_yx, detection_sigma, island_sigma, minimum_pixels, maximum_bounds_pixels, atrous_plan, minimum_support_fraction)

Fit one bounded measurement parent from its own original pixels.

The window must hold the parent's support and the reviewed context margin around it; no decision here reads further, so one task can measure one fit parent exactly. A parent whose window exceeds the admitted bound is deferred rather than fitted on truncated pixels.

geometry is the caller's local beam and pixel geometry at the centre of bounds, as :func:~hebog.algorithms.astrometry.compact_geometries_from_wcs builds it for a whole batch at once. Deriving it here instead would pay Astropy's per-call frame machinery once per parent, which costs more than the fit.

Source code in src/hebog/algorithms/component_measurement.py

def measure_fit_parent_components(  # noqa: PLR0913, PLR0917
    residual_window: np.ndarray,
    rms_window: np.ndarray,
    valid_window: np.ndarray,
    fit_parent_window: np.ndarray,
    direct_window: np.ndarray,
    measurement_window: np.ndarray,
    geometry: CompactMeasurementGeometry,
    moment_config: CompactMomentConfig,
    fit_config: CompactGaussianFitConfig,
    *,
    parent_index: int,
    bounds: ImageBounds,
    image_shape_yx: tuple[int, int],
    detection_sigma: float,
    island_sigma: float,
    minimum_pixels: int,
    maximum_bounds_pixels: int,
    atrous_plan: ResidualAtrousPlan,
    minimum_support_fraction: float,
) -> FitParentMeasurement:
    """Fit one bounded measurement parent from its own original pixels.

    The window must hold the parent's support and the reviewed context margin
    around it; no decision here reads further, so one task can measure one
    fit parent exactly. A parent whose window exceeds the admitted bound is
    deferred rather than fitted on truncated pixels.

    ``geometry`` is the caller's local beam and pixel geometry at the centre
    of ``bounds``, as
    :func:`~hebog.algorithms.astrometry.compact_geometries_from_wcs` builds
    it for a whole batch at once. Deriving it here instead would pay
    Astropy's per-call frame machinery once per parent, which costs more
    than the fit.
    """
    if not compact_window_is_admitted(
        bounds, maximum_bounds_pixels=maximum_bounds_pixels
    ):
        return FitParentMeasurement(deferred=True)
    parent_support = fit_parent_window == parent_index
    local_valid = _fit_parent_local_valid(
        residual_window,
        rms_window,
        valid_window,
        fit_parent_window,
        parent_index,
    )
    seeds = np.where(
        parent_support & (residual_window > 0.0) & local_valid,
        direct_window,
        0,
    ).astype(np.int32)
    indexes = tuple(int(index) for index in np.unique(seeds) if index > 0)
    if not indexes:
        return FitParentMeasurement(deferred=True)
    island_id = f"measurement-parent-{parent_index}"
    regions = tuple(
        _region(
            seeds,
            index,
            residual_window,
            rms_window,
            (bounds.y_start, bounds.x_start),
            island_id,
        )
        for index in indexes
    )
    island = _measurement_island(
        regions, seeds, bounds, parent_index, image_shape_yx
    )
    compact = _ComponentFitInput(
        island,
        bounds,
        regions,
        residual_window,
        rms_window,
        local_valid,
        seeds,
    )
    moments = measure_compact_moments(compact, geometry, moment_config)[1:]
    # These are native component measurements, not Gaussian source
    # surrogates. Apply the already configured component extension rule
    # to the whole joint solution; do not splice per-component fits from
    # competing source/component models. Source flux is an aperture.
    fitted = fit_compact_gaussian_mixture(
        compact,
        moments,
        geometry,
        replace(
            fit_config,
            extension_significance_sigma=fit_config.component_extension_significance_sigma,
        ),
    )
    labelled = tuple(zip(indexes, fitted, strict=True))
    complete = tuple(
        (index, fit)
        for index, fit in labelled
        if isinstance(fit, ValidCompactGaussianFit)
        and "fit-at-bound" not in fit.quality_flags
    )
    admitted = _admit_fallbacks(
        complete,
        compact,
        fit_config,
        atrous_plan,
        minimum_support_fraction,
        detection_sigma,
        island_sigma,
        minimum_pixels,
    )
    by_index = dict(admitted)
    fits = tuple((index, by_index.get(index, fit)) for index, fit in labelled)
    measured = FitParentMeasurement(fits=fits, support_bounds=bounds)
    expected_indexes = set(np.unique(measurement_window[parent_support])) - {0}
    if len(complete) != len(expected_indexes) or any(
        isinstance(fit, FailedCompactGaussianFit) for _, fit in admitted
    ):
        return measured
    evidence: list[ComponentGroupingEvidence] = []
    model, groups = _model_and_groups(complete, bounds)
    covariance = geometry.restoring_beam_covariance_pixels_squared
    assert covariance is not None
    beam_xx, beam_xy, beam_yy = covariance
    loop_groups = (
        _resolved_emission_loop(
            residual_window,
            rms_window,
            local_valid,
            atrous_plan,
            fits=complete,
            bounds=bounds,
            beam_covariance=np.array(((beam_xx, beam_xy), (beam_xy, beam_yy))),
            island_sigma=island_sigma,
            minimum_support_fraction=minimum_support_fraction,
            evidence=evidence,
        )
        if len(groups) > 1
        else ()
    )
    loop_labels = {index for group in loop_groups for index in group}
    remaining_groups = tuple(
        remaining for group in groups if (remaining := group - loop_labels)
    )
    if not remaining_groups:
        return replace(
            measured, extended_groups=loop_groups, evidence=tuple(evidence)
        )

    # A fit parent joins every owner whose fit context touches another's, so
    # in a crowded field it holds many independent sources. Residual
    # emission fails only the compact groups whose own support it touches.
    features = _seeded_residual_features(
        residual_window - model,
        rms_window,
        local_valid,
        detection_sigma=detection_sigma,
        island_sigma=island_sigma,
        atrous_plan=atrous_plan,
        minimum_support_fraction=minimum_support_fraction,
    )
    attribution = (
        expand_source_measurement_labels(
            fit_parent_window,
            valid_window,
            radius_pixels=ceil(float(np.hypot(*bounds.shape_yx))),
        )
        == parent_index
    )
    compact_groups = tuple(
        remaining
        for remaining in remaining_groups
        if not _leaves_unmodelled_emission(
            features,
            attribution,
            minimum_pixels,
            parent_support & np.isin(measurement_window, tuple(remaining)),
        )
    )
    return replace(
        measured,
        compact_groups=compact_groups,
        extended_groups=loop_groups,
        evidence=tuple(evidence),
    )

reconcile_component_measurements(*, parents, features)

Reconcile the parents' and features' records into one measurement.

Every step that spans fit parents has already run: each connected support feature contributed its extended groups and grouping evidence. What is left is a reduction over records, so it holds no image-sized array at all: the support the parents contributed stays in the generation the cores wrote it to.

Source code in src/hebog/algorithms/component_measurement.py

def reconcile_component_measurements(
    *,
    parents: tuple[FitParentMeasurement, ...],
    features: tuple[SupportFeatureGroups, ...],
) -> ComponentMeasurements:
    """Reconcile the parents' and features' records into one measurement.

    Every step that spans fit parents has already run: each connected support
    feature contributed its extended groups and grouping evidence. What is
    left is a reduction over records, so it holds no image-sized array at all:
    the support the parents contributed stays in the generation the cores
    wrote it to.
    """
    output: list[tuple[int, CompactGaussianFitResult]] = []
    compact_groups: list[frozenset[int]] = []
    extended_groups: list[frozenset[int]] = []
    evidence: list[ComponentGroupingEvidence] = []
    deferred = 0
    for measured in parents:
        deferred += int(measured.deferred)
        output.extend(measured.fits)
        compact_groups.extend(measured.compact_groups)
        extended_groups.extend(measured.extended_groups)
        evidence.extend(measured.evidence)
    for item in features:
        extended_groups.extend(item.extended_groups)
        evidence.extend(item.evidence)
    # Reconcile admitted sources as whole owners. An extended proposal may
    # include only some Gaussian members of an accepted compact source; it
    # must not split the other members into a second, overlapping source.
    # Independent compact owners remain separate unless evidence names them.
    proposed_extended_labels = {
        index for group in extended_groups for index in group
    }
    reconciled = tuple(
        group
        for group in _merge_overlapping_groups(
            [*extended_groups, *compact_groups]
        )
        if group & proposed_extended_labels
    )
    proposed_compact_groups = tuple(compact_groups)
    protected_compact_labels = {
        index for group in proposed_compact_groups for index in group
    }
    evidence = [
        replace(
            item,
            protected_labels=(
                item.protected_labels
                | (item.component_labels & protected_compact_labels)
            ),
        )
        for item in evidence
    ]
    evidence.extend(
        ComponentGroupingEvidence("directional-fwhm-overlap", (), group)
        for group in compact_groups
        if len(group) > 1
    )
    extended_labels = {index for group in reconciled for index in group}
    compact_groups = [
        group - extended_labels
        for group in compact_groups
        if group - extended_labels
    ]
    return ComponentMeasurements(
        tuple(output),
        tuple(compact_groups),
        deferred,
        None,
        tuple(reconciled),
        proposed_compact_groups,
        tuple(
            sorted(
                set(evidence),
                key=lambda item: (
                    item.reason,
                    item.scale_ids,
                    tuple(sorted(item.component_labels)),
                ),
            )
        ),
    )

support_feature_margin_pixels(atrous_plan)

Return the context one support feature reads beyond its own bounds.

Source code in src/hebog/algorithms/component_measurement.py

def support_feature_margin_pixels(atrous_plan: ResidualAtrousPlan) -> int:
    """Return the context one support feature reads beyond its own bounds."""
    return _adequacy_filter_bank(atrous_plan).maximum_halo_pixels

support_feature_window(feature_bounds, *, margin, image_shape_yx, maximum_bounds_pixels)

Return one feature's grouping window, or absence when unbounded.

A feature wider than the reviewed compact bound is ADR-008 T3 work and contributes no grouping, exactly as the whole-plane pass leaves it.

Source code in src/hebog/algorithms/component_measurement.py

def support_feature_window(
    feature_bounds: ImageBounds,
    *,
    margin: int,
    image_shape_yx: tuple[int, int],
    maximum_bounds_pixels: int,
) -> ImageBounds | None:
    """Return one feature's grouping window, or absence when unbounded.

    A feature wider than the reviewed compact bound is ADR-008 T3 work and
    contributes no grouping, exactly as the whole-plane pass leaves it.
    """
    bounds = ImageBounds(
        max(0, feature_bounds.y_start - margin),
        min(image_shape_yx[0], feature_bounds.y_stop + margin),
        max(0, feature_bounds.x_start - margin),
        min(image_shape_yx[1], feature_bounds.x_stop + margin),
    )
    if not compact_window_is_admitted(
        bounds, maximum_bounds_pixels=maximum_bounds_pixels
    ):
        return None
    return bounds

hebog.stages.object_extents

Records and window helpers the per-object stages share.

ADR-008's pass D evaluates each object inside the window that holds it and reduces the results hierarchically. Each per-object stage first has the cores observe every object's extent, reduces those observations into one record per parent here, and then reads each object through a window cropped from a wider read.

CoreExtent dataclass

One parent's bounds and first pixel as one core observes them.

Source code in src/hebog/stages/object_extents.py

@dataclass(frozen=True, slots=True)
class CoreExtent:
    """One parent's bounds and first pixel as one core observes them."""

    parent_label: int
    direct_bounds: ImageBounds | None
    measurement_bounds: ImageBounds
    first_pixel_yx: tuple[int, int] | None
    tile_id: str
    direct_pixel_count: int

ExtentBatchResult dataclass

Array-free parent extents from one bounded scan task.

Source code in src/hebog/stages/object_extents.py

@dataclass(frozen=True, slots=True)
class ExtentBatchResult:
    """Array-free parent extents from one bounded scan task."""

    extents: tuple[CoreExtent, ...]

ParentExtent dataclass

One parent's support in each label plane, reduced over the cores.

Source code in src/hebog/stages/object_extents.py

@dataclass(frozen=True, slots=True)
class ParentExtent:
    """One parent's support in each label plane, reduced over the cores."""

    parent_label: int
    direct_bounds: ImageBounds
    measurement_bounds: ImageBounds
    first_pixel_yx: tuple[int, int]
    direct_pixel_count: int

PublishBatchResult dataclass

Persisted product identities and the labels each plane carried.

observed_labels pairs every label a core wrote with the plane that carried it — 0 for direct ownership, 1 for measurement ownership — so the stage can require both planes to name the same components without holding either.

Source code in src/hebog/stages/object_extents.py

@dataclass(frozen=True, slots=True)
class PublishBatchResult:
    """Persisted product identities and the labels each plane carried.

    ``observed_labels`` pairs every label a core wrote with the plane that
    carried it — 0 for direct ownership, 1 for measurement ownership — so the
    stage can require both planes to name the same components without holding
    either.
    """

    product_chunks: tuple[ProductChunk, ...]
    observed_labels: tuple[tuple[int, int], ...] = ()
    maximum_parent_read_pixels: int = 0

bounds_intersect(first, second)

Return whether two half-open bounds share a pixel.

Source code in src/hebog/stages/object_extents.py

def bounds_intersect(first: ImageBounds, second: ImageBounds) -> bool:
    """Return whether two half-open bounds share a pixel."""
    return (
        first.y_start < second.y_stop
        and second.y_start < first.y_stop
        and first.x_start < second.x_stop
        and second.x_start < first.x_stop
    )

core_label_extents(labels, bounds)

Return each label's bounds, first pixel and size in one owned core.

Source code in src/hebog/stages/object_extents.py

def core_label_extents(
    labels: npt.NDArray[np.int32],
    bounds: ImageBounds,
) -> dict[int, tuple[ImageBounds, tuple[int, int], int]]:
    """Return each label's bounds, first pixel and size in one owned core."""
    extents = label_extents(labels)
    return {
        int(value): (
            ImageBounds(
                bounds.y_start + int(y_start),
                bounds.y_start + int(y_stop),
                bounds.x_start + int(x_start),
                bounds.x_start + int(x_stop),
            ),
            (bounds.y_start + int(first_y), bounds.x_start + int(first_x)),
            int(pixel_count),
        )
        for (
            value,
            y_start,
            y_stop,
            x_start,
            x_stop,
            first_y,
            first_x,
            pixel_count,
        ) in zip(
            extents.values,
            extents.y_start,
            extents.y_stop,
            extents.x_start,
            extents.x_stop,
            extents.first_y,
            extents.first_x,
            extents.pixel_count,
            strict=True,
        )
    }

crop_slices(bounds, window)

Return the slices selecting one window inside a wider read.

Source code in src/hebog/stages/object_extents.py

def crop_slices(
    bounds: ImageBounds, window: ImageBounds
) -> tuple[slice, slice]:
    """Return the slices selecting one window inside a wider read."""
    return (
        slice(window.y_start - bounds.y_start, window.y_stop - bounds.y_start),
        slice(window.x_start - bounds.x_start, window.x_stop - bounds.x_start),
    )

parent_holders(results)

Name the cores in which each parent holds a pixel.

Source code in src/hebog/stages/object_extents.py

def parent_holders(
    results: tuple[ExtentBatchResult, ...],
) -> dict[int, frozenset[str]]:
    """Name the cores in which each parent holds a pixel."""
    holders: dict[int, set[str]] = {}
    for result in results:
        for extent in result.extents:
            holders.setdefault(extent.parent_label, set()).add(extent.tile_id)
    return {label: frozenset(tiles) for label, tiles in holders.items()}

reduce_extents(results)

Merge every core's observation into one record per parent.

Source code in src/hebog/stages/object_extents.py

def reduce_extents(
    results: tuple[ExtentBatchResult, ...],
) -> tuple[ParentExtent, ...]:
    """Merge every core's observation into one record per parent."""
    direct: dict[int, ImageBounds | None] = {}
    measurement: dict[int, ImageBounds | None] = {}
    first: dict[int, tuple[int, int] | None] = {}
    direct_pixel_counts: dict[int, int] = {}
    for result in results:
        for extent in result.extents:
            label = extent.parent_label
            direct_pixel_counts[label] = (
                direct_pixel_counts.get(label, 0) + extent.direct_pixel_count
            )
            direct[label] = union_bounds(
                direct.get(label),
                extent.direct_bounds,
            )
            measurement[label] = union_bounds(
                measurement.get(label),
                extent.measurement_bounds,
            )
            candidate = extent.first_pixel_yx
            if candidate is not None:
                known = first.get(label)
                first[label] = (
                    candidate if known is None else min(known, candidate)
                )
    parents: list[ParentExtent] = []
    for label in sorted(measurement):
        direct_bounds = direct.get(label)
        first_pixel = first.get(label)
        measurement_bounds = measurement[label]
        if (
            direct_bounds is None
            or first_pixel is None
            or measurement_bounds is None
        ):
            raise ValueError(
                "every measurement parent must own direct support"
            )
        parents.append(
            ParentExtent(
                parent_label=label,
                direct_bounds=direct_bounds,
                measurement_bounds=measurement_bounds,
                first_pixel_yx=first_pixel,
                direct_pixel_count=direct_pixel_counts[label],
            )
        )
    return tuple(
        sorted(
            parents,
            key=lambda parent: (parent.first_pixel_yx, parent.parent_label),
        )
    )

union_bounds(first, second)

Return the smallest bounds containing both inputs, if either exists.

Source code in src/hebog/stages/object_extents.py

def union_bounds(
    first: ImageBounds | None,
    second: ImageBounds | None,
) -> ImageBounds | None:
    """Return the smallest bounds containing both inputs, if either exists."""
    if first is None:
        return second
    if second is None:
        return first
    return ImageBounds(
        min(first.y_start, second.y_start),
        max(first.y_stop, second.y_stop),
        min(first.x_start, second.x_start),
        max(first.x_stop, second.x_stop),
    )

hebog.stages.component_topology

Component topology, the first per-object round of the composition.

Every parent's deblending needs its complete support and nothing beyond it, so one task decides one parent exactly, and the cores then write the component labels they own.

Component numbering stays canonical because the driver offsets each parent's local labels by the components every earlier parent produced, in ascending parent order, which is the order a whole-plane pass would use. The driver needs only each parent's component count for that, so no component pixel reaches it: a core writing a deblended parent decides its memberships again from the parent's own windows, which gives the same memberships bit for bit.

ComponentTopologyStageConfig dataclass

Reviewed deblending policy and the bounded task limits.

Source code in src/hebog/stages/component_topology.py

@dataclass(frozen=True, slots=True)
class ComponentTopologyStageConfig:
    """Reviewed deblending policy and the bounded task limits."""

    deblend: CompactDeblendConfig
    maximum_tiles_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before stage products are initialized."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")

__post_init__()

Reject an unbounded task before stage products are initialized.

Source code in src/hebog/stages/component_topology.py

def __post_init__(self) -> None:
    """Reject an unbounded task before stage products are initialized."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")

ComponentTopologyStageResult dataclass

Published component labels and scalar execution evidence.

Source code in src/hebog/stages/component_topology.py

@dataclass(frozen=True, slots=True)
class ComponentTopologyStageResult:
    """Published component labels and scalar execution evidence."""

    generation: ProductGenerationManifest
    component_count: int
    parent_count: int
    deblended_parent_count: int
    deferred_parent_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_parent_read_pixels: int
    parent_batch_count: int

component_topology_product_names()

Return the canonical published component-topology product set.

Source code in src/hebog/stages/component_topology.py

def component_topology_product_names() -> tuple[str, ...]:
    """Return the canonical published component-topology product set."""
    return _TOPOLOGY_PRODUCT_NAMES

run_component_topology_stage(support_source, detection_source, manifest, *, config, executor, sink)

Deblend every parent in its own window and publish the components.

Three rounds: the cores observe each parent's extent, one task per batch of parents deblends them inside those extents and counts their components, and the cores write the component labels they own. Parent order is canonical, so the component numbering does not move with tile geometry or completion order.

The driver holds records only. A parent published as one component is its own support in both planes, so the cores that hold it relabel it from their own windows. That covers a parent beyond either hard compact-work bound, which is deferred exactly as the whole-plane deblender defers it, from its extent alone, and whose window is never read. A parent that splits is deblended again by each core that holds it, inside the same windows, which decides the same memberships; returning them instead would make the driver hold every component pixel in the image.

Source code in src/hebog/stages/component_topology.py

def run_component_topology_stage(  # noqa: PLR0913
    support_source: CompletedProductSource,
    detection_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: ComponentTopologyStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
) -> ComponentTopologyStageResult:
    """Deblend every parent in its own window and publish the components.

    Three rounds: the cores observe each parent's extent, one task per batch
    of parents deblends them inside those extents and counts their
    components, and the cores write the component labels they own. Parent
    order is canonical, so the component numbering does not move with tile
    geometry or completion order.

    The driver holds records only. A parent published as one component is
    its own support in both planes, so the cores that hold it relabel it from
    their own windows. That covers a parent beyond either hard compact-work
    bound, which is deferred exactly as the whole-plane deblender defers it,
    from its extent alone, and whose window is never read. A parent that
    splits is deblended again by each core that holds it, inside the same
    windows, which decides the same memberships; returning them instead
    would make the driver hold every component pixel in the image.
    """
    _validate_stage_inputs(support_source, detection_source, manifest, sink)
    scan_batches = _tile_batches(
        tuple(
            _TileRequest(partition=partition) for partition in manifest.tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    scan_results = tuple(
        executor.map_batches(
            partial(_scan_extents, support_source=support_source),
            scan_batches,
            requirement=read_requirement(
                EXTENT_SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no parent extent results")
    parents = reduce_extents(scan_results)
    parent_batches = _parent_batches(
        tuple(
            parent
            for parent in parents
            if not _deferred(parent, config.deblend)
        ),
        core_shape_yx=manifest.tile_core_shape_yx,
        maximum_batch_read_pixels=config.maximum_batch_read_pixels,
        deblend=config.deblend,
    )
    deblend_results = map_round(
        executor,
        partial(
            _deblend_batch,
            support_source=support_source,
            detection_source=detection_source,
            config=config,
            image_shape_yx=manifest.image_shape_yx,
        ),
        parent_batches,
        round_name="component deblend",
        requirement=read_requirement(
            _DEBLEND_BYTES_PER_READ_PIXEL,
            (batch.read_bounds for batch in parent_batches),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    counts = {
        count.parent_label: count
        for result in deblend_results
        for count in result.parents
    }
    single_components, component_offsets, component_count = _numbered_parents(
        parents, counts
    )
    for product_name in _TOPOLOGY_PRODUCT_NAMES:
        sink.initialize_product(
            product_name=product_name,
            dtype=np.dtype("<i4"),
        )
    publish_batches = _tile_batches(
        _write_requests(
            manifest,
            parents,
            single_components=single_components,
            component_offsets=component_offsets,
            holders=parent_holders(scan_results),
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    publish_results = tuple(
        executor.map_batches(
            partial(
                _publish_batch,
                support_source=support_source,
                detection_source=detection_source,
                sink=sink,
                config=config,
                image_shape_yx=manifest.image_shape_yx,
                core_shape_yx=manifest.tile_core_shape_yx,
            ),
            publish_batches,
            # A parent that splits is deblended again inside its own window,
            # which its round bounded as it bounds the deblend batches.
            requirement=read_requirement(
                _TOPOLOGY_PUBLICATION_BYTES_PER_READ_PIXEL,
                (
                    *(partition.read_bounds for partition in manifest.tiles),
                    *(batch.read_bounds for batch in parent_batches),
                ),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not publish_results:
        raise ValueError("executor returned no component publication results")
    _require_matching_component_identities(publish_results, component_count)
    return ComponentTopologyStageResult(
        generation=sink.publish_generation(
            product_names=_TOPOLOGY_PRODUCT_NAMES,
            chunks=(
                chunk
                for result in publish_results
                for chunk in result.product_chunks
            ),
        ),
        component_count=component_count,
        parent_count=len(parents),
        deblended_parent_count=len(component_offsets),
        deferred_parent_count=len(parents)
        - len(counts)
        + sum(int(count.deferred) for count in counts.values()),
        partition_count=len(manifest.tiles),
        executor_task_count=(
            len(scan_batches) + len(parent_batches) + len(publish_batches)
        ),
        maximum_graph_width=max(
            len(scan_batches),
            len(parent_batches),
            len(publish_batches),
        ),
        maximum_parent_read_pixels=max(
            (
                result.maximum_parent_read_pixels
                for result in (*deblend_results, *publish_results)
            ),
            default=0,
        ),
        parent_batch_count=len(parent_batches),
    )

hebog.stages.fit_parents

Fit parents: the owners fitted together, numbered and published by core.

Owners whose fit contexts touch need a joint model, and that connectivity follows a chain of any length, so it is reconciled from compact per-core summaries before any fit runs.

FitParentStageConfig dataclass

The reviewed fit context margin, the joint-fit bounds and task limit.

read_margin_pixels and maximum_bounds_pixels are the window the component-fit round reads around a parent and the compact bound that admits it, so they must be that round's own. The joint fit's work limits are not settings: the stage applies the fit's fixed ones, to a parent's owners and their direct pixels as an owned-region fit counts them. A parent beyond either is fitted island by island.

Source code in src/hebog/stages/fit_parents.py

@dataclass(frozen=True, slots=True)
class FitParentStageConfig:
    """The reviewed fit context margin, the joint-fit bounds and task limit.

    ``read_margin_pixels`` and ``maximum_bounds_pixels`` are the window the
    component-fit round reads around a parent and the compact bound that
    admits it, so they must be that round's own. The joint fit's work
    limits are not settings: the stage applies the fit's fixed ones, to a
    parent's owners and their direct pixels as an ``owned-region`` fit
    counts them. A parent beyond either is fitted island by island.
    """

    context_margin_pixels: int
    maximum_tiles_per_batch: int
    read_margin_pixels: int
    maximum_bounds_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before stage products are initialized."""
        for name in ("maximum_tiles_per_batch", "maximum_bounds_pixels"):
            value = getattr(self, name)
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")
        if self.context_margin_pixels < 0:
            raise ValueError("context margin must be non-negative")
        if (
            isinstance(self.read_margin_pixels, bool)
            or not isinstance(self.read_margin_pixels, Integral)
            or self.read_margin_pixels < 0
        ):
            raise ValueError("read margin must be a non-negative integer")

__post_init__()

Reject an unbounded task before stage products are initialized.

Source code in src/hebog/stages/fit_parents.py

def __post_init__(self) -> None:
    """Reject an unbounded task before stage products are initialized."""
    for name in ("maximum_tiles_per_batch", "maximum_bounds_pixels"):
        value = getattr(self, name)
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")
    if self.context_margin_pixels < 0:
        raise ValueError("context margin must be non-negative")
    if (
        isinstance(self.read_margin_pixels, bool)
        or not isinstance(self.read_margin_pixels, Integral)
        or self.read_margin_pixels < 0
    ):
        raise ValueError("read margin must be a non-negative integer")

FitParentStageResult dataclass

Published fit-parent labels and scalar execution evidence.

Source code in src/hebog/stages/fit_parents.py

@dataclass(frozen=True, slots=True)
class FitParentStageResult:
    """Published fit-parent labels and scalar execution evidence."""

    generation: ProductGenerationManifest
    fit_parent_count: int
    context_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    reconciliation_round_count: int

fit_parent_product_names()

Return the canonical published fit-parent product set.

Source code in src/hebog/stages/fit_parents.py

def fit_parent_product_names() -> tuple[str, ...]:
    """Return the canonical published fit-parent product set."""
    return _FIT_PARENT_PRODUCT_NAMES

run_fit_parent_stage(component_source, manifest, *, config, executor, sink)

Reconcile the fit contexts owners share and publish their numbers.

Owners whose contexts touch need a joint model, and that connectivity follows a chain of any length, so it is reconciled from compact per-core summaries before any fit runs. A parent of several owners that no joint fit can hold, by its work limits or the compact bound on its window, is fitted island by island: only then are the islands scanned and reconciled in one further round. The cores then write the fit-parent number each support pixel belongs to.

Source code in src/hebog/stages/fit_parents.py

def run_fit_parent_stage(
    component_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: FitParentStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
) -> FitParentStageResult:
    """Reconcile the fit contexts owners share and publish their numbers.

    Owners whose contexts touch need a joint model, and that connectivity
    follows a chain of any length, so it is reconciled from compact per-core
    summaries before any fit runs. A parent of several owners that no joint
    fit can hold, by its work limits or the compact bound on its window, is
    fitted island by island: only then are the islands scanned and
    reconciled in one further round. The cores then write the fit-parent
    number each support pixel belongs to.
    """
    if sink.manifest != manifest:
        raise ValueError("fit-parent sink must use the stage manifest")
    required_halo = config.context_margin_pixels
    if manifest.halo_yx != (required_halo, required_halo):
        raise ValueError(
            "fit-parent manifest must provide the exact context margin"
        )
    if component_source.manifest.image_shape_yx != manifest.image_shape_yx:
        raise ValueError(
            "component generation must match the fit-parent image shape"
        )
    if not {
        "component-measurement-labels",
        "component-direct-labels",
    } <= set(component_source.read_generation().product_names):
        raise ValueError(
            "component generation must publish direct and measurement "
            "component labels"
        )
    scan_batches = _context_batches(
        tuple(
            _ContextPublicationRequest(
                partition=partition,
                fit_parent_by_local_label=(),
            )
            for partition in manifest.tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    context_reads = tuple(
        partition.read_bounds for partition in manifest.tiles
    )
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_contexts,
                component_source=component_source,
                image_shape_yx=manifest.image_shape_yx,
                context_margin_pixels=config.context_margin_pixels,
            ),
            scan_batches,
            requirement=read_requirement(
                _CONTEXT_BYTES_PER_READ_PIXEL,
                context_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no fit-context results")
    tiles = tuple(tile for result in scan_results for tile in result.tiles)
    contexts = reconcile_candidate_tiles(
        manifest,
        tuple(tile.summary for tile in tiles),
    )
    numbers = _fit_parent_numbers(tiles, contexts)
    split = _parents_to_fit_by_island(
        tiles,
        contexts,
        numbers,
        config=config,
        image_shape_yx=manifest.image_shape_yx,
    )
    by_context, by_island = numbers, {}
    islands: ReconciledIslands | None = None
    island_task_count = 0
    if split:
        # Only a parent no joint fit can hold needs its islands, so an image
        # without one never labels or reconciles them.
        island_results = tuple(
            executor.map_batches(
                partial(
                    _scan_islands,
                    component_source=component_source,
                    image_shape_yx=manifest.image_shape_yx,
                ),
                scan_batches,
                requirement=read_requirement(
                    _CONTEXT_BYTES_PER_READ_PIXEL,
                    context_reads,
                    chunk_shape_yx=manifest.tile_core_shape_yx,
                ),
            )
        )
        if not island_results:
            raise ValueError("executor returned no fit-island results")
        island_tiles = tuple(
            tile for result in island_results for tile in result.tiles
        )
        islands = reconcile_candidate_tiles(
            manifest,
            tuple(tile.summary for tile in island_tiles),
        )
        by_context, by_island = _numbers_with_islands(
            tiles,
            contexts,
            numbers,
            island_tiles=island_tiles,
            islands=islands,
            split=split,
        )
        island_task_count = len(scan_batches)
    sink.initialize_product(
        product_name="fit-parent-labels",
        dtype=np.dtype("<i4"),
    )
    publish_batches = _context_batches(
        tuple(
            _ContextPublicationRequest(
                partition=tile.partition,
                fit_parent_by_local_label=_local_numbers(
                    contexts.mapping_for_tile(tile.partition.tile_id),
                    by_context,
                ),
                fit_parent_by_local_island_label=(
                    ()
                    if islands is None
                    else _local_numbers(
                        islands.mapping_for_tile(tile.partition.tile_id),
                        by_island,
                    )
                ),
            )
            for tile in tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    publish_results = tuple(
        executor.map_batches(
            partial(
                _publish_fit_parents,
                component_source=component_source,
                sink=sink,
                image_shape_yx=manifest.image_shape_yx,
                context_margin_pixels=config.context_margin_pixels,
            ),
            publish_batches,
            requirement=read_requirement(
                _FIT_PARENT_PUBLICATION_BYTES_PER_READ_PIXEL,
                context_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not publish_results:
        raise ValueError("executor returned no fit-parent publication results")
    return FitParentStageResult(
        generation=sink.publish_generation(
            product_names=_FIT_PARENT_PRODUCT_NAMES,
            chunks=(
                chunk
                for result in publish_results
                for chunk in result.product_chunks
            ),
        ),
        fit_parent_count=len({*by_context.values(), *by_island.values()}),
        context_count=len(contexts.islands),
        partition_count=len(manifest.tiles),
        executor_task_count=(
            len(scan_batches) + island_task_count + len(publish_batches)
        ),
        maximum_graph_width=max(len(scan_batches), len(publish_batches)),
        reconciliation_round_count=contexts.reduction_round_count
        + (0 if islands is None else islands.reduction_round_count),
    )

hebog.stages.component_fits

Component fits: every fit parent measured in its own context window.

The measuring round also describes the direct components each parent owns, and the cores then write each measured parent's persistent support.

ComponentFitStageConfig dataclass

Reviewed measurement policy and the bounded task limits.

Source code in src/hebog/stages/component_fits.py

@dataclass(frozen=True, slots=True)
class ComponentFitStageConfig:
    """Reviewed measurement policy and the bounded task limits."""

    moment: CompactMomentConfig
    fit: CompactGaussianFitConfig
    atrous_plan: ResidualAtrousPlan
    detection_sigma: float
    island_sigma: float
    minimum_pixels: int
    maximum_bounds_pixels: int
    minimum_support_fraction: float
    maximum_tiles_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before stage products are initialized."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")

    @property
    def margin_pixels(self) -> int:
        """Return the context a fit parent reads beyond its own support."""
        return fit_parent_margin_pixels(self.fit, self.atrous_plan)

margin_pixels property

Return the context a fit parent reads beyond its own support.

__post_init__()

Reject an unbounded task before stage products are initialized.

Source code in src/hebog/stages/component_fits.py

def __post_init__(self) -> None:
    """Reject an unbounded task before stage products are initialized."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")

ComponentFitStageResult dataclass

Published measurement support and every fit parent's records.

component_records describes every direct component the association decision reads, in canonical first-pixel order. The parent that reads a component's pixels builds it: the residual, the validity and both component planes are already on that task, so nothing after the fits reads a component's window again.

Source code in src/hebog/stages/component_fits.py

@dataclass(frozen=True, slots=True)
class ComponentFitStageResult:
    """Published measurement support and every fit parent's records.

    ``component_records`` describes every direct component the association
    decision reads, in canonical first-pixel order. The parent that reads a
    component's pixels builds it: the residual, the validity and both
    component planes are already on that task, so nothing after the fits
    reads a component's window again.
    """

    generation: ProductGenerationManifest
    parents: tuple[FitParentMeasurement, ...]
    component_records: tuple[DetectionComponentRecord, ...]
    fit_parent_count: int
    deferred_parent_count: int
    wide_parent_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_parent_read_pixels: int
    parent_batch_count: int

component_fit_product_names()

Return the canonical published component-fit product set.

Source code in src/hebog/stages/component_fits.py

def component_fit_product_names() -> tuple[str, ...]:
    """Return the canonical published component-fit product set."""
    return _MEASUREMENT_PRODUCT_NAMES

run_component_fit_stage(source, background_rms_source, detection_source, component_source, fit_parent_source, manifest, *, config, component_count, wcs_header_text, beam, executor, sink)

Fit every parent in its own context window and publish its support.

Three rounds: the cores observe each fit parent's extent, one task per batch of parents measures them inside that extent plus the reviewed context margin, and the cores derive each measured parent's persistent support again inside that window and write their own pixels of it. Only the last round writes, and no round returns an array.

The measuring round also describes the direct components each parent owns, because the residual and the validity those records need are already on the task that fits them.

A parent whose window the reviewed compact bound refuses is deferred, as the whole-plane pass defers it, so its window is never read. Its components are described instead from the pixels each core holding them returns, restored to raster order, which gives the records a window would give, bit for bit. No read is then wider than the budget, one core, or one window that bound admits, whichever is largest.

component_count is the count the topology stage that published component_source returned. The records must describe exactly those components, so a fit-parent read that misses one stops the stage rather than publishing fewer components.

wcs_header_text is the caller's own header as :meth:astropy.io.fits.Header.tostring writes it, not a WCS; see :func:~hebog.algorithms.astrometry.celestial_wcs_from_header_text.

Source code in src/hebog/stages/component_fits.py

def run_component_fit_stage(  # noqa: PLR0913, PLR0917
    source: WindowReadable,
    background_rms_source: CompletedProductSource,
    detection_source: CompletedProductSource,
    component_source: CompletedProductSource,
    fit_parent_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: ComponentFitStageConfig,
    component_count: int,
    wcs_header_text: str,
    beam: RestoringBeam,
    executor: Executor,
    sink: ZarrProductSink,
) -> ComponentFitStageResult:
    """Fit every parent in its own context window and publish its support.

    Three rounds: the cores observe each fit parent's extent, one task per
    batch of parents measures them inside that extent plus the reviewed
    context margin, and the cores derive each measured parent's persistent
    support again inside that window and write their own pixels of it. Only
    the last round writes, and no round returns an array.

    The measuring round also describes the direct components each parent
    owns, because the residual and the validity those records need are
    already on the task that fits them.

    A parent whose window the reviewed compact bound refuses is deferred, as
    the whole-plane pass defers it, so its window is never read. Its
    components are described instead from the pixels each core holding them
    returns, restored to raster order, which gives the records a window would
    give, bit for bit. No read is then wider than the budget, one core, or
    one window that bound admits, whichever is largest.

    ``component_count`` is the count the topology stage that published
    ``component_source`` returned. The records must describe exactly those
    components, so a fit-parent read that misses one stops the stage rather
    than publishing fewer components.

    ``wcs_header_text`` is the caller's own header as
    :meth:`astropy.io.fits.Header.tostring` writes it, not a ``WCS``; see
    :func:`~hebog.algorithms.astrometry.celestial_wcs_from_header_text`.
    """
    if sink.manifest != manifest:
        raise ValueError("component fit sink must use the stage manifest")
    if manifest.halo_yx != (0, 0):
        raise ValueError("measurement support writes cores without a halo")
    for product_source, names in (
        (background_rms_source, ("background", "rms")),
        (detection_source, ("valid-pixels",)),
        (
            component_source,
            ("component-direct-labels", "component-measurement-labels"),
        ),
        (fit_parent_source, ("fit-parent-labels",)),
    ):
        if product_source.manifest.image_shape_yx != manifest.image_shape_yx:
            raise ValueError(
                "published generations must match the fit image shape"
            )
        if not set(names).issubset(
            product_source.read_generation().product_names
        ):
            raise ValueError(
                "published generations must carry every fit plane read"
            )
    image_shape_yx = manifest.image_shape_yx
    margin = config.margin_pixels
    scan_batches = _support_batches(
        tuple(
            _SupportRequest(partition=partition, parents=())
            for partition in manifest.tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_fit_parent_extents,
                fit_parent_source=fit_parent_source,
            ),
            scan_batches,
            requirement=read_requirement(
                EXTENT_SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no fit-parent extent results")
    extents = tuple(
        _FitParentExtent(
            parent_index=parent.parent_label,
            read_bounds=parent.direct_bounds.expanded(margin, image_shape_yx),
        )
        for parent in sorted(
            reduce_extents(scan_results),
            key=lambda item: item.parent_label,
        )
    )
    fitted = tuple(
        extent
        for extent in extents
        if compact_window_is_admitted(
            extent.read_bounds,
            maximum_bounds_pixels=config.maximum_bounds_pixels,
        )
    )
    deferred = tuple(
        extent
        for extent in extents
        if not compact_window_is_admitted(
            extent.read_bounds,
            maximum_bounds_pixels=config.maximum_bounds_pixels,
        )
    )
    fit_batches = _fit_batches(
        fitted,
        core_shape_yx=manifest.tile_core_shape_yx,
        maximum_batch_read_pixels=config.maximum_batch_read_pixels,
        maximum_bounds_pixels=config.maximum_bounds_pixels,
        maximum_parents_per_batch=objects_per_batch(
            len(fitted), executor.capacity
        ),
    )
    fit_results = map_round(
        executor,
        partial(
            _fit_batch,
            source=source,
            background_rms_source=background_rms_source,
            detection_source=detection_source,
            component_source=component_source,
            fit_parent_source=fit_parent_source,
            config=config,
            wcs_header_text=wcs_header_text,
            beam=beam,
            image_shape_yx=image_shape_yx,
        ),
        fit_batches,
        round_name="component fit",
        requirement=read_requirement(
            _FIT_BYTES_PER_READ_PIXEL,
            (batch.read_bounds for batch in fit_batches),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    deferred_batches = _deferred_batches(
        deferred,
        parent_holders(scan_results),
        manifest,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    deferred_results = map_round(
        executor,
        partial(
            _gather_deferred_components,
            source=source,
            background_rms_source=background_rms_source,
            detection_source=detection_source,
            component_source=component_source,
            fit_parent_source=fit_parent_source,
            image_width=image_shape_yx[1],
        ),
        deferred_batches,
        round_name="deferred component",
        requirement=read_requirement(
            _DEFERRED_BYTES_PER_READ_PIXEL,
            (
                core.partition.read_bounds
                for batch in deferred_batches
                for core in batch.cores
            ),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    measured = tuple(
        sorted(
            (
                *(item for result in fit_results for item in result.parents),
                *(
                    (parent.parent_index, FitParentMeasurement(deferred=True))
                    for parent in deferred
                ),
            ),
            key=lambda item: item[0],
        )
    )
    component_records = _reduce_component_records(
        (
            *(
                record
                for result in fit_results
                for record in result.component_records
            ),
            *_deferred_component_records(
                deferred_batches,
                deferred_results,
                image_width=image_shape_yx[1],
            ),
        ),
        component_count=component_count,
    )
    # A parent the measurement deferred, by its bound or for want of a
    # seed, contributes no support, exactly as the whole-plane pass.
    contributing = {
        parent_index
        for parent_index, parent in measured
        if not parent.deferred
    }
    supported = tuple(
        extent for extent in fitted if extent.parent_index in contributing
    )
    sink.initialize_product(
        product_name="measurement-support",
        dtype=np.dtype(np.bool_),
    )
    publish_batches = _support_batches(
        tuple(
            _SupportRequest(
                partition=partition,
                parents=tuple(
                    extent
                    for extent in supported
                    if bounds_intersect(
                        extent.read_bounds, partition.core_bounds
                    )
                ),
            )
            for partition in manifest.tiles
        ),
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    publish_results = tuple(
        executor.map_batches(
            partial(
                _publish_support,
                source=source,
                background_rms_source=background_rms_source,
                detection_source=detection_source,
                fit_parent_source=fit_parent_source,
                config=config,
                sink=sink,
                core_shape_yx=manifest.tile_core_shape_yx,
            ),
            publish_batches,
            requirement=read_requirement(
                _SUPPORT_PUBLICATION_BYTES_PER_READ_PIXEL,
                (
                    *(partition.read_bounds for partition in manifest.tiles),
                    *(extent.read_bounds for extent in supported),
                ),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not publish_results:
        raise ValueError("executor returned no support publication results")
    return ComponentFitStageResult(
        generation=sink.publish_generation(
            product_names=_MEASUREMENT_PRODUCT_NAMES,
            chunks=(
                chunk
                for result in publish_results
                for chunk in result.product_chunks
            ),
        ),
        parents=tuple(parent for _, parent in measured),
        component_records=component_records,
        fit_parent_count=len(extents),
        deferred_parent_count=sum(
            int(parent.deferred) for _, parent in measured
        ),
        wide_parent_count=len(deferred),
        partition_count=len(manifest.tiles),
        executor_task_count=(
            len(scan_batches)
            + len(fit_batches)
            + len(deferred_batches)
            + len(publish_batches)
        ),
        maximum_graph_width=max(
            len(scan_batches),
            len(fit_batches),
            len(deferred_batches),
            len(publish_batches),
        ),
        maximum_parent_read_pixels=max(
            (
                *(result.maximum_parent_read_pixels for result in fit_results),
                *(
                    result.maximum_core_read_pixels
                    for result in deferred_results
                ),
                *(
                    result.maximum_parent_read_pixels
                    for result in publish_results
                ),
            ),
            default=0,
        ),
        parent_batch_count=len(fit_batches),
    )

hebog.stages.extended_groups

Extended groups: the components every connected support feature holds.

ExtendedGroupStageConfig dataclass

Reviewed grouping policy and the bounded task limits.

Source code in src/hebog/stages/extended_groups.py

@dataclass(frozen=True, slots=True)
class ExtendedGroupStageConfig:
    """Reviewed grouping policy and the bounded task limits."""

    atrous_plan: ResidualAtrousPlan
    detection_sigma: float
    island_sigma: float
    minimum_pixels: int
    maximum_bounds_pixels: int
    minimum_support_fraction: float
    maximum_tiles_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before any round is submitted."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")

    @property
    def margin_pixels(self) -> int:
        """Return the context one support feature reads beyond its bounds."""
        return support_feature_margin_pixels(self.atrous_plan)

margin_pixels property

Return the context one support feature reads beyond its bounds.

__post_init__()

Reject an unbounded task before any round is submitted.

Source code in src/hebog/stages/extended_groups.py

def __post_init__(self) -> None:
    """Reject an unbounded task before any round is submitted."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")

ExtendedGroupStageResult dataclass

Every support feature's extended groups and grouping evidence.

Source code in src/hebog/stages/extended_groups.py

@dataclass(frozen=True, slots=True)
class ExtendedGroupStageResult:
    """Every support feature's extended groups and grouping evidence."""

    features: tuple[SupportFeatureGroups, ...]
    feature_count: int
    grouped_feature_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_feature_read_pixels: int
    feature_batch_count: int
    reconciliation_round_count: int

SupportFeature dataclass

One reconciled support feature and the window that groups it.

Source code in src/hebog/stages/extended_groups.py

@dataclass(frozen=True, slots=True)
class SupportFeature:
    """One reconciled support feature and the window that groups it."""

    feature_label: int
    first_pixel_yx: tuple[int, int]
    window: ImageBounds

run_extended_group_stage(source, background_rms_source, detection_source, component_source, measurement_source, manifest, *, config, parents, wcs_header_text, beam, executor)

Group the components every connected support feature holds.

Two rounds and no write. The cores label the accumulated measurement support and observe which components lie in each local feature, and the reconciliation joins the features that meet across a core boundary. One task per batch of features then evaluates both cross-parent steps inside that feature's window, which is its reconciled bounds plus the reviewed margin.

wcs_header_text is the caller's own header as :meth:astropy.io.fits.Header.tostring writes it, not a WCS; see :func:~hebog.algorithms.astrometry.celestial_wcs_from_header_text.

Source code in src/hebog/stages/extended_groups.py

def run_extended_group_stage(  # noqa: PLR0913, PLR0917
    source: WindowReadable,
    background_rms_source: CompletedProductSource,
    detection_source: CompletedProductSource,
    component_source: CompletedProductSource,
    measurement_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: ExtendedGroupStageConfig,
    parents: tuple[FitParentMeasurement, ...],
    wcs_header_text: str,
    beam: RestoringBeam,
    executor: Executor,
) -> ExtendedGroupStageResult:
    """Group the components every connected support feature holds.

    Two rounds and no write. The cores label the accumulated measurement
    support and observe which components lie in each local feature, and the
    reconciliation joins the features that meet across a core boundary. One
    task per batch of features then evaluates both cross-parent steps inside
    that feature's window, which is its reconciled bounds plus the reviewed
    margin.

    ``wcs_header_text`` is the caller's own header as
    :meth:`astropy.io.fits.Header.tostring` writes it, not a ``WCS``; see
    :func:`~hebog.algorithms.astrometry.celestial_wcs_from_header_text`.
    """
    if manifest.halo_yx != (0, 0):
        raise ValueError("support features read cores without a halo")
    for product_source, names in (
        (background_rms_source, ("background", "rms")),
        (detection_source, ("valid-pixels",)),
        (component_source, ("component-measurement-labels",)),
        (measurement_source, ("measurement-support",)),
    ):
        if product_source.manifest.image_shape_yx != manifest.image_shape_yx:
            raise ValueError(
                "published generations must match the grouping image shape"
            )
        if not set(names).issubset(
            product_source.read_generation().product_names
        ):
            raise ValueError(
                "published generations must carry every grouping plane read"
            )
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_support_features,
                detection_source=detection_source,
                component_source=component_source,
                measurement_source=measurement_source,
                image_shape_yx=manifest.image_shape_yx,
            ),
            _feature_scan_batches(
                manifest.tiles,
                maximum_tiles_per_batch=config.maximum_tiles_per_batch,
            ),
            requirement=read_requirement(
                _FEATURE_SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no support-feature results")
    tiles = tuple(tile for result in scan_results for tile in result.tiles)
    reconciled = reconcile_candidate_tiles(
        manifest,
        tuple(tile.summary for tile in tiles),
    )
    features = _support_features(
        reconciled,
        margin=config.margin_pixels,
        image_shape_yx=manifest.image_shape_yx,
        maximum_bounds_pixels=config.maximum_bounds_pixels,
    )
    batches = _group_batches(
        features,
        core_shape_yx=manifest.tile_core_shape_yx,
        maximum_batch_read_pixels=config.maximum_batch_read_pixels,
        maximum_features_per_batch=objects_per_batch(
            len(features), executor.capacity
        ),
        maximum_bounds_pixels=config.maximum_bounds_pixels,
        fits=tuple(item for parent in parents for item in parent.fits),
        protected_labels=frozenset(
            index
            for parent in parents
            for group in parent.compact_groups
            for index in group
        ),
        component_bounds=_component_bounds(scan_results),
    )
    group_results: tuple[_GroupBatchResult, ...] = ()
    if batches:
        group_results = tuple(
            executor.map_batches(
                partial(
                    _group_batch,
                    source=source,
                    background_rms_source=background_rms_source,
                    detection_source=detection_source,
                    component_source=component_source,
                    measurement_source=measurement_source,
                    config=config,
                    wcs_header_text=wcs_header_text,
                    beam=beam,
                ),
                batches,
                requirement=read_requirement(
                    _GROUP_BYTES_PER_READ_PIXEL,
                    (batch.read_bounds for batch in batches),
                    chunk_shape_yx=manifest.tile_core_shape_yx,
                ),
            )
        )
        if not group_results:
            raise ValueError("executor returned no extended group results")
    return ExtendedGroupStageResult(
        features=tuple(
            groups
            for _, groups in sorted(
                (item for result in group_results for item in result.features),
                key=lambda item: item[0],
            )
        ),
        feature_count=len(reconciled.islands),
        grouped_feature_count=len(features),
        partition_count=len(manifest.tiles),
        executor_task_count=len(manifest.tiles) + len(batches),
        maximum_graph_width=max(len(manifest.tiles), len(batches)),
        maximum_feature_read_pixels=max(
            (result.maximum_feature_read_pixels for result in group_results),
            default=0,
        ),
        feature_batch_count=len(batches),
        reconciliation_round_count=reconciled.reduction_round_count,
    )

Source association

hebog.algorithms.multiscale_association

Deterministic bounded association of exact multiscale supports.

ScaleDetectionPlane dataclass

One bounded exact-support label plane and its local label records.

detections[index - 1] describes pixels labelled index. Labels are deliberately local and may change with partition or task order; stable detection identities determine every published association.

Source code in src/hebog/algorithms/multiscale_association.py

@dataclass(frozen=True, slots=True)
class ScaleDetectionPlane:
    """One bounded exact-support label plane and its local label records.

    ``detections[index - 1]`` describes pixels labelled ``index``. Labels are
    deliberately local and may change with partition or task order; stable
    detection identities determine every published association.
    """

    scale_order: int
    component_labels: npt.NDArray[np.int32]
    detections: tuple[ScaleDetection, ...]
    origin_yx: tuple[int, int] = (0, 0)

    def __post_init__(self) -> None:
        """Copy the bounded labels into immutable canonical storage."""
        if self.scale_order < 1:
            raise ValueError("scale order must be positive")
        labels = np.asarray(self.component_labels)
        if labels.ndim != _IMAGE_DIMENSIONS:
            raise ValueError("scale component labels must be two-dimensional")
        if not np.issubdtype(labels.dtype, np.integer):
            raise ValueError("scale component labels must be integers")
        if min(self.origin_yx) < 0:
            raise ValueError("scale component origin must be non-negative")
        if bool(np.any(labels < 0)):
            raise ValueError("scale component labels must be non-negative")
        if int(np.max(labels, initial=0)) > np.iinfo(np.int32).max:
            raise ValueError("scale component labels must fit signed 32-bit")
        canonical = np.asarray(labels, dtype=np.int32).copy()
        canonical.setflags(write=False)
        object.__setattr__(self, "component_labels", canonical)

__post_init__()

Copy the bounded labels into immutable canonical storage.

Source code in src/hebog/algorithms/multiscale_association.py

def __post_init__(self) -> None:
    """Copy the bounded labels into immutable canonical storage."""
    if self.scale_order < 1:
        raise ValueError("scale order must be positive")
    labels = np.asarray(self.component_labels)
    if labels.ndim != _IMAGE_DIMENSIONS:
        raise ValueError("scale component labels must be two-dimensional")
    if not np.issubdtype(labels.dtype, np.integer):
        raise ValueError("scale component labels must be integers")
    if min(self.origin_yx) < 0:
        raise ValueError("scale component origin must be non-negative")
    if bool(np.any(labels < 0)):
        raise ValueError("scale component labels must be non-negative")
    if int(np.max(labels, initial=0)) > np.iinfo(np.int32).max:
        raise ValueError("scale component labels must fit signed 32-bit")
    canonical = np.asarray(labels, dtype=np.int32).copy()
    canonical.setflags(write=False)
    object.__setattr__(self, "component_labels", canonical)

ScaleDetectionRecords dataclass

One scale's features, without the plane that labelled them.

Source code in src/hebog/algorithms/multiscale_association.py

@dataclass(frozen=True, slots=True)
class ScaleDetectionRecords:
    """One scale's features, without the plane that labelled them."""

    scale_order: int
    detections: tuple[ScaleDetection, ...]

    def __post_init__(self) -> None:
        """Reject an unconfigured scale before any decision reads it."""
        if self.scale_order < 1:
            raise ValueError("scale order must be positive")

__post_init__()

Reject an unconfigured scale before any decision reads it.

Source code in src/hebog/algorithms/multiscale_association.py

def __post_init__(self) -> None:
    """Reject an unconfigured scale before any decision reads it."""
    if self.scale_order < 1:
        raise ValueError("scale order must be positive")

ScaleDetections

Bases: Protocol

One scale's stable feature records, with or without its labels.

Every hierarchy decision reads identities, bounds and responses, never the label plane, so a pass that has already reconciled its features can supply :class:ScaleDetectionRecords and hold no image-sized array.

Source code in src/hebog/algorithms/multiscale_association.py

class ScaleDetections(Protocol):
    """One scale's stable feature records, with or without its labels.

    Every hierarchy decision reads identities, bounds and responses, never
    the label plane, so a pass that has already reconciled its features can
    supply :class:`ScaleDetectionRecords` and hold no image-sized array.
    """

    @property
    def scale_order(self) -> int:
        """Return the configured scale these features belong to."""
        ...

    @property
    def detections(self) -> tuple[ScaleDetection, ...]:
        """Return the features, ordered by their canonical first pixel."""
        ...

detections property

Return the features, ordered by their canonical first pixel.

scale_order property

Return the configured scale these features belong to.

adjacent_scale_overlap_edges(planes)

Return canonical fine-to-coarse exact-overlap hierarchy edges.

This is the same bounded overlap evidence used by :func:associate_adjacent_scale_detections, exposed so a catalogue-source hierarchy can retain parent direction without constructing another graph implementation.

Source code in src/hebog/algorithms/multiscale_association.py

def adjacent_scale_overlap_edges(
    planes: tuple[ScaleDetectionPlane, ...],
) -> tuple[tuple[str, str], ...]:
    """Return canonical fine-to-coarse exact-overlap hierarchy edges.

    This is the same bounded overlap evidence used by
    :func:`associate_adjacent_scale_detections`, exposed so a catalogue-source
    hierarchy can retain parent direction without constructing another graph
    implementation.
    """
    if not planes:
        return ()
    ordered_planes, _ = _validated_inputs(planes)
    edges: set[tuple[str, str]] = set()
    for first, second in pairwise(ordered_planes):
        if second.scale_order != first.scale_order + 1:
            continue
        first_labels = first.component_labels
        second_labels = second.component_labels
        overlap = (first_labels > 0) & (second_labels > 0)
        if not bool(np.any(overlap)):
            continue
        label_pairs = np.unique(
            np.column_stack((first_labels[overlap], second_labels[overlap])),
            axis=0,
        )
        for first_label, second_label in label_pairs:
            edges.add(
                (
                    first.detections[int(first_label) - 1].detection_id,
                    second.detections[int(second_label) - 1].detection_id,
                )
            )
    return tuple(sorted(edges))

associate_adjacent_scale_detections(planes)

Associate exact supports only across adjacent configured scales.

The kernel operates on bounded label planes. It creates vectorized overlap edges between adjacent scales and reduces their connected components to stable associations. Same-scale fragments can therefore join only through an accepted adjacent-scale path.

Source code in src/hebog/algorithms/multiscale_association.py

def associate_adjacent_scale_detections(
    planes: tuple[ScaleDetectionPlane, ...],
) -> tuple[CrossScaleAssociation, ...]:
    """Associate exact supports only across adjacent configured scales.

    The kernel operates on bounded label planes. It creates vectorized overlap
    edges between adjacent scales and reduces their connected components to
    stable associations. Same-scale fragments can therefore join only through
    an accepted adjacent-scale path.
    """
    if not planes:
        return ()
    ordered_planes, detections_by_id = _validated_inputs(planes)
    if not detections_by_id:
        return ()
    components = _DisjointDetections(tuple(detections_by_id))
    _join_adjacent_overlaps(ordered_planes, components)
    associations = tuple(
        _build_association(detection_ids, detections_by_id)
        for detection_ids in components.groups()
    )
    return tuple(sorted(associations, key=lambda item: item.association_id))

build_scale_detection_plane(significant_support, response_jy_per_beam, signal_to_noise, valid_pixels, *, scale_order, nominal_scale_beam_fwhm, origin_yx=(0, 0))

Build stable exact features from one reviewed significant scale mask.

Source code in src/hebog/algorithms/multiscale_association.py

def build_scale_detection_plane(  # noqa: PLR0913
    significant_support: npt.ArrayLike,
    response_jy_per_beam: npt.ArrayLike,
    signal_to_noise: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    *,
    scale_order: int,
    nominal_scale_beam_fwhm: float,
    origin_yx: tuple[int, int] = (0, 0),
) -> ScaleDetectionPlane:
    """Build stable exact features from one reviewed significant scale mask."""
    support = np.asarray(significant_support)
    response = np.asarray(response_jy_per_beam)
    snr = np.asarray(signal_to_noise)
    valid = np.asarray(valid_pixels)
    if support.ndim != _IMAGE_DIMENSIONS or support.dtype != np.bool_:
        raise ValueError("significant scale support must be a boolean plane")
    if valid.shape != support.shape or valid.dtype != np.bool_:
        raise ValueError("scale validity must be one aligned boolean plane")
    if any(
        array.shape != support.shape
        or not np.issubdtype(array.dtype, np.number)
        or np.iscomplexobj(array)
        for array in (response, snr)
    ):
        raise ValueError("scale response and SNR must be aligned real planes")
    if np.any(support & ~valid):
        raise ValueError("scale support must be scientifically valid")
    if (
        isinstance(scale_order, bool)
        or scale_order < 1
        or not isfinite(nominal_scale_beam_fwhm)
        or nominal_scale_beam_fwhm <= 0.0
        or len(origin_yx) != _IMAGE_DIMENSIONS
        or min(origin_yx) < 0
    ):
        raise ValueError("scale metadata must be positive and canonical")
    labels, count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(
            support,
            structure=np.ones((3, 3), dtype=np.int8),
        ),
    )
    groups = group_labelled_pixels(labels, label_count=count)
    feature_response = np.asarray(response, dtype=np.float64).reshape(-1)[
        groups.flat_positions
    ]
    feature_snr = np.asarray(snr, dtype=np.float64).reshape(-1)[
        groups.flat_positions
    ]
    if (
        not bool(np.all(np.isfinite(feature_response)))
        or not bool(np.all(np.isfinite(feature_snr)))
        or bool(np.any(groups.maximum(feature_response) <= 0.0))
    ):
        raise ValueError(
            "significant scale features require finite positive response"
        )
    return ScaleDetectionPlane(
        scale_order=scale_order,
        component_labels=np.asarray(labels, dtype=np.int32),
        detections=_scale_detections(
            groups,
            count=count,
            peak_response_jy_per_beam=groups.maximum(feature_response),
            peak_signal_to_noise=groups.maximum(feature_snr),
            support_shape_yx=support.shape,
            origin_yx=origin_yx,
            scale_order=scale_order,
            nominal_scale_beam_fwhm=nominal_scale_beam_fwhm,
        ),
        origin_yx=origin_yx,
    )

build_scale_detection_plane_from_islands(significant_support, islands, *, scale_order, nominal_scale_beam_fwhm)

Describe one scale from its stored support and reconciled islands.

A tiled pass reduces each feature's peak response and signal to noise while the filter response is still in the task's memory, so a later pass can describe the same features from the stored support mask alone. The reconciled islands are ordered by their globally row-major canonical pixel, which is the order scipy.ndimage.label assigns to the same mask, so the labelling here reproduces the tiled global labels exactly.

Source code in src/hebog/algorithms/multiscale_association.py

def build_scale_detection_plane_from_islands(
    significant_support: npt.ArrayLike,
    islands: tuple[DetectedIsland, ...],
    *,
    scale_order: int,
    nominal_scale_beam_fwhm: float,
) -> ScaleDetectionPlane:
    """Describe one scale from its stored support and reconciled islands.

    A tiled pass reduces each feature's peak response and signal to noise
    while the filter response is still in the task's memory, so a later pass
    can describe the same features from the stored support mask alone. The
    reconciled islands are ordered by their globally row-major canonical
    pixel, which is the order ``scipy.ndimage.label`` assigns to the same
    mask, so the labelling here reproduces the tiled global labels exactly.
    """
    support = np.asarray(significant_support)
    if support.ndim != _IMAGE_DIMENSIONS or support.dtype != np.bool_:
        raise ValueError("significant scale support must be a boolean plane")
    labels, count = cast(
        tuple[npt.NDArray[np.int32], int],
        connected_component_labels(
            support,
            structure=np.ones((3, 3), dtype=np.int8),
        ),
    )
    if count != len(islands):
        raise ValueError(
            "reconciled scale islands must describe the stored support once"
        )
    groups = group_labelled_pixels(labels, label_count=count)
    canonical = tuple(
        (int(groups.first_y[index]), int(groups.first_x[index]))
        for index in range(count)
    )
    if any(
        island.global_label != index + 1
        or island.first_pixel_yx != canonical[index]
        or island.pixel_count != int(groups.sizes[index])
        for index, island in enumerate(islands)
    ):
        raise ValueError(
            "reconciled scale islands must match the stored support exactly"
        )
    peak_response = np.asarray(
        [island.peak_response_jy_per_beam for island in islands],
        dtype=np.float64,
    )
    peak_snr = np.asarray(
        [island.peak_signal_to_noise for island in islands],
        dtype=np.float64,
    )
    if (
        not bool(np.all(np.isfinite(peak_response)))
        or not bool(np.all(np.isfinite(peak_snr)))
        or bool(np.any(peak_response <= 0.0))
    ):
        raise ValueError(
            "significant scale features require finite positive response"
        )
    return ScaleDetectionPlane(
        scale_order=scale_order,
        component_labels=np.asarray(labels, dtype=np.int32),
        detections=_scale_detections(
            groups,
            count=count,
            peak_response_jy_per_beam=peak_response,
            peak_signal_to_noise=peak_snr,
            support_shape_yx=support.shape,
            origin_yx=(0, 0),
            scale_order=scale_order,
            nominal_scale_beam_fwhm=nominal_scale_beam_fwhm,
        ),
    )

persistent_adjacent_scale_support(planes)

Return exact support whose features persist across adjacent scales.

Source code in src/hebog/algorithms/multiscale_association.py

def persistent_adjacent_scale_support(
    planes: tuple[ScaleDetectionPlane, ...],
) -> npt.NDArray[np.bool_]:
    """Return exact support whose features persist across adjacent scales."""
    if not planes:
        raise ValueError("persistent support requires a scale detection plane")
    ordered_planes, _ = _validated_inputs(planes)
    retained = persistent_scale_labels(
        ordered_planes,
        adjacent_scale_overlap_edges(ordered_planes),
    )
    support = np.zeros(
        ordered_planes[0].component_labels.shape,
        dtype=np.bool_,
    )
    for plane in ordered_planes:
        support |= persistent_scale_support_window(
            plane.component_labels,
            retained[plane.scale_order],
        )
    support.setflags(write=False)
    return support

persistent_scale_labels(scales, parent_edges)

Return each scale's labels whose feature persists to a neighbour.

Persistence is decided from the reduced adjacent-scale overlap edges and the feature records alone, so it holds no plane.

Source code in src/hebog/algorithms/multiscale_association.py

def persistent_scale_labels(
    scales: tuple[ScaleDetections, ...],
    parent_edges: tuple[tuple[str, str], ...],
) -> dict[int, tuple[int, ...]]:
    """Return each scale's labels whose feature persists to a neighbour.

    Persistence is decided from the reduced adjacent-scale overlap edges and
    the feature records alone, so it holds no plane.
    """
    detections_by_id = {
        detection.detection_id: detection
        for scale in scales
        for detection in scale.detections
    }
    components = _DisjointDetections(tuple(detections_by_id))
    for child_id, parent_id in parent_edges:
        components.union(child_id, parent_id)
    persistent_ids = {
        detection_id
        for detection_ids in components.groups()
        for association in (
            _build_association(detection_ids, detections_by_id),
        )
        if (
            len(association.contributing_scale_orders)
            >= _MINIMUM_PERSISTENT_SCALE_COUNT
        )
        for detection_id in association.scale_detection_ids
    }
    return {
        scale.scale_order: tuple(
            index
            for index, detection in enumerate(scale.detections, start=1)
            if detection.detection_id in persistent_ids
        )
        for scale in scales
    }

persistent_scale_support_window(labels, retained_labels)

Return the pixels of one window that one scale's retained labels own.

Source code in src/hebog/algorithms/multiscale_association.py

def persistent_scale_support_window(
    labels: npt.NDArray[np.int32],
    retained_labels: tuple[int, ...],
) -> npt.NDArray[np.bool_]:
    """Return the pixels of one window that one scale's retained labels own."""
    if not retained_labels:
        return np.zeros(labels.shape, dtype=np.bool_)
    return np.asarray(
        np.isin(labels, np.asarray(retained_labels, dtype=np.int32)),
        dtype=np.bool_,
    )

persistent_seeded_scale_support(scale_snrs, scale_responses, valid, *, detection_sigma, island_sigma, minimum_pixels)

Apply one seeded adjacent-scale rule to non-publication support.

Callers pair each noise-calibrated SNR with the same beam-aware filtered response in Jy/beam, not the unfiltered residual. Source-protected statistics and source-owned photometry can reuse this support without admitting new catalogue detections or changing a published mask.

Source code in src/hebog/algorithms/multiscale_association.py

def persistent_seeded_scale_support(  # noqa: PLR0913
    scale_snrs: tuple[npt.NDArray[np.float64], ...],
    scale_responses: tuple[npt.NDArray[np.float64], ...],
    valid: npt.NDArray[np.bool_],
    *,
    detection_sigma: float,
    island_sigma: float,
    minimum_pixels: int,
) -> npt.NDArray[np.bool_]:
    """Apply one seeded adjacent-scale rule to non-publication support.

    Callers pair each noise-calibrated SNR with the same beam-aware filtered
    response in Jy/beam, not the unfiltered residual. Source-protected
    statistics and source-owned photometry can reuse this support without
    admitting new catalogue detections or changing a published mask.
    """
    planes: list[ScaleDetectionPlane] = []
    for order, (snr, response) in enumerate(
        zip(scale_snrs, scale_responses, strict=True), start=1
    ):
        labels, count = cast(
            tuple[npt.NDArray[np.int32], int],
            connected_component_labels(
                valid & (snr >= island_sigma), np.ones((3, 3))
            ),
        )
        sizes = np.bincount(labels.ravel(), minlength=count + 1)
        seeded = np.unique(labels[valid & (snr >= detection_sigma)])
        accepted = np.zeros(count + 1, dtype=np.bool_)
        accepted[seeded] = sizes[seeded] >= minimum_pixels
        accepted[0] = False
        planes.append(
            build_scale_detection_plane(
                accepted[labels],
                response,
                snr,
                valid,
                scale_order=order,
                nominal_scale_beam_fwhm=float(2 ** (order - 1)),
            )
        )
    return persistent_adjacent_scale_support(tuple(planes))

scale_detection_id(scale_order, canonical_pixel_yx)

Derive one stable feature identity from scale and global owner pixel.

A later pass names the same features from the reconciled island records alone, so this identity is part of the published contract rather than an internal detail of the plane builder.

Source code in src/hebog/algorithms/multiscale_association.py

def scale_detection_id(
    scale_order: int,
    canonical_pixel_yx: tuple[int, int],
) -> str:
    """Derive one stable feature identity from scale and global owner pixel.

    A later pass names the same features from the reconciled island records
    alone, so this identity is part of the published contract rather than an
    internal detail of the plane builder.
    """
    digest = sha256(_DETECTION_ID_NAMESPACE)
    for value in (scale_order, *canonical_pixel_yx):
        digest.update(str(value).encode("ascii"))
        digest.update(b"\0")
    return f"scale-detection-{digest.hexdigest()}"

scale_detections_from_islands(islands, *, scale_order, nominal_scale_beam_fwhm)

Describe one scale's features from its reconciled islands alone.

A tiled pass reduces each feature's extent, canonical pixel and peak response while its filter response is still in the task's memory, so a later pass names the same features without labelling a plane again.

Source code in src/hebog/algorithms/multiscale_association.py

def scale_detections_from_islands(
    islands: tuple[DetectedIsland, ...],
    *,
    scale_order: int,
    nominal_scale_beam_fwhm: float,
) -> ScaleDetectionRecords:
    """Describe one scale's features from its reconciled islands alone.

    A tiled pass reduces each feature's extent, canonical pixel and peak
    response while its filter response is still in the task's memory, so a
    later pass names the same features without labelling a plane again.
    """
    return ScaleDetectionRecords(
        scale_order=scale_order,
        detections=tuple(
            ScaleDetection(
                detection_id=scale_detection_id(
                    scale_order, island.first_pixel_yx
                ),
                parent_island_id=None,
                scale_order=scale_order,
                nominal_scale_beam_fwhm=nominal_scale_beam_fwhm,
                support_pixel_count=island.pixel_count,
                valid_support_fraction=1.0,
                bounds_yx=(
                    island.bounds.y_start,
                    island.bounds.y_stop,
                    island.bounds.x_start,
                    island.bounds.x_stop,
                ),
                canonical_pixel_yx=island.first_pixel_yx,
                peak_response_jy_per_beam=_island_peak_response(island),
                peak_signal_to_noise=island.peak_signal_to_noise,
                touches_image_edge=island.touches_image_edge,
            )
            for island in islands
        ),
    )

hebog.algorithms.component_records

Stable records of the detection components sources are built from.

build_detection_component_record(label_value, raster_indices, association_signal, *, image_width)

Describe one component from its own valid support pixels.

raster_indices are global y * image_width + x positions in ascending raster order, and association_signal is aligned with them. A component too wide to read at once arrives as pixels from each core that holds it; restored to raster order, they are exactly the pixels a window over it presents, so this is the record :func:build_detection_component_records builds there, bit for bit.

Raises:

Type	Description
ValueError	If the component has no pixel, the arrays disagree, or the pixels are not in strictly ascending raster order.

Source code in src/hebog/algorithms/component_records.py

def build_detection_component_record(
    label_value: int,
    raster_indices: npt.NDArray[np.int64],
    association_signal: npt.NDArray[np.float64],
    *,
    image_width: int,
) -> DetectionComponentRecord:
    """Describe one component from its own valid support pixels.

    ``raster_indices`` are global ``y * image_width + x`` positions in
    ascending raster order, and ``association_signal`` is aligned with them.
    A component too wide to read at once arrives as pixels from each core
    that holds it; restored to raster order, they are exactly the pixels a
    window over it presents, so this is the record
    :func:`build_detection_component_records` builds there, bit for bit.

    Raises:
        ValueError: If the component has no pixel, the arrays disagree, or
            the pixels are not in strictly ascending raster order.
    """
    indices = np.asarray(raster_indices, dtype=np.int64)
    signal = np.asarray(association_signal, dtype=np.float64)
    if indices.ndim != 1 or indices.shape != signal.shape or not indices.size:
        raise ValueError(
            "component pixels must be non-empty and aligned with their signal"
        )
    if bool(np.any(np.diff(indices) <= 0)):
        raise ValueError("component pixels must be in ascending raster order")
    y_pixels, x_pixels = np.divmod(indices, image_width)
    return _component_record(
        label_value,
        y_pixels,
        x_pixels,
        signal,
        canonical_reference_yx=None,
    )

build_detection_component_records(component_labels, association_signal, valid_pixels, *, origin_yx=(0, 0), canonical_component_references_yx=None)

Build stable component records from immutable exact owner support.

Source code in src/hebog/algorithms/component_records.py

def build_detection_component_records(
    component_labels: npt.ArrayLike,
    association_signal: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    *,
    origin_yx: tuple[int, int] = (0, 0),
    canonical_component_references_yx: (
        Mapping[int, tuple[int, int]] | None
    ) = None,
) -> tuple[DetectionComponentRecord, ...]:
    """Build stable component records from immutable exact owner support."""
    if len(origin_yx) != _IMAGE_DIMENSIONS or min(origin_yx) < 0:
        raise ValueError("component origin must contain non-negative y-x")
    labels, signal, valid = validated_component_planes(
        component_labels,
        association_signal,
        valid_pixels,
    )
    records: list[DetectionComponentRecord] = []
    # One pass gives every component its window, so the geometry below
    # costs each component's own pixels instead of the whole plane.
    component_windows = label_windows(labels)
    for label_value in sorted(
        int(value) for value in np.unique(labels) if value > 0
    ):
        crop = component_windows[label_value - 1]
        if crop is None:
            raise ValueError("component labels must own at least one pixel")
        support = (labels[crop] == label_value) & valid[crop]
        local_y, local_x = np.nonzero(support)
        records.append(
            _component_record(
                label_value,
                local_y + (origin_yx[0] + crop[0].start),
                local_x + (origin_yx[1] + crop[1].start),
                signal[crop][support],
                canonical_reference_yx=(
                    None
                    if canonical_component_references_yx is None
                    else canonical_component_references_yx.get(label_value)
                ),
            )
        )
    return tuple(sorted(records, key=lambda item: item.canonical_pixel_yx))

stable_id(namespace, prefix, values)

Hash canonical identity fields into one stable domain identifier.

Source code in src/hebog/algorithms/component_records.py

def stable_id(namespace: bytes, prefix: str, values: tuple[str, ...]) -> str:
    """Hash canonical identity fields into one stable domain identifier."""
    digest = sha256(namespace)
    for value in values:
        digest.update(value.encode("ascii"))
        digest.update(b"\0")
    return f"{prefix}-{digest.hexdigest()}"

validated_component_planes(component_labels, association_signal, valid_pixels)

Return aligned component, signal, and validity planes.

Source code in src/hebog/algorithms/component_records.py

def validated_component_planes(
    component_labels: npt.ArrayLike,
    association_signal: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
) -> tuple[
    npt.NDArray[np.int64],
    npt.NDArray[np.float64],
    npt.NDArray[np.bool_],
]:
    """Return aligned component, signal, and validity planes."""
    label_values = np.asarray(component_labels)
    signal_values = np.asarray(association_signal)
    valid = np.asarray(valid_pixels)
    if (
        label_values.ndim != _IMAGE_DIMENSIONS
        or not np.issubdtype(label_values.dtype, np.integer)
        or bool(np.any(label_values < 0))
    ):
        raise ValueError(
            "component labels must be a non-negative integer plane"
        )
    if (
        signal_values.ndim != _IMAGE_DIMENSIONS
        or signal_values.shape != label_values.shape
        or not np.issubdtype(signal_values.dtype, np.number)
        or np.iscomplexobj(signal_values)
    ):
        raise ValueError("association signal must be one aligned real plane")
    if (
        valid.ndim != _IMAGE_DIMENSIONS
        or valid.shape != label_values.shape
        or valid.dtype != np.bool_
    ):
        raise ValueError("valid pixels must be one aligned boolean plane")
    if bool(np.any((label_values > 0) & ~valid)):
        raise ValueError("component owner pixels must be scientifically valid")
    return (
        np.asarray(label_values, dtype=np.int64),
        np.asarray(signal_values, dtype=np.float64),
        np.asarray(valid, dtype=np.bool_),
    )

hebog.algorithms.feature_envelopes

Scale-feature envelopes and the overlap summary association decides on.

The association stage builds the same summary tile by tile from these envelopes; :func:summarize_hierarchy_overlaps builds it from whole planes.

FeatureOverlaps dataclass

Every pixel fact the hierarchy decision needs about one feature.

The feature is the object of this work: each field is derived inside the feature's own exact bounds plus the reviewed B3 footprint, so one task can produce it without seeing the plane around it.

Source code in src/hebog/algorithms/feature_envelopes.py

@dataclass(frozen=True, slots=True)
class FeatureOverlaps:
    """Every pixel fact the hierarchy decision needs about one feature.

    The feature is the object of this work: each field is derived inside the
    feature's own exact bounds plus the reviewed B3 footprint, so one task can
    produce it without seeing the plane around it.
    """

    feature_id: str
    scale_order: int
    exact_component_ids: frozenset[str]
    has_envelope: bool
    influence_component_ids: frozenset[str]
    support_component: int

HierarchyOverlaps dataclass

Every pixel fact the source hierarchy decision needs.

Each field is a reduction that a tile core or one feature's window can produce and that merges associatively, so the decision that consumes it is pure record logic and cannot depend on tile geometry. support_component is zero where a feature is not wholly inside one retained support component. influence_component_ids is filled only for the features :func:influence_candidate_feature_ids admits, and envelope_edges only for the scale pairs :func:envelope_pair_is_needed admits, because no decision reads the rest and deriving them is pixel work.

Source code in src/hebog/algorithms/feature_envelopes.py

@dataclass(frozen=True, slots=True)
class HierarchyOverlaps:
    """Every pixel fact the source hierarchy decision needs.

    Each field is a reduction that a tile core or one feature's window can
    produce and that merges associatively, so the decision that consumes it
    is pure record logic and cannot depend on tile geometry.
    ``support_component`` is zero where a feature is not wholly inside one
    retained support component. ``influence_component_ids`` is filled only
    for the features :func:`influence_candidate_feature_ids` admits, and
    ``envelope_edges`` only for the scale pairs
    :func:`envelope_pair_is_needed` admits, because no decision reads the
    rest and deriving them is pixel work.
    """

    features: tuple[FeatureOverlaps, ...]
    finest_features_by_component: Mapping[str, tuple[str, ...]]
    parent_edges: tuple[tuple[str, str], ...]
    support_component_by_component: Mapping[str, int]
    envelope_edges: frozenset[frozenset[str]]

    def by_id(self) -> dict[str, FeatureOverlaps]:
        """Index the feature overlaps by their stable identities."""
        return {item.feature_id: item for item in self.features}

    def adjacency(self, feature_ids: frozenset[str]) -> dict[str, set[str]]:
        """Return the envelope overlap graph over the named features."""
        enveloped = {
            item.feature_id
            for item in self.features
            if item.has_envelope and item.feature_id in feature_ids
        }
        adjacency: dict[str, set[str]] = {
            feature_id: set() for feature_id in enveloped
        }
        for edge in self.envelope_edges:
            first, second = sorted(edge)
            if first in enveloped and second in enveloped:
                adjacency[first].add(second)
                adjacency[second].add(first)
        return adjacency

adjacency(feature_ids)

Return the envelope overlap graph over the named features.

Source code in src/hebog/algorithms/feature_envelopes.py

def adjacency(self, feature_ids: frozenset[str]) -> dict[str, set[str]]:
    """Return the envelope overlap graph over the named features."""
    enveloped = {
        item.feature_id
        for item in self.features
        if item.has_envelope and item.feature_id in feature_ids
    }
    adjacency: dict[str, set[str]] = {
        feature_id: set() for feature_id in enveloped
    }
    for edge in self.envelope_edges:
        first, second = sorted(edge)
        if first in enveloped and second in enveloped:
            adjacency[first].add(second)
            adjacency[second].add(first)
    return adjacency

by_id()

Index the feature overlaps by their stable identities.

Source code in src/hebog/algorithms/feature_envelopes.py

def by_id(self) -> dict[str, FeatureOverlaps]:
    """Index the feature overlaps by their stable identities."""
    return {item.feature_id: item for item in self.features}

envelope_pair_is_needed(first_scale_order, second_scale_order, *, terminal_scale_order)

Return whether any decision asks about this pair of scales.

Within-scale overlap builds each scale's adjacency graph, and the last two scales are paired once to corroborate a displaced terminal child. No other combination is read.

Source code in src/hebog/algorithms/feature_envelopes.py

def envelope_pair_is_needed(
    first_scale_order: int,
    second_scale_order: int,
    *,
    terminal_scale_order: int,
) -> bool:
    """Return whether any decision asks about this pair of scales.

    Within-scale overlap builds each scale's adjacency graph, and the last
    two scales are paired once to corroborate a displaced terminal child.
    No other combination is read.
    """
    if first_scale_order == second_scale_order:
        return True
    return {first_scale_order, second_scale_order} == {
        terminal_scale_order - 1,
        terminal_scale_order,
    }

influence_candidate_feature_ids(exact_component_ids, parent_edges)

Return the features whose B3 influence a decision can ever read.

:func:_feature_influence_candidate rejects every other feature on record evidence alone, so deriving their influence would be pixel work nothing consumes.

Source code in src/hebog/algorithms/feature_envelopes.py

def influence_candidate_feature_ids(
    exact_component_ids: Mapping[str, frozenset[str]],
    parent_edges: tuple[tuple[str, str], ...],
) -> frozenset[str]:
    """Return the features whose B3 influence a decision can ever read.

    :func:`_feature_influence_candidate` rejects every other feature on
    record evidence alone, so deriving their influence would be pixel work
    nothing consumes.
    """
    parents: dict[str, set[str]] = {}
    children: dict[str, set[str]] = {}
    for child_id, parent_id in parent_edges:
        parents.setdefault(child_id, set()).add(parent_id)
        children.setdefault(parent_id, set()).add(child_id)
    candidates: set[str] = set()
    for feature_id, child_ids in children.items():
        if len(child_ids) != 1:
            continue
        child_id = next(iter(child_ids))
        exact = exact_component_ids.get(feature_id, frozenset())
        if (
            parents.get(child_id) != {feature_id}
            or len(exact) != 1
            or exact_component_ids.get(child_id, frozenset()) != exact
        ):
            continue
        candidates.add(feature_id)
    return frozenset(candidates)

scale_feature_envelope_bounds(feature_bounds, *, scale_order, image_shape_yx)

Return one feature's B3 envelope box, or absence beyond the footprint.

The box follows from the feature's own bounds, so a driver knows it from records alone and one task can evaluate the envelope inside it.

Source code in src/hebog/algorithms/feature_envelopes.py

def scale_feature_envelope_bounds(
    feature_bounds: ImageBounds,
    *,
    scale_order: int,
    image_shape_yx: tuple[int, int],
) -> ImageBounds | None:
    """Return one feature's B3 envelope box, or absence beyond the footprint.

    The box follows from the feature's own bounds, so a driver knows it from
    records alone and one task can evaluate the envelope inside it.
    """
    halos = residual_atrous_scale_halos_pixels()
    if scale_order > len(halos):
        return None
    radius = halos[scale_order - 1]
    return ImageBounds(
        max(0, feature_bounds.y_start - radius),
        min(image_shape_yx[0], feature_bounds.y_stop + radius),
        max(0, feature_bounds.x_start - radius),
        min(image_shape_yx[1], feature_bounds.x_stop + radius),
    )

scale_feature_envelope_support(exact_support, valid, *, scale_order)

Dilate one feature's exact support over its own envelope window.

Both arrays cover the envelope box that :func:scale_feature_envelope_bounds returns.

Source code in src/hebog/algorithms/feature_envelopes.py

def scale_feature_envelope_support(
    exact_support: npt.NDArray[np.bool_],
    valid: npt.NDArray[np.bool_],
    *,
    scale_order: int,
) -> npt.NDArray[np.bool_]:
    """Dilate one feature's exact support over its own envelope window.

    Both arrays cover the envelope box that
    :func:`scale_feature_envelope_bounds` returns.
    """
    radius = residual_atrous_scale_halos_pixels()[scale_order - 1]
    return np.asarray(
        binary_dilation(
            exact_support,
            structure=np.ones((3, 3), dtype=np.bool_),
            iterations=radius,
            mask=valid,
        ),
        dtype=np.bool_,
    )

scale_feature_envelopes_overlap(first_support, first_bounds, second_support, second_bounds)

Return whether two bounded envelopes share a supported pixel.

Source code in src/hebog/algorithms/feature_envelopes.py

def scale_feature_envelopes_overlap(
    first_support: npt.NDArray[np.bool_],
    first_bounds: ImageBounds,
    second_support: npt.NDArray[np.bool_],
    second_bounds: ImageBounds,
) -> bool:
    """Return whether two bounded envelopes share a supported pixel."""
    y0 = max(first_bounds.y_start, second_bounds.y_start)
    y1 = min(first_bounds.y_stop, second_bounds.y_stop)
    x0 = max(first_bounds.x_start, second_bounds.x_start)
    x1 = min(first_bounds.x_stop, second_bounds.x_stop)
    if y0 >= y1 or x0 >= x1:
        return False
    return bool(
        np.any(
            first_support[
                y0 - first_bounds.y_start : y1 - first_bounds.y_start,
                x0 - first_bounds.x_start : x1 - first_bounds.x_start,
            ]
            & second_support[
                y0 - second_bounds.y_start : y1 - second_bounds.y_start,
                x0 - second_bounds.x_start : x1 - second_bounds.x_start,
            ]
        )
    )

scale_feature_influence_bounds(envelope_bounds, *, scale_order, image_shape_yx)

Return the symmetric reviewed B3 influence box of one envelope.

Source code in src/hebog/algorithms/feature_envelopes.py

def scale_feature_influence_bounds(
    envelope_bounds: ImageBounds,
    *,
    scale_order: int,
    image_shape_yx: tuple[int, int],
) -> ImageBounds:
    """Return the symmetric reviewed B3 influence box of one envelope."""
    radius = residual_atrous_scale_halos_pixels()[scale_order - 1]
    return ImageBounds(
        max(0, envelope_bounds.y_start - radius),
        min(image_shape_yx[0], envelope_bounds.y_stop + radius),
        max(0, envelope_bounds.x_start - radius),
        min(image_shape_yx[1], envelope_bounds.x_stop + radius),
    )

scale_feature_influence_component_labels(envelope_seed, valid, component_labels, *, scale_order)

Return the owner labels within the symmetric reviewed B3 influence.

Every array covers the influence box that :func:scale_feature_influence_bounds returns, with the envelope support already placed at its offset inside envelope_seed. The labels are those of component_labels; the caller names their components.

Source code in src/hebog/algorithms/feature_envelopes.py

def scale_feature_influence_component_labels(
    envelope_seed: npt.NDArray[np.bool_],
    valid: npt.NDArray[np.bool_],
    component_labels: npt.NDArray[np.int64],
    *,
    scale_order: int,
) -> frozenset[int]:
    """Return the owner labels within the symmetric reviewed B3 influence.

    Every array covers the influence box that
    :func:`scale_feature_influence_bounds` returns, with the envelope support
    already placed at its offset inside ``envelope_seed``. The labels are
    those of ``component_labels``; the caller names their components.
    """
    radius = residual_atrous_scale_halos_pixels()[scale_order - 1]
    influence = np.asarray(
        binary_dilation(
            envelope_seed,
            structure=np.ones((3, 3), dtype=np.bool_),
            iterations=radius,
            mask=valid,
        ),
        dtype=np.bool_,
    )
    return frozenset(
        int(label_value)
        for label_value in np.unique(component_labels[influence])
        if int(label_value) > 0
    )

summarize_hierarchy_overlaps(records, labels, planes, valid, significant)

Derive every pixel fact the hierarchy decision needs, whole-plane.

This is the serial oracle for the tiled rounds: each field is the same reduction a tile core or one feature's window produces, evaluated here over complete planes.

Source code in src/hebog/algorithms/feature_envelopes.py

def summarize_hierarchy_overlaps(
    records: tuple[DetectionComponentRecord, ...],
    labels: npt.NDArray[np.int64],
    planes: tuple[ScaleDetectionPlane, ...],
    valid: npt.NDArray[np.bool_],
    significant: npt.NDArray[np.bool_],
) -> HierarchyOverlaps:
    """Derive every pixel fact the hierarchy decision needs, whole-plane.

    This is the serial oracle for the tiled rounds: each field is the same
    reduction a tile core or one feature's window produces, evaluated here
    over complete planes.
    """
    component_id_by_label = {
        record.label_value: record.component_id for record in records
    }
    support_labels, support_by_component = _connected_support_components(
        records, labels, significant, valid
    )
    envelopes = tuple(
        envelope
        for plane in planes
        for envelope in _feature_envelopes(plane, valid)
    )
    enveloped = {envelope.feature_id: envelope for envelope in envelopes}
    scale_by_feature: dict[str, int] = {}
    exact_by_feature: dict[str, frozenset[str]] = {}
    support_by_feature: dict[str, int] = {}
    plane_by_feature: dict[str, tuple[ScaleDetectionPlane, int]] = {}
    for plane in planes:
        supports = _feature_support_components(plane, support_labels)
        for feature_label, detection in enumerate(plane.detections, start=1):
            feature_id = detection.detection_id
            scale_by_feature[feature_id] = plane.scale_order
            plane_by_feature[feature_id] = (plane, feature_label)
            exact_by_feature[feature_id] = _feature_exact_component_ids(
                plane, feature_label, labels, component_id_by_label
            )
            support_by_feature[feature_id] = supports.get(feature_id, 0)
    parent_edges = adjacent_scale_overlap_edges(planes)
    influence_candidates = influence_candidate_feature_ids(
        exact_by_feature, parent_edges
    )
    return HierarchyOverlaps(
        features=tuple(
            FeatureOverlaps(
                feature_id=feature_id,
                scale_order=scale_by_feature[feature_id],
                exact_component_ids=exact_by_feature[feature_id],
                has_envelope=feature_id in enveloped,
                influence_component_ids=_feature_influence_component_ids(
                    plane_by_feature[feature_id][0],
                    enveloped[feature_id],
                    labels,
                    valid,
                    component_id_by_label,
                )
                if feature_id in influence_candidates
                and feature_id in enveloped
                else frozenset(),
                support_component=support_by_feature[feature_id],
            )
            for feature_id in scale_by_feature
        ),
        finest_features_by_component={
            record.component_id: _attached_finest_features(
                record, labels, planes
            )
            for record in records
        },
        parent_edges=parent_edges,
        support_component_by_component=support_by_component,
        envelope_edges=_envelope_overlap_edges(
            envelopes,
            scale_by_feature,
            terminal_scale_order=planes[-1].scale_order,
        ),
    )

hebog.algorithms.source_association

Conservative deterministic grouping of immutable detection components.

associate_components_by_multiscale_hierarchy(records, component_labels, scale_detection_planes, valid_pixels, *, significant_multiscale_support=None)

Partition direct owners at a corroborated common scale feature.

Direct component labels are immutable. Each owner attaches to the finest exact undilated scale features it intersects, then follows only unique adjacent-scale overlap parents. Owners group at their finest shared feature only when that feature persists to a parent, or when every owner directly attaches to the same feature. When exact sibling lineages remain separate, connected significant multiscale support may corroborate a persistent mutually unique pair or non-terminal cycle, but cannot create source membership by itself. A mutually unique adjacent-scale feature may also recover exactly one unresolved displaced owner when the parent and child share one direct anchor and two applications of the same fixed B3 footprint contain only that anchor and the displaced owner. This persistent feature evidence is self-corroborating, but it remains subject to whole-group reconciliation and cannot split or partially overlap an established source. At the last retained scale, a cycle may construct a parent when its constituent features have exact or mutually unique displaced children. One directly owned feature may lack a child without vetoing an otherwise persistent whole-source graph, but only when every feature shares one retained support component and no direct member has a competing cycle. A persistent feature without a direct owner may corroborate cycle geometry but cannot join catalogue membership, which still requires at least three immutable direct components. Displaced and missing-child evidence cannot create cycles or membership. Partial overlap with an exact group, missing support, branching, or conflicting convergence fails closed.

Source code in src/hebog/algorithms/source_association.py

def associate_components_by_multiscale_hierarchy(
    records: tuple[DetectionComponentRecord, ...],
    component_labels: npt.ArrayLike,
    scale_detection_planes: tuple[ScaleDetectionPlane, ...],
    valid_pixels: npt.ArrayLike,
    *,
    significant_multiscale_support: npt.ArrayLike | None = None,
) -> SourceAssociationResult:
    """Partition direct owners at a corroborated common scale feature.

    Direct component labels are immutable. Each owner attaches to the finest
    exact undilated scale features it intersects, then follows only unique
    adjacent-scale overlap parents. Owners group at their finest shared
    feature only when that feature persists to a parent, or when every owner
    directly attaches to the same feature. When exact sibling lineages remain
    separate, connected significant multiscale support may corroborate a
    persistent mutually unique pair or non-terminal cycle, but cannot create
    source membership by itself.
    A mutually unique adjacent-scale feature may also recover exactly one
    unresolved displaced owner when the parent and child share one direct
    anchor and two applications of the same fixed B3 footprint contain only
    that anchor and the displaced owner. This persistent feature evidence is
    self-corroborating, but it remains subject to whole-group reconciliation
    and cannot split or partially overlap an established source.
    At the last retained scale, a cycle may construct a parent when its
    constituent features have exact or mutually unique displaced children. One
    directly owned feature may lack a child without vetoing an otherwise
    persistent whole-source graph, but only when every feature shares one
    retained support component and no direct member has a competing cycle.
    A persistent feature without a direct owner may corroborate cycle geometry
    but cannot join catalogue membership, which still requires at least three
    immutable direct components. Displaced and missing-child evidence cannot
    create cycles or membership. Partial overlap with an exact group, missing
    support, branching, or conflicting convergence fails closed.
    """
    labels, planes, valid = _hierarchy_inputs(
        records,
        component_labels,
        scale_detection_planes,
        valid_pixels,
    )
    significant = _validated_significant_support(
        significant_multiscale_support,
        labels,
        valid,
    )
    return associate_from_hierarchy_overlaps(
        records,
        planes,
        summarize_hierarchy_overlaps(
            records, labels, planes, valid, significant
        ),
    )

associate_from_hierarchy_overlaps(records, planes, overlaps)

Decide source membership from reduced pixel facts alone.

Every pixel question has already been answered by the tile cores and the per-feature windows that produced overlaps, so this step is record logic and cannot depend on tile geometry or completion order.

Source code in src/hebog/algorithms/source_association.py

def associate_from_hierarchy_overlaps(
    records: tuple[DetectionComponentRecord, ...],
    planes: tuple[ScaleDetections, ...],
    overlaps: HierarchyOverlaps,
) -> SourceAssociationResult:
    """Decide source membership from reduced pixel facts alone.

    Every pixel question has already been answered by the tile cores and the
    per-feature windows that produced ``overlaps``, so this step is record
    logic and cannot depend on tile geometry or completion order.
    """
    feature_index = _feature_by_id(planes)
    parent_edges = overlaps.parent_edges
    parent_sets: dict[str, set[str]] = {}
    for child_id, parent_id in parent_edges:
        child_scale = feature_index[child_id][0]
        parent_scale = feature_index[parent_id][0]
        if parent_scale != child_scale + 1:
            raise ValueError("source hierarchy parent scales must be adjacent")
        parent_sets.setdefault(child_id, set()).add(parent_id)
    parents_by_id = {
        child_id: tuple(sorted(parent_ids))
        for child_id, parent_ids in parent_sets.items()
    }

    attachments = _hierarchy_attachments(
        records,
        overlaps.finest_features_by_component,
        parents_by_id,
        feature_index,
    )
    scale_aware_parents = _scale_aware_parent_evidence(
        _ScaleAwareInputs(
            records=records,
            planes=planes,
            overlaps=overlaps,
            attachments=attachments,
            parents_by_id=parents_by_id,
            feature_index=feature_index,
        )
    )
    connected_support = _connected_support_evidence(
        overlaps.support_component_by_component,
        attachments.ambiguous_component_ids,
    )
    terminal_cycles = _terminal_cycle_evidence(
        planes,
        overlaps,
        attachments,
        parent_edges,
    )
    scale_aware_parents = _filter_scale_aware_parents_by_connected_support(
        scale_aware_parents,
        connected_support,
    )
    exact_groups = _hierarchy_groups(
        records,
        attachments,
        feature_index,
        parents_by_id,
    )
    groups, applied_scale_aware_parents = _apply_scale_aware_parent_groups(
        exact_groups,
        scale_aware_parents,
    )
    groups, applied_terminal_cycles = _apply_terminal_cycle_groups(
        groups,
        terminal_cycles,
    )
    memberships = _source_memberships(groups)
    return SourceAssociationResult(
        components=tuple(sorted(records, key=lambda item: item.component_id)),
        edges=(),
        memberships=memberships,
        ambiguous_component_ids=tuple(
            sorted(attachments.ambiguous_component_ids)
        ),
        hierarchy_diagnostics=_hierarchy_diagnostics(
            memberships,
            planes,
            parent_edges,
            attachments,
            applied_scale_aware_parents,
            connected_support,
            applied_terminal_cycles,
        ),
    )

constrain_source_memberships(association, measured_groups)

Apply compact-model or resolved-morphology evidence before publication.

Unconstrained components remain separate. A hierarchy is a proposal, not positive evidence that a failed or deferred fit belongs to its neighbours. Explicit compact or resolved-morphology groups retain their membership.

Source code in src/hebog/algorithms/source_association.py

def constrain_source_memberships(
    association: SourceAssociationResult,
    measured_groups: tuple[frozenset[int], ...],
) -> SourceAssociationResult:
    """Apply compact-model or resolved-morphology evidence before publication.

    Unconstrained components remain separate. A hierarchy is a proposal, not
    positive evidence that a failed or deferred fit belongs to its neighbours.
    Explicit compact or resolved-morphology groups retain their membership.
    """
    by_label = {
        item.label_value: item.component_id for item in association.components
    }
    labels = [value for group in measured_groups for value in group]
    if any(not group for group in measured_groups) or len(labels) != len(
        set(labels)
    ):
        raise ValueError("compact model groups must be nonempty and disjoint")
    if not set(labels) <= by_label.keys():
        raise ValueError("compact model group has an unknown component label")
    constrained = {
        by_label[value] for group in measured_groups for value in group
    }
    groups = tuple(
        frozenset((item.component_id,))
        for item in association.components
        if item.component_id not in constrained
    ) + tuple(
        frozenset(by_label[value] for value in group)
        for group in measured_groups
    )
    memberships = _source_memberships(groups)
    diagnostics = association.hierarchy_diagnostics
    if diagnostics is not None:
        diagnostics = replace(
            diagnostics,
            catalogue_source_count=len(memberships),
            membership_size_histogram=tuple(
                sorted(
                    Counter(
                        len(item.component_ids) for item in memberships
                    ).items()
                )
            ),
            unique_convergence_count=sum(
                len(item.component_ids) > 1 for item in memberships
            ),
        )
    return replace(
        association,
        memberships=memberships,
        ambiguous_component_ids=tuple(
            value
            for value in association.ambiguous_component_ids
            if value not in constrained
        ),
        hierarchy_diagnostics=diagnostics,
    )

hebog.stages.association

Scheduler-facing rounds that answer the source hierarchy's pixel questions.

ADR-008's association round produces :class:~hebog.algorithms.source_association.HierarchyOverlaps and nothing else: the decision that consumes it is pure record logic, so it cannot depend on tile geometry or completion order. Three rounds produce it. The cores observe which components, features and retained support components meet; one task per feature derives that feature's B3 influence inside its own window; and one task per candidate pair decides whether two envelopes overlap inside the box that holds them both.

HierarchyOverlapStageConfig dataclass

The bounded task limits of the three association rounds.

Source code in src/hebog/stages/association.py

@dataclass(frozen=True, slots=True)
class HierarchyOverlapStageConfig:
    """The bounded task limits of the three association rounds."""

    maximum_tiles_per_batch: int
    maximum_features_per_batch: int
    maximum_pairs_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before any round is submitted."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_features_per_batch", self.maximum_features_per_batch),
            ("maximum_pairs_per_batch", self.maximum_pairs_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")

__post_init__()

Reject an unbounded task before any round is submitted.

Source code in src/hebog/stages/association.py

def __post_init__(self) -> None:
    """Reject an unbounded task before any round is submitted."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_features_per_batch", self.maximum_features_per_batch),
        ("maximum_pairs_per_batch", self.maximum_pairs_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")

HierarchyOverlapStageResult dataclass

The reduced pixel facts and this stage's scalar execution evidence.

Source code in src/hebog/stages/association.py

@dataclass(frozen=True, slots=True)
class HierarchyOverlapStageResult:
    """The reduced pixel facts and this stage's scalar execution evidence."""

    generation: ProductGenerationManifest
    overlaps: HierarchyOverlaps
    partition_count: int
    feature_count: int
    enveloped_feature_count: int
    candidate_pair_count: int
    executor_task_count: int
    maximum_graph_width: int
    reconciliation_round_count: int
    maximum_feature_read_pixels: int

hierarchy_overlap_product_names()

Return the canonical published hierarchy-overlap product set.

Source code in src/hebog/stages/association.py

def hierarchy_overlap_product_names() -> tuple[str, ...]:
    """Return the canonical published hierarchy-overlap product set."""
    return _PRODUCT_NAMES

run_hierarchy_overlap_stage(detection_source, component_source, manifest, *, config, records, scale_detections, executor, sink)

Answer every pixel question the source hierarchy decision asks.

Three rounds and no write. The cores observe which components, features and retained support components meet, and the support components are reconciled across core boundaries; one task per feature derives that feature's reviewed B3 influence inside its own window; and one task per candidate pair decides whether two envelopes overlap inside the box that holds them both. Candidate pairs follow from the reconciled bounds, so only the pairs that can possibly meet read pixels.

A feature whose influence window, or a pair whose box, exceeds the read budget is never read whole. Both are dilations by the reviewed B3 radius, so each core that the work can reach decides its own pixels under that halo, and the parts join exactly.

Source code in src/hebog/stages/association.py

def run_hierarchy_overlap_stage(  # noqa: PLR0913
    detection_source: CompletedProductSource,
    component_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: HierarchyOverlapStageConfig,
    records: tuple[DetectionComponentRecord, ...],
    scale_detections: Sequence[ScaleDetections],
    executor: Executor,
    sink: ZarrProductSink,
) -> HierarchyOverlapStageResult:
    """Answer every pixel question the source hierarchy decision asks.

    Three rounds and no write. The cores observe which components, features
    and retained support components meet, and the support components are
    reconciled across core boundaries; one task per feature derives that
    feature's reviewed B3 influence inside its own window; and one task per
    candidate pair decides whether two envelopes overlap inside the box that
    holds them both. Candidate pairs follow from the reconciled bounds, so
    only the pairs that can possibly meet read pixels.

    A feature whose influence window, or a pair whose box, exceeds the read
    budget is never read whole. Both are dilations by the reviewed B3
    radius, so each core that the work can reach decides its own pixels
    under that halo, and the parts join exactly.
    """
    _require_overlap_inputs(detection_source, component_source, manifest)
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_core_overlaps,
                detection_source=detection_source,
                component_source=component_source,
                image_shape_yx=manifest.image_shape_yx,
            ),
            _core_batches(
                manifest.tiles,
                maximum_tiles_per_batch=config.maximum_tiles_per_batch,
            ),
            requirement=read_requirement(
                _OVERLAP_SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no overlap scan results")
    cores = tuple(core for result in scan_results for core in result.cores)
    support = reconcile_candidate_tiles(
        manifest,
        tuple(core.summary for core in cores),
    )
    features = _features(
        scale_detections,
        image_shape_yx=manifest.image_shape_yx,
    )
    merged = _merge_cores(cores, support, features, records)
    budget = config.maximum_batch_read_pixels
    candidates = influence_candidate_feature_ids(
        merged.exact_component_ids, merged.parent_edges
    )
    influenced = tuple(
        feature
        for feature in features
        if feature.envelope_bounds is not None
        and feature.feature_id in candidates
    )
    influence_window = partial(
        _influence_window, image_shape_yx=manifest.image_shape_yx
    )
    component_id_by_label = {
        record.label_value: record.component_id for record in records
    }
    influence_batches = _influence_batches(
        tuple(
            feature
            for feature in influenced
            if read_pixels(influence_window(feature)) <= budget
        ),
        maximum_features_per_batch=config.maximum_features_per_batch,
        maximum_batch_read_pixels=budget,
        image_shape_yx=manifest.image_shape_yx,
        core_shape_yx=manifest.tile_core_shape_yx,
    )
    influence_results = map_round(
        executor,
        partial(
            _influence_batch,
            detection_source=detection_source,
            component_source=component_source,
            image_shape_yx=manifest.image_shape_yx,
        ),
        influence_batches,
        round_name="feature influence",
        requirement=read_requirement(
            _INFLUENCE_BYTES_PER_READ_PIXEL,
            (batch.read_bounds for batch in influence_batches),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    pairs = _candidate_pairs(
        features,
        terminal_scale_order=_SCALE_ORDERS[-1],
    )
    pair_batches = _pair_batches(
        tuple(
            pair
            for pair in pairs
            if read_pixels(_pair_read_bounds_of(pair)) <= budget
        ),
        maximum_pairs_per_batch=config.maximum_pairs_per_batch,
        maximum_batch_read_pixels=budget,
        core_shape_yx=manifest.tile_core_shape_yx,
    )
    pair_results = map_round(
        executor,
        partial(_pair_batch, detection_source=detection_source),
        pair_batches,
        round_name="envelope pair",
        requirement=read_requirement(
            _PAIR_BYTES_PER_READ_PIXEL,
            (batch.read_bounds for batch in pair_batches),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    wide_batches = _wide_batches(
        tuple(
            feature
            for feature in influenced
            if read_pixels(influence_window(feature)) > budget
        ),
        tuple(
            pair
            for pair in pairs
            if read_pixels(_pair_read_bounds_of(pair)) > budget
        ),
        holders=_feature_holders(cores),
        manifest=manifest,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    wide_results = map_round(
        executor,
        partial(
            _wide_overlap_batch,
            detection_source=detection_source,
            component_source=component_source,
            image_shape_yx=manifest.image_shape_yx,
        ),
        wide_batches,
        round_name="wide overlap",
        requirement=read_requirement(
            _WIDE_OVERLAP_BYTES_PER_READ_PIXEL,
            (
                core.partition.read_bounds
                for batch in wide_batches
                for core in batch.cores
            ),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    wide_influence, wide_edges = _reduce_wide_results(
        wide_batches, wide_results
    )
    influence = {
        feature_id: tuple(
            sorted(component_id_by_label[label] for label in component_labels)
        )
        for feature_id, component_labels in (
            *(
                item
                for result in influence_results
                for item in result.influence
            ),
            *wide_influence.items(),
        )
    }
    edges = wide_edges | frozenset(
        frozenset(edge) for result in pair_results for edge in result.edges
    )
    return HierarchyOverlapStageResult(
        generation=_publish_persistent_scale_support(
            manifest,
            scale_detections,
            merged.parent_edges,
            maximum_tiles_per_batch=config.maximum_tiles_per_batch,
            detection_source=detection_source,
            executor=executor,
            sink=sink,
        ),
        overlaps=_reduce_overlaps(merged, features, influence, edges),
        partition_count=len(manifest.tiles),
        feature_count=len(features),
        enveloped_feature_count=sum(
            1 for feature in features if feature.envelope_bounds is not None
        ),
        candidate_pair_count=len(pairs),
        executor_task_count=(
            2 * len(manifest.tiles)
            + len(influence_batches)
            + len(pair_batches)
            + len(wide_batches)
        ),
        maximum_graph_width=max(
            len(manifest.tiles),
            len(influence_batches),
            len(pair_batches),
            len(wide_batches),
        ),
        reconciliation_round_count=support.reduction_round_count,
        maximum_feature_read_pixels=max(
            (
                *(result.maximum_read_pixels for result in influence_results),
                *(result.maximum_read_pixels for result in pair_results),
                *(result.maximum_read_pixels for result in wide_results),
            ),
            default=0,
        ),
    )

hebog.stages.sources

Scheduler-facing rounds that own the continuum catalogue's source planes.

ADR-008's catalogue rounds decide one object at a time. The owners a source holds are a record map, so the cores write the source labels from the shard that reaches them; the persistent support a source owns spans tiles, so it is reconciled first and then assigned inside each connected component's own window, by the cores that hold the component, so no assignment reaches the driver.

SourceLabelStageConfig dataclass

The bounded task limit of the source-label rounds.

Source code in src/hebog/stages/sources.py

@dataclass(frozen=True, slots=True)
class SourceLabelStageConfig:
    """The bounded task limit of the source-label rounds."""

    maximum_tiles_per_batch: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before any product is initialized."""
        if (
            isinstance(self.maximum_tiles_per_batch, bool)
            or not isinstance(self.maximum_tiles_per_batch, Integral)
            or self.maximum_tiles_per_batch < 1
        ):
            raise ValueError(
                "maximum_tiles_per_batch must be a positive integer"
            )

__post_init__()

Reject an unbounded task before any product is initialized.

Source code in src/hebog/stages/sources.py

def __post_init__(self) -> None:
    """Reject an unbounded task before any product is initialized."""
    if (
        isinstance(self.maximum_tiles_per_batch, bool)
        or not isinstance(self.maximum_tiles_per_batch, Integral)
        or self.maximum_tiles_per_batch < 1
    ):
        raise ValueError(
            "maximum_tiles_per_batch must be a positive integer"
        )

SourceLabelStageResult dataclass

Published source labels and scalar execution evidence.

Source code in src/hebog/stages/sources.py

@dataclass(frozen=True, slots=True)
class SourceLabelStageResult:
    """Published source labels and scalar execution evidence."""

    generation: ProductGenerationManifest
    source_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int

SourceSupportStageConfig dataclass

Reviewed support policy and the bounded task limits.

Source code in src/hebog/stages/sources.py

@dataclass(frozen=True, slots=True)
class SourceSupportStageConfig:
    """Reviewed support policy and the bounded task limits."""

    maximum_tiles_per_batch: int
    maximum_objects_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before any product is initialized."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_objects_per_batch", self.maximum_objects_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")

__post_init__()

Reject an unbounded task before any product is initialized.

Source code in src/hebog/stages/sources.py

def __post_init__(self) -> None:
    """Reject an unbounded task before any product is initialized."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_objects_per_batch", self.maximum_objects_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")

SourceSupportStageResult dataclass

Published source measurement labels and execution evidence.

Source code in src/hebog/stages/sources.py

@dataclass(frozen=True, slots=True)
class SourceSupportStageResult:
    """Published source measurement labels and execution evidence."""

    generation: ProductGenerationManifest
    support_component_count: int
    wide_component_count: int
    assigned_component_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_component_read_pixels: int
    reconciliation_round_count: int

run_source_label_stage(component_source, manifest, *, config, source_by_owner, executor, sink)

Write the catalogue source each component owner belongs to.

Two rounds: the cores observe which owners they hold, and each then writes the labels of the shard that reaches it. The membership map is a record, so no accepted-label table is broadcast whole.

Source code in src/hebog/stages/sources.py

def run_source_label_stage(  # noqa: PLR0913
    component_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: SourceLabelStageConfig,
    source_by_owner: Mapping[int, int],
    executor: Executor,
    sink: ZarrProductSink,
) -> SourceLabelStageResult:
    """Write the catalogue source each component owner belongs to.

    Two rounds: the cores observe which owners they hold, and each then
    writes the labels of the shard that reaches it. The membership map is a
    record, so no accepted-label table is broadcast whole.
    """
    if sink.manifest != manifest:
        raise ValueError("source label sink must use the stage manifest")
    if manifest.halo_yx != (0, 0):
        raise ValueError("source labels write cores without a halo")
    if component_source.manifest.image_shape_yx != manifest.image_shape_yx:
        raise ValueError(
            "published generations must match the source image shape"
        )
    if "component-measurement-labels" not in (
        component_source.read_generation().product_names
    ):
        raise ValueError(
            "published generations must carry the component owner labels"
        )
    scan_batches = tuple(
        _OwnerScanBatch(partitions=group)
        for group in _core_batches(
            manifest.tiles,
            maximum_tiles_per_batch=config.maximum_tiles_per_batch,
        )
    )
    scan_results = tuple(
        executor.map_batches(
            partial(_scan_owners, component_source=component_source),
            scan_batches,
            requirement=read_requirement(
                _OWNER_SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no owner scan results")
    owners_by_tile = {
        tile_id: owners
        for result in scan_results
        for tile_id, owners in result.owners_by_tile
    }
    sink.initialize_product(
        product_name="source-labels",
        dtype=np.dtype("<i4"),
    )
    write_batches = tuple(
        _LabelWriteBatch(requests=group)
        for group in _core_batches(
            tuple(
                _LabelWriteRequest(
                    partition=partition,
                    source_by_owner=tuple(
                        (owner, source_by_owner[owner])
                        for owner in owners_by_tile[partition.tile_id]
                        if owner in source_by_owner
                    ),
                )
                for partition in manifest.tiles
            ),
            maximum_tiles_per_batch=config.maximum_tiles_per_batch,
        )
    )
    write_results = tuple(
        executor.map_batches(
            partial(
                _publish_source_labels,
                component_source=component_source,
                sink=sink,
            ),
            write_batches,
            requirement=read_requirement(
                _LABEL_WRITE_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not write_results:
        raise ValueError("executor returned no source label results")
    return SourceLabelStageResult(
        generation=sink.publish_generation(
            product_names=_LABEL_PRODUCT_NAMES,
            chunks=(
                chunk
                for result in write_results
                for chunk in result.product_chunks
            ),
        ),
        source_count=len(set(source_by_owner.values())),
        partition_count=len(manifest.tiles),
        executor_task_count=len(scan_batches) + len(write_batches),
        maximum_graph_width=max(len(scan_batches), len(write_batches)),
    )

run_source_support_stage(label_source, detection_source, scale_support_source, measurement_support_source, manifest, *, config, executor, sink)

Assign the persistent support each catalogue source owns.

Two rounds: the cores label the support they carry and the pieces that meet across a core boundary are reconciled, then each core assigns the connected components it holds, every unseeded pixel to its nearest source seed inside that component's own window, and writes its share over the seeds it already holds. A component spanning several cores is assigned by each of them from the same window, so they agree, and the driver holds each component's bounds rather than its assignment.

A component whose window exceeds maximum_batch_read_pixels is never read whole. The cores holding it return its seeds in a round between those two, and each core that holds its unseeded pixels gets back the seeds that can own them and assigns them itself, which is the assignment its window would make. The driver holds the seeds, never the unseeded support.

Source code in src/hebog/stages/sources.py

def run_source_support_stage(  # noqa: PLR0913
    label_source: CompletedProductSource,
    detection_source: CompletedProductSource,
    scale_support_source: CompletedProductSource,
    measurement_support_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: SourceSupportStageConfig,
    executor: Executor,
    sink: ZarrProductSink,
) -> SourceSupportStageResult:
    """Assign the persistent support each catalogue source owns.

    Two rounds: the cores label the support they carry and the pieces that
    meet across a core boundary are reconciled, then each core assigns the
    connected components it holds, every unseeded pixel to its nearest
    source seed inside that component's own window, and writes its share
    over the seeds it already holds. A component spanning several cores is
    assigned by each of them from the same window, so they agree, and the
    driver holds each component's bounds rather than its assignment.

    A component whose window exceeds ``maximum_batch_read_pixels`` is never
    read whole. The cores holding it return its seeds in a round between
    those two, and each core that holds its unseeded pixels gets back the
    seeds that can own them and assigns them itself, which is the
    assignment its window would make. The driver holds the seeds, never the
    unseeded support.
    """
    if sink.manifest != manifest:
        raise ValueError("source support sink must use the stage manifest")
    if manifest.halo_yx != (0, 0):
        raise ValueError("source support writes cores without a halo")
    for product_source, names in (
        (label_source, ("source-labels",)),
        (detection_source, ("valid-pixels",)),
        (scale_support_source, ("persistent-scale-support",)),
        (measurement_support_source, ("measurement-support",)),
    ):
        if product_source.manifest.image_shape_yx != manifest.image_shape_yx:
            raise ValueError(
                "published generations must match the support image shape"
            )
        if not set(names).issubset(
            product_source.read_generation().product_names
        ):
            raise ValueError(
                "published generations must carry every support plane read"
            )
    scan_batches = tuple(
        _SupportScanBatch(partitions=group)
        for group in _core_batches(
            manifest.tiles,
            maximum_tiles_per_batch=config.maximum_tiles_per_batch,
        )
    )
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_support,
                label_source=label_source,
                detection_source=detection_source,
                scale_support_source=scale_support_source,
                measurement_support_source=measurement_support_source,
                image_shape_yx=manifest.image_shape_yx,
            ),
            scan_batches,
            requirement=read_requirement(
                _SUPPORT_SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no support scan results")
    reconciled = reconcile_candidate_tiles(
        manifest,
        tuple(
            summary for result in scan_results for summary in result.summaries
        ),
    )
    budget = config.maximum_batch_read_pixels
    image_width = manifest.image_shape_yx[1]
    narrow = {
        island.global_label: _SupportComponent(
            bounds=island.bounds, first_pixel_yx=island.first_pixel_yx
        )
        for island in reconciled.islands
        if read_pixels(island.bounds) <= budget
    }
    held_narrow = {
        core.partition.tile_id: tuple(
            narrow[label] for label in sorted(set(core.mapping.global_labels))
        )
        for core in cores_holding(
            frozenset(narrow), manifest, reconciled.tile_mappings
        )
    }
    wide_cores = cores_holding(
        frozenset(
            island.global_label
            for island in reconciled.islands
            if island.global_label not in narrow
        ),
        manifest,
        reconciled.tile_mappings,
    )
    wide_batches = tuple(
        _WideSupportBatch(
            cores=wide_cores[start : start + config.maximum_tiles_per_batch]
        )
        for start in range(0, len(wide_cores), config.maximum_tiles_per_batch)
    )
    wide_results = map_round(
        executor,
        partial(
            _gather_wide_support,
            label_source=label_source,
            detection_source=detection_source,
            scale_support_source=scale_support_source,
            measurement_support_source=measurement_support_source,
            image_shape_yx=manifest.image_shape_yx,
        ),
        wide_batches,
        round_name="wide support",
        requirement=read_requirement(
            _WIDE_SUPPORT_BYTES_PER_READ_PIXEL,
            (core.partition.read_bounds for core in wide_cores),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    wide_seeds = _wide_seed_shards(
        wide_batches, wide_results, image_width=image_width
    )
    wide_mappings = {
        core.partition.tile_id: core.mapping for core in wide_cores
    }
    sink.initialize_product(
        product_name="source-measurement-labels",
        dtype=np.dtype("<i4"),
    )
    write_batches = tuple(
        _SupportWriteBatch(requests=group)
        for group in _core_batches(
            tuple(
                _SupportWriteRequest(
                    partition=partition,
                    components=held_narrow.get(partition.tile_id, ()),
                    wide=(
                        _WideShare(
                            mapping=wide_mappings[partition.tile_id],
                            seeds=wide_seeds.by_tile[partition.tile_id],
                        )
                        if partition.tile_id in wide_seeds.by_tile
                        else None
                    ),
                )
                for partition in manifest.tiles
            ),
            maximum_tiles_per_batch=config.maximum_tiles_per_batch,
        )
    )
    write_results = tuple(
        executor.map_batches(
            partial(
                _publish_source_support,
                label_source=label_source,
                detection_source=detection_source,
                scale_support_source=scale_support_source,
                measurement_support_source=measurement_support_source,
                sink=sink,
                config=config,
                image_shape_yx=manifest.image_shape_yx,
                core_shape_yx=manifest.tile_core_shape_yx,
            ),
            write_batches,
            requirement=read_requirement(
                _SUPPORT_WRITE_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not write_results:
        raise ValueError("executor returned no source support results")
    return SourceSupportStageResult(
        generation=sink.publish_generation(
            product_names=_SUPPORT_PRODUCT_NAMES,
            chunks=(
                chunk
                for result in write_results
                for chunk in result.product_chunks
            ),
        ),
        support_component_count=len(reconciled.islands),
        wide_component_count=sum(
            island.global_label not in narrow for island in reconciled.islands
        ),
        assigned_component_count=len(
            {
                first_pixel
                for result in write_results
                for first_pixel in result.assigned_components
            }
        )
        + wide_seeds.assigning_component_count,
        partition_count=len(manifest.tiles),
        executor_task_count=(
            len(scan_batches) + len(wide_batches) + len(write_batches)
        ),
        maximum_graph_width=max(
            len(scan_batches), len(wide_batches), len(write_batches)
        ),
        maximum_component_read_pixels=max(
            (
                *(
                    result.maximum_component_read_pixels
                    for result in write_results
                ),
                *(result.maximum_core_read_pixels for result in wide_results),
            ),
            default=0,
        ),
        reconciliation_round_count=reconciled.reduction_round_count,
    )

source_label_product_names()

Return the canonical published source-label product set.

Source code in src/hebog/stages/sources.py

def source_label_product_names() -> tuple[str, ...]:
    """Return the canonical published source-label product set."""
    return _LABEL_PRODUCT_NAMES

source_support_product_names()

Return the canonical published source-support product set.

Source code in src/hebog/stages/sources.py

def source_support_product_names() -> tuple[str, ...]:
    """Return the canonical published source-support product set."""
    return _SUPPORT_PRODUCT_NAMES

hebog.science.detection_islands

Detection islands of the installed composition and their ownership.

DetectionIslandCatalogue dataclass

Every measured island and the islands each owner's support reaches.

Source code in src/hebog/science/detection_islands.py

@dataclass(frozen=True, slots=True)
class DetectionIslandCatalogue:
    """Every measured island and the islands each owner's support reaches."""

    islands: tuple[CatalogueIsland, ...]
    island_ids_by_owner: Mapping[int, tuple[str, ...]]

build_detection_island_catalogue(residual_jy_per_beam, rms_jy_per_beam, retained_mask, owner_labels, parent_labels, *, beam_area_pixels)

Measure every island of one complete retained mask, in raster order.

This is the readable whole-plane reference for the tiled rounds in :mod:hebog.stages.islands, which label each core and reconcile the fragments instead of labelling the plane. Rows are ordered by the island's canonical first pixel, which is the order labelling a whole plane gives.

owner_labels are the measurement labels of the components the catalogue publishes; an owner is linked to every island its retained support reaches, because published support can be split between islands. parent_labels are the measurement labels of the owners those components were deblended from: a component whose own support reaches no island is linked to the islands its parent's support reaches.

Source code in src/hebog/science/detection_islands.py

def build_detection_island_catalogue(  # noqa: PLR0913
    residual_jy_per_beam: npt.NDArray[np.float64],
    rms_jy_per_beam: npt.NDArray[np.float64],
    retained_mask: npt.NDArray[np.bool_],
    owner_labels: npt.NDArray[np.integer[Any]],
    parent_labels: npt.NDArray[np.integer[Any]],
    *,
    beam_area_pixels: float,
) -> DetectionIslandCatalogue:
    """Measure every island of one complete retained mask, in raster order.

    This is the readable whole-plane reference for the tiled rounds in
    :mod:`hebog.stages.islands`, which label each core and reconcile the
    fragments instead of labelling the plane. Rows are ordered by the island's
    canonical first pixel, which is the order labelling a whole plane gives.

    ``owner_labels`` are the measurement labels of the components the
    catalogue publishes; an owner is linked to every island its retained
    support reaches, because published support can be split between islands.
    ``parent_labels`` are the measurement labels of the owners those
    components were deblended from: a component whose own support reaches
    no island is linked to the islands its parent's support reaches.
    """
    labels = _connected_island_labels(retained_mask)
    windows = label_windows(labels)
    islands: list[CatalogueIsland] = []
    identifier_by_label: dict[int, str] = {}
    for label_value, crop in enumerate(windows, start=1):
        if crop is None:
            raise ValueError("detection islands must own at least one pixel")
        support = labels[crop] == label_value
        first = np.argwhere(support)[0]
        first_pixel_yx = (
            int(first[0]) + crop[0].start,
            int(first[1]) + crop[1].start,
        )
        island = measure_detection_island(
            residual_jy_per_beam[crop],
            rms_jy_per_beam[crop],
            support,
            first_pixel_yx=first_pixel_yx,
            beam_area_pixels=beam_area_pixels,
        )
        identifier_by_label[label_value] = island.identifier
        islands.append(island)
    return DetectionIslandCatalogue(
        islands=tuple(islands),
        island_ids_by_owner=island_ids_by_component(
            _owner_island_pairs(labels, owner_labels, retained_mask),
            _owner_island_pairs(labels, parent_labels, retained_mask),
            observed_label_pairs(owner_labels, parent_labels),
            identifier_by_island_label=identifier_by_label,
        ),
    )

detection_island_identifier(first_pixel_yx)

Name one detection island by the first pixel it owns.

Raster order makes the first pixel canonical: it does not depend on how the mask was partitioned or in which order the tiles completed.

Examples:

>>> detection_island_identifier((12, 40))
'island-detection-12-40'

Source code in src/hebog/science/detection_islands.py

def detection_island_identifier(first_pixel_yx: tuple[int, int]) -> str:
    """Name one detection island by the first pixel it owns.

    Raster order makes the first pixel canonical: it does not depend on how
    the mask was partitioned or in which order the tiles completed.

    Examples:
        >>> detection_island_identifier((12, 40))
        'island-detection-12-40'
    """
    return f"island-detection-{first_pixel_yx[0]}-{first_pixel_yx[1]}"

island_ids_by_component(component_pairs, parent_pairs, component_parents, *, identifier_by_island_label)

Name the islands each component reaches, or else its parent's.

A component reaches the islands its own support overlaps. Support refinement can leave part of an owner outside the retained mask, such as a faint component deblended onto a brighter one's rim; that component then belongs to the islands its parent owner reaches, as a Gaussian belongs to the island it was fitted in, rather than to none. The pairs come either from one whole plane or from the cores that observed them.

Examples:

>>> island_ids_by_component(
...     ((1, 10),),
...     ((7, 10),),
...     ((1, 7), (2, 7)),
...     identifier_by_island_label={10: "island-a"},
... )
{1: ('island-a',), 2: ('island-a',)}

Source code in src/hebog/science/detection_islands.py

def island_ids_by_component(
    component_pairs: tuple[tuple[int, int], ...],
    parent_pairs: tuple[tuple[int, int], ...],
    component_parents: tuple[tuple[int, int], ...],
    *,
    identifier_by_island_label: Mapping[int, str],
) -> dict[int, tuple[str, ...]]:
    """Name the islands each component reaches, or else its parent's.

    A component reaches the islands its own support overlaps. Support
    refinement can leave part of an owner outside the retained mask, such as
    a faint component deblended onto a brighter one's rim; that component
    then belongs to the islands its parent owner reaches, as a Gaussian
    belongs to the island it was fitted in, rather than to none. The pairs
    come either from one whole plane or from the cores that observed them.

    Examples:
        >>> island_ids_by_component(
        ...     ((1, 10),),
        ...     ((7, 10),),
        ...     ((1, 7), (2, 7)),
        ...     identifier_by_island_label={10: "island-a"},
        ... )
        {1: ('island-a',), 2: ('island-a',)}
    """
    own = island_ids_by_owner(
        component_pairs,
        identifier_by_island_label=identifier_by_island_label,
    )
    parents = island_ids_by_owner(
        parent_pairs,
        identifier_by_island_label=identifier_by_island_label,
    )
    inherited: dict[int, set[str]] = {}
    for component, parent in component_parents:
        if component not in own and parent in parents:
            inherited.setdefault(component, set()).update(parents[parent])
    return dict(
        sorted(
            (
                *own.items(),
                *(
                    (component, tuple(sorted(identifiers)))
                    for component, identifiers in inherited.items()
                ),
            )
        )
    )

island_ids_by_owner(pairs, *, identifier_by_island_label)

Name the islands every owner reaches, from observed label pairs.

The pairs come either from one whole plane or from the cores that observed them, so this join is the same in both compositions.

Source code in src/hebog/science/detection_islands.py

def island_ids_by_owner(
    pairs: tuple[tuple[int, int], ...],
    *,
    identifier_by_island_label: Mapping[int, str],
) -> dict[int, tuple[str, ...]]:
    """Name the islands every owner reaches, from observed label pairs.

    The pairs come either from one whole plane or from the cores that
    observed them, so this join is the same in both compositions.
    """
    owners: dict[int, set[str]] = {}
    for owner, island_label in pairs:
        owners.setdefault(owner, set()).add(
            identifier_by_island_label[island_label]
        )
    return {
        owner: tuple(sorted(identifiers))
        for owner, identifiers in sorted(owners.items())
    }

measure_detection_island(residual_jy_per_beam, rms_jy_per_beam, support, *, first_pixel_yx, beam_area_pixels)

Summarise one island over the exact pixels its mask retains.

Every array covers the window that holds the island, and support is that island alone: an island is mask connectivity, not owner support, so nothing here consults a label plane. The flux is the signed residual over those pixels, which is why an island row is reported without an uncertainty.

Raises:

Type	Description
ValueError	If the island retains no pixel in this window.

Source code in src/hebog/science/detection_islands.py

def measure_detection_island(
    residual_jy_per_beam: npt.NDArray[np.float64],
    rms_jy_per_beam: npt.NDArray[np.float64],
    support: npt.NDArray[np.bool_],
    *,
    first_pixel_yx: tuple[int, int],
    beam_area_pixels: float,
) -> CatalogueIsland:
    """Summarise one island over the exact pixels its mask retains.

    Every array covers the window that holds the island, and ``support`` is
    that island alone: an island is mask connectivity, not owner support, so
    nothing here consults a label plane. The flux is the signed residual over
    those pixels, which is why an island row is reported without an
    uncertainty.

    Raises:
        ValueError: If the island retains no pixel in this window.
    """
    if not bool(np.any(support)):
        raise ValueError("detection island must retain at least one pixel")
    residual = residual_jy_per_beam[support]
    return CatalogueIsland(
        identifier=detection_island_identifier(first_pixel_yx),
        pixel_count=int(support.sum()),
        integrated_flux_jy=float(
            np.sum(residual, dtype=np.float64) / beam_area_pixels
        ),
        local_rms_jy_per_beam=float(np.median(rms_jy_per_beam[support])),
        mean_brightness_jy_per_beam=float(np.mean(residual)),
    )

observed_label_pairs(first_labels, second_labels)

Pair every label of one plane with each label of another it meets.

This pairs an owner with the islands its support reaches, or a component with the parent owner whose support holds it, over a whole plane or one core; the island join is the same either way.

Examples:

>>> import numpy as np
>>> observed_label_pairs(
...     np.array([[3, 3, 0], [0, 4, 4]]),
...     np.array([[1, 2, 2], [0, 2, 0]]),
... )
((3, 1), (3, 2), (4, 2))

Source code in src/hebog/science/detection_islands.py

def observed_label_pairs(
    first_labels: npt.NDArray[np.integer[Any]],
    second_labels: npt.NDArray[np.integer[Any]],
) -> tuple[tuple[int, int], ...]:
    """Pair every label of one plane with each label of another it meets.

    This pairs an owner with the islands its support reaches, or a component
    with the parent owner whose support holds it, over a whole plane or one
    core; the island join is the same either way.

    Examples:
        >>> import numpy as np
        >>> observed_label_pairs(
        ...     np.array([[3, 3, 0], [0, 4, 4]]),
        ...     np.array([[1, 2, 2], [0, 2, 0]]),
        ... )
        ((3, 1), (3, 2), (4, 2))
    """
    selected = (first_labels > 0) & (second_labels > 0)
    if not bool(np.any(selected)):
        return ()
    pairs = np.unique(
        np.column_stack((first_labels[selected], second_labels[selected])),
        axis=0,
    )
    return tuple((int(first), int(second)) for first, second in pairs)

hebog.stages.islands

Scheduler-facing rounds that measure one catalogue row per island.

ADR-008's island round is the one object round whose objects are mask connectivity rather than owner support: an island is a connected region of the published retained mask, so it spans tiles exactly as a label plane does and is reconciled the same way. Its labels are never published, because only these rounds read them, and an island's own bounds hold it entirely, so the round that measures it recovers it by labelling its own window.

An island is not bounded in area, so its window can exceed the read budget: a filament may cross the whole image. Such an island is never read whole. The cores that hold it relabel themselves exactly as the scan did, the reconciled mapping names its pixels there, and the row is measured from those pixels alone.

DetectionIslandStageConfig dataclass

The reviewed beam area and the bounded task limits.

Source code in src/hebog/stages/islands.py

@dataclass(frozen=True, slots=True)
class DetectionIslandStageConfig:
    """The reviewed beam area and the bounded task limits."""

    beam_area_pixels: float
    maximum_tiles_per_batch: int
    maximum_objects_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before any round is submitted."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_objects_per_batch", self.maximum_objects_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")
        if (
            not np.isfinite(self.beam_area_pixels)
            or self.beam_area_pixels <= 0.0
        ):
            raise ValueError("beam_area_pixels must be finite and positive")

__post_init__()

Reject an unbounded task before any round is submitted.

Source code in src/hebog/stages/islands.py

def __post_init__(self) -> None:
    """Reject an unbounded task before any round is submitted."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_objects_per_batch", self.maximum_objects_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")
    if (
        not np.isfinite(self.beam_area_pixels)
        or self.beam_area_pixels <= 0.0
    ):
        raise ValueError("beam_area_pixels must be finite and positive")

DetectionIslandStageResult dataclass

Every measured island, its owners, and scalar evidence.

Source code in src/hebog/stages/islands.py

@dataclass(frozen=True, slots=True)
class DetectionIslandStageResult:
    """Every measured island, its owners, and scalar evidence."""

    islands: tuple[CatalogueIsland, ...]
    island_ids_by_owner: Mapping[int, tuple[str, ...]]
    island_count: int
    wide_island_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_island_read_pixels: int
    boundary_summary_array_bytes: int
    reconciliation_round_count: int

run_detection_island_stage(source, background_rms_source, publication_source, component_source, manifest, *, config, executor)

Measure every island of the retained mask, each in its own window.

Two rounds and one reconciliation: the cores label their own retained mask and observe which owners its islands hold, the fragments reconcile into islands numbered by canonical first pixel, and one task per batch of islands measures their rows. The round publishes no plane, because only these rounds read the island labels.

island_ids_by_owner names, for every component, the islands its own support reaches, or, when it reaches none, the islands of the parent owner it was deblended from.

An island whose own window exceeds maximum_batch_read_pixels takes a third round instead of a window: the cores holding it return its pixels, and its row is measured from them, so no read is wider than the budget or one core, whichever is larger.

Source code in src/hebog/stages/islands.py

def run_detection_island_stage(  # noqa: PLR0913
    source: WindowReadable,
    background_rms_source: CompletedProductSource,
    publication_source: CompletedProductSource,
    component_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: DetectionIslandStageConfig,
    executor: Executor,
) -> DetectionIslandStageResult:
    """Measure every island of the retained mask, each in its own window.

    Two rounds and one reconciliation: the cores label their own retained
    mask and observe which owners its islands hold, the fragments reconcile
    into islands numbered by canonical first pixel, and one task per batch of
    islands measures their rows. The round publishes no plane, because only
    these rounds read the island labels.

    ``island_ids_by_owner`` names, for every component, the islands its own
    support reaches, or, when it reaches none, the islands of the parent
    owner it was deblended from.

    An island whose own window exceeds ``maximum_batch_read_pixels`` takes a
    third round instead of a window: the cores holding it return its pixels,
    and its row is measured from them, so no read is wider than the budget or
    one core, whichever is larger.
    """
    _require_island_inputs(
        background_rms_source,
        publication_source,
        component_source,
        manifest,
    )
    core_batches = _core_batches(
        manifest.tiles,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    scan_results = tuple(
        executor.map_batches(
            partial(
                _scan_islands,
                publication_source=publication_source,
                component_source=component_source,
                image_shape_yx=manifest.image_shape_yx,
            ),
            core_batches,
            requirement=read_requirement(
                _SCAN_BYTES_PER_READ_PIXEL,
                (partition.read_bounds for partition in manifest.tiles),
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not scan_results:
        raise ValueError("executor returned no island topology results")
    tiles = tuple(tile for result in scan_results for tile in result.tiles)
    reconciled = reconcile_candidate_tiles(
        manifest,
        tuple(tile.summary for tile in tiles),
    )
    islands = _islands(reconciled.islands)
    owner_pairs = _global_owner_pairs(tiles, reconciled.tile_mappings)
    identifier_by_island_label = {
        island.global_label: detection_island_identifier(island.first_pixel_yx)
        for island in islands
    }
    budget = config.maximum_batch_read_pixels
    spanning = tuple(
        island for island in islands if read_pixels(island.bounds) > budget
    )
    row_batches = _island_batches(
        tuple(
            island
            for island in islands
            if read_pixels(island.bounds) <= budget
        ),
        core_shape_yx=manifest.tile_core_shape_yx,
        maximum_objects_per_batch=config.maximum_objects_per_batch,
        maximum_batch_read_pixels=budget,
    )
    row_results: tuple[_RowBatchResult, ...] = ()
    if row_batches:
        row_results = tuple(
            executor.map_batches(
                partial(
                    _measure_islands,
                    source=source,
                    background_rms_source=background_rms_source,
                    publication_source=publication_source,
                    config=config,
                ),
                row_batches,
                requirement=read_requirement(
                    _ROW_BYTES_PER_READ_PIXEL,
                    (batch.read_bounds for batch in row_batches),
                    chunk_shape_yx=manifest.tile_core_shape_yx,
                ),
            )
        )
        if not row_results:
            raise ValueError("executor returned no island row results")
    spanning_cores = cores_holding(
        frozenset(island.global_label for island in spanning),
        manifest,
        reconciled.tile_mappings,
    )
    pixel_batches = tuple(
        _SpanningBatch(
            cores=spanning_cores[
                start : start + config.maximum_tiles_per_batch
            ]
        )
        for start in range(
            0, len(spanning_cores), config.maximum_tiles_per_batch
        )
    )
    pixel_results: tuple[_PixelBatchResult, ...] = ()
    if pixel_batches:
        pixel_results = tuple(
            executor.map_batches(
                partial(
                    _gather_island_pixels,
                    source=source,
                    background_rms_source=background_rms_source,
                    publication_source=publication_source,
                    image_shape_yx=manifest.image_shape_yx,
                ),
                pixel_batches,
                requirement=read_requirement(
                    _SPANNING_BYTES_PER_READ_PIXEL,
                    (core.partition.read_bounds for core in spanning_cores),
                    chunk_shape_yx=manifest.tile_core_shape_yx,
                ),
            )
        )
        if not pixel_results:
            raise ValueError("executor returned no island pixel results")
    rows = (
        *(item for result in row_results for item in result.rows),
        *_spanning_rows(
            spanning,
            pixel_results,
            image_width=manifest.image_shape_yx[1],
            beam_area_pixels=config.beam_area_pixels,
        ),
    )
    return DetectionIslandStageResult(
        islands=tuple(
            row for _, row in sorted(rows, key=lambda item: item[0])
        ),
        island_ids_by_owner=island_ids_by_component(
            owner_pairs.components,
            owner_pairs.parents,
            tuple(
                sorted(
                    {pair for tile in tiles for pair in tile.component_parents}
                )
            ),
            identifier_by_island_label=identifier_by_island_label,
        ),
        island_count=len(islands),
        wide_island_count=len(spanning),
        partition_count=len(manifest.tiles),
        executor_task_count=(
            len(core_batches) + len(row_batches) + len(pixel_batches)
        ),
        maximum_graph_width=max(
            len(core_batches), len(row_batches), len(pixel_batches)
        ),
        maximum_island_read_pixels=max(
            (
                *(result.maximum_island_read_pixels for result in row_results),
                *(result.maximum_core_read_pixels for result in pixel_results),
            ),
            default=0,
        ),
        boundary_summary_array_bytes=sum(
            result.summary_array_bytes for result in scan_results
        ),
        reconciliation_round_count=reconciled.reduction_round_count,
    )

Catalogue rows and astrometry

hebog.data_models.astrometry

Compact celestial transformation and beam-deconvolution records.

CelestialCompactGaussianFit dataclass

One valid compact fit transformed to reviewed catalogue meanings.

Source code in src/hebog/data_models/astrometry.py

@dataclass(frozen=True, slots=True)
class CelestialCompactGaussianFit:
    """One valid compact fit transformed to reviewed catalogue meanings."""

    pixel_fit: ValidCompactGaussianFit
    position: SkyPosition
    fitted_flux: FluxMeasurement
    fitted_shape: GaussianShape
    deconvolution_status: Literal[
        "resolved",
        "major-axis-only",
        "unresolved",
        "unavailable",
    ]
    deconvolved_shape: GaussianShape | None
    deconvolved_major_fwhm_degrees: float | None
    quality_flags: tuple[str, ...]

GaussianDeconvolution dataclass

Identifiable axes or explicit absence after beam removal.

Source code in src/hebog/data_models/astrometry.py

@dataclass(frozen=True, slots=True)
class GaussianDeconvolution:
    """Identifiable axes or explicit absence after beam removal."""

    status: Literal[
        "resolved",
        "major-axis-only",
        "unresolved",
        "unavailable",
    ]
    shape: GaussianShape | None
    quality_flags: tuple[
        Literal[
            "deconvolution-uncertainty-unavailable",
            "extension-not-significant",
            "major-axis-not-significant",
            "major-axis-only",
            "marginal-deconvolution",
            "minor-axis-not-significant",
            "resolved",
            "unresolved",
        ],
        ...,
    ]
    major_axis_fwhm_degrees: float | None = None

    def __post_init__(self) -> None:
        """Keep full, one-axis, and absent states unambiguous."""
        if self.status == "resolved":
            if self.shape is None or self.major_axis_fwhm_degrees is not None:
                raise ValueError("resolved deconvolution requires an ellipse")
        elif self.status == "major-axis-only":
            if (
                self.shape is not None
                or self.major_axis_fwhm_degrees is None
                or not isfinite(self.major_axis_fwhm_degrees)
                or self.major_axis_fwhm_degrees <= 0
            ):
                raise ValueError(
                    "major-axis-only deconvolution requires one positive axis"
                )
        elif (
            self.shape is not None or self.major_axis_fwhm_degrees is not None
        ):
            raise ValueError("absent deconvolution cannot contain an axis")

__post_init__()

Keep full, one-axis, and absent states unambiguous.

Source code in src/hebog/data_models/astrometry.py

def __post_init__(self) -> None:
    """Keep full, one-axis, and absent states unambiguous."""
    if self.status == "resolved":
        if self.shape is None or self.major_axis_fwhm_degrees is not None:
            raise ValueError("resolved deconvolution requires an ellipse")
    elif self.status == "major-axis-only":
        if (
            self.shape is not None
            or self.major_axis_fwhm_degrees is None
            or not isfinite(self.major_axis_fwhm_degrees)
            or self.major_axis_fwhm_degrees <= 0
        ):
            raise ValueError(
                "major-axis-only deconvolution requires one positive axis"
            )
    elif (
        self.shape is not None or self.major_axis_fwhm_degrees is not None
    ):
        raise ValueError("absent deconvolution cannot contain an axis")

LocalTangentPlaneTransform dataclass

ICRS center and local east/north Jacobian for pixel (x, y).

Source code in src/hebog/data_models/astrometry.py

@dataclass(frozen=True, slots=True)
class LocalTangentPlaneTransform:
    """ICRS center and local east/north Jacobian for pixel `(x, y)`."""

    position: SkyPosition
    jacobian_degrees_per_pixel: tuple[
        tuple[float, float],
        tuple[float, float],
    ]

hebog.algorithms.astrometry

Astropy WCS transforms and covariance-based beam deconvolution.

celestial_wcs_from_header_text(header_text)

Rebuild a celestial WCS from the header text it was written from.

Work that crosses a task boundary carries this text rather than a :class:~astropy.wcs.WCS. Astropy pickles a WCS through a header it reformats itself, which perturbs the transform in its last bits and moves fitted uncertainties by parts in 1e9; rebuilding from the caller's own header text instead keeps every executor bit-identical. header_text is :meth:astropy.io.fits.Header.tostring output, so its cards are fixed-width and unseparated.

Source code in src/hebog/algorithms/astrometry.py

def celestial_wcs_from_header_text(header_text: str) -> WCS:
    """Rebuild a celestial WCS from the header text it was written from.

    Work that crosses a task boundary carries this text rather than a
    :class:`~astropy.wcs.WCS`. Astropy pickles a ``WCS`` through a header it
    reformats itself, which perturbs the transform in its last bits and moves
    fitted uncertainties by parts in ``1e9``; rebuilding from the caller's own
    header text instead keeps every executor bit-identical. ``header_text``
    is :meth:`astropy.io.fits.Header.tostring` output, so its cards are
    fixed-width and unseparated.
    """
    return WCS(fits.Header.fromstring(header_text), relax=True).celestial

celestial_wcs_from_metadata(metadata)

Reconstruct an independent celestial WCS from serialized metadata.

Source code in src/hebog/algorithms/astrometry.py

def celestial_wcs_from_metadata(metadata: ImageMetadata) -> WCS:
    """Reconstruct an independent celestial WCS from serialized metadata."""
    header = fits.Header.fromstring(
        metadata.celestial_wcs.fits_header,
        sep="\n",
    )
    return WCS(header, relax=True).celestial

compact_geometries_from_wcs(beam, celestial_wcs, positions_xy)

Transform beam and pixels for several positions in one conversion.

Equivalent to :func:compact_geometry_from_wcs at each position. One conversion pays Astropy's per-call frame machinery once rather than once per measured object, which dominates it at these window sizes.

Source code in src/hebog/algorithms/astrometry.py

def compact_geometries_from_wcs(
    beam: RestoringBeam,
    celestial_wcs: WCS,
    positions_xy: Sequence[tuple[float, float]],
) -> tuple[CompactMeasurementGeometry, ...]:
    """Transform beam and pixels for several positions in one conversion.

    Equivalent to :func:`compact_geometry_from_wcs` at each position. One
    conversion pays Astropy's per-call frame machinery once rather than once
    per measured object, which dominates it at these window sizes.
    """
    return tuple(
        compact_geometry_from_transform(rotated, transform)
        for rotated, transform in zip(
            restoring_beams_in_icrs(beam, celestial_wcs, positions_xy),
            local_tangent_plane_transforms_from_wcs(
                celestial_wcs, positions_xy
            ),
            strict=True,
        )
    )

compact_geometry_at_pixel(metadata, position_xy, *, celestial_wcs=None, transform=None)

Derive reviewed local pixel and restoring-beam solid angles.

Source code in src/hebog/algorithms/astrometry.py

def compact_geometry_at_pixel(
    metadata: ImageMetadata,
    position_xy: tuple[float, float],
    *,
    celestial_wcs: WCS | None = None,
    transform: LocalTangentPlaneTransform | None = None,
) -> CompactMeasurementGeometry:
    """Derive reviewed local pixel and restoring-beam solid angles."""
    local_transform = (
        transform
        if transform is not None
        else local_tangent_plane_transform(
            metadata,
            position_xy,
            celestial_wcs=celestial_wcs,
        )
    )
    return compact_geometry_from_transform(metadata.beam, local_transform)

compact_geometry_from_transform(beam, transform)

Derive local measurement geometry without unrelated spectral data.

Source code in src/hebog/algorithms/astrometry.py

def compact_geometry_from_transform(
    beam: RestoringBeam,
    transform: LocalTangentPlaneTransform,
) -> CompactMeasurementGeometry:
    """Derive local measurement geometry without unrelated spectral data."""
    jacobian = np.asarray(
        transform.jacobian_degrees_per_pixel,
        dtype=np.float64,
    )
    pixel_area_square_degrees = abs(float(np.linalg.det(jacobian)))
    pixel_solid_angle = pixel_area_square_degrees * (pi / 180.0) ** 2
    beam_solid_angle = gaussian_beam_solid_angle_steradians(
        major_fwhm_degrees=beam.major_fwhm_degrees,
        minor_fwhm_degrees=beam.minor_fwhm_degrees,
    )
    inverse_jacobian = np.linalg.inv(jacobian)
    pixel_noise_covariance = (
        inverse_jacobian @ _sky_covariance(beam) @ inverse_jacobian.T
    )
    covariance_values = (
        float(pixel_noise_covariance[0, 0]),
        float(pixel_noise_covariance[0, 1]),
        float(pixel_noise_covariance[1, 1]),
    )
    return CompactMeasurementGeometry(
        pixel_solid_angle_steradians=pixel_solid_angle,
        restoring_beam_solid_angle_steradians=beam_solid_angle,
        restoring_beam_covariance_pixels_squared=covariance_values,
        noise_correlation_covariance_pixels_squared=covariance_values,
    )

compact_geometry_from_wcs(beam, celestial_wcs, position_xy)

Transform native beam and pixels into the same ICRS tangent basis.

Source code in src/hebog/algorithms/astrometry.py

def compact_geometry_from_wcs(
    beam: RestoringBeam,
    celestial_wcs: WCS,
    position_xy: tuple[float, float],
) -> CompactMeasurementGeometry:
    """Transform native beam and pixels into the same ICRS tangent basis."""
    return compact_geometry_from_transform(
        restoring_beam_in_icrs(beam, celestial_wcs, position_xy),
        local_tangent_plane_transform_from_wcs(celestial_wcs, position_xy),
    )

deconvolve_gaussian_shapes(fitted_shape, beam, *, relative_tolerance)

Subtract beam covariance with explicit absence states.

Source code in src/hebog/algorithms/astrometry.py

def deconvolve_gaussian_shapes(
    fitted_shape: GaussianShape,
    beam: RestoringBeam,
    *,
    relative_tolerance: float,
) -> GaussianDeconvolution:
    """Subtract beam covariance with explicit absence states."""
    if not isfinite(relative_tolerance) or not 0 < relative_tolerance < 1:
        raise ValueError("relative_tolerance must be finite and in (0, 1)")
    beam_covariance = _sky_covariance(beam)
    intrinsic = _sky_covariance(fitted_shape) - beam_covariance
    eigenvalues = np.linalg.eigvalsh(intrinsic)
    tolerance = relative_tolerance * float(
        np.max(np.linalg.eigvalsh(beam_covariance))
    )
    if float(np.max(eigenvalues)) <= tolerance:
        return GaussianDeconvolution(
            status="unresolved",
            shape=None,
            quality_flags=("unresolved",),
        )
    if float(np.min(eigenvalues)) <= tolerance:
        major_fwhm = float(sqrt(float(np.max(eigenvalues))) * _FWHM_PER_SIGMA)
        return GaussianDeconvolution(
            status="major-axis-only",
            shape=None,
            quality_flags=("major-axis-only", "marginal-deconvolution"),
            major_axis_fwhm_degrees=major_fwhm,
        )
    return GaussianDeconvolution(
        status="resolved",
        shape=_shape_from_sky_covariance(intrinsic),
        quality_flags=("resolved",),
    )

local_tangent_plane_transform(metadata, position_xy, *, celestial_wcs=None)

Return the ICRS center and local east/north pixel Jacobian.

ICRS and FK5 inputs are both transformed to ICRS through Astropy's frame tie, matching :func:restoring_beam_in_icrs.

Source code in src/hebog/algorithms/astrometry.py

def local_tangent_plane_transform(
    metadata: ImageMetadata,
    position_xy: tuple[float, float],
    *,
    celestial_wcs: WCS | None = None,
) -> LocalTangentPlaneTransform:
    """Return the ICRS center and local east/north pixel Jacobian.

    ICRS and FK5 inputs are both transformed to ICRS through Astropy's frame
    tie, matching :func:`restoring_beam_in_icrs`.
    """
    if metadata.celestial_wcs.coordinate_frame.lower() not in {"icrs", "fk5"}:
        raise ValueError(
            "compact astrometry requires an ICRS or FK5 celestial WCS"
        )
    wcs = (
        celestial_wcs
        if celestial_wcs is not None
        else celestial_wcs_from_metadata(metadata)
    )
    return local_tangent_plane_transform_from_wcs(wcs, position_xy)

local_tangent_plane_transform_from_wcs(celestial_wcs, position_xy)

Return the ICRS tangent transform from an explicit celestial WCS.

Source code in src/hebog/algorithms/astrometry.py

def local_tangent_plane_transform_from_wcs(
    celestial_wcs: WCS,
    position_xy: tuple[float, float],
) -> LocalTangentPlaneTransform:
    """Return the ICRS tangent transform from an explicit celestial WCS."""
    if not celestial_wcs.has_celestial:
        raise ValueError("astrometry requires a celestial WCS")
    x, y = position_xy
    wcs = celestial_wcs.celestial
    step_x, step_y = _finite_difference_steps_pixels(wcs)
    # One conversion for the centre and the four finite-difference
    # neighbours together. Converting them one at a time builds five sky
    # coordinates and five frame transforms per measured source, which
    # costs more than everything else in the catalogue.
    sampled = wcs.pixel_to_world(
        np.asarray([x, x + step_x, x - step_x, x, x], dtype=np.float64),
        np.asarray([y, y, y, y + step_y, y - step_y], dtype=np.float64),
    ).icrs
    center = sampled[0]
    east, north = center.spherical_offsets_to(sampled[1:])
    columns: list[tuple[float, float]] = []
    for plus, minus, step in ((0, 1, step_x), (2, 3, step_y)):
        columns.append(
            (
                (east[plus].degree - east[minus].degree) / (2.0 * step),
                (north[plus].degree - north[minus].degree) / (2.0 * step),
            )
        )
    jacobian = (
        (columns[0][0], columns[1][0]),
        (columns[0][1], columns[1][1]),
    )
    return LocalTangentPlaneTransform(
        position=SkyPosition(
            right_ascension_degrees=float(center.ra.degree % 360.0),
            declination_degrees=float(center.dec.degree),
            right_ascension_error_degrees=None,
            declination_error_degrees=None,
        ),
        jacobian_degrees_per_pixel=jacobian,
    )

local_tangent_plane_transforms_from_wcs(celestial_wcs, positions_xy)

Return one ICRS tangent transform per position, in one conversion.

Equivalent to :func:local_tangent_plane_transform_from_wcs at each position, and exact rather than approximate: Astropy applies the same element-wise transform whether it is given one coordinate or many, so the sampled sky positions are bit-for-bit those of the single-position calls. Its cost is the per-call frame machinery, which one conversion pays once instead of once per measured object.

Source code in src/hebog/algorithms/astrometry.py

def local_tangent_plane_transforms_from_wcs(
    celestial_wcs: WCS,
    positions_xy: Sequence[tuple[float, float]],
) -> tuple[LocalTangentPlaneTransform, ...]:
    """Return one ICRS tangent transform per position, in one conversion.

    Equivalent to :func:`local_tangent_plane_transform_from_wcs` at each
    position, and exact rather than approximate: Astropy applies the same
    element-wise transform whether it is given one coordinate or many, so
    the sampled sky positions are bit-for-bit those of the single-position
    calls. Its cost is the per-call frame machinery, which one conversion
    pays once instead of once per measured object.
    """
    if not celestial_wcs.has_celestial:
        raise ValueError("astrometry requires a celestial WCS")
    wcs = celestial_wcs.celestial
    # Size the step before the empty batch returns, so an unusable pixel
    # scale fails closed whether or not this batch holds an object.
    step_x, step_y = _finite_difference_steps_pixels(wcs)
    if not positions_xy:
        return ()
    xs = np.asarray([x for x, _ in positions_xy], dtype=np.float64)
    ys = np.asarray([y for _, y in positions_xy], dtype=np.float64)
    count = xs.size
    sampled = wcs.pixel_to_world(
        np.concatenate((xs, xs + step_x, xs - step_x, xs, xs)),
        np.concatenate((ys, ys, ys, ys + step_y, ys - step_y)),
    ).icrs
    centers = sampled[:count]
    offsets = [
        centers.spherical_offsets_to(
            sampled[index * count : (index + 1) * count]
        )
        for index in range(1, 5)
    ]
    east_degrees = [offset[0].degree for offset in offsets]
    north_degrees = [offset[1].degree for offset in offsets]
    right_ascension = centers.ra.degree % 360.0
    declination = centers.dec.degree
    return tuple(
        LocalTangentPlaneTransform(
            position=SkyPosition(
                right_ascension_degrees=float(right_ascension[index]),
                declination_degrees=float(declination[index]),
                right_ascension_error_degrees=None,
                declination_error_degrees=None,
            ),
            jacobian_degrees_per_pixel=(
                (
                    float(
                        (east_degrees[0][index] - east_degrees[1][index])
                        / (2.0 * step_x)
                    ),
                    float(
                        (east_degrees[2][index] - east_degrees[3][index])
                        / (2.0 * step_y)
                    ),
                ),
                (
                    float(
                        (north_degrees[0][index] - north_degrees[1][index])
                        / (2.0 * step_x)
                    ),
                    float(
                        (north_degrees[2][index] - north_degrees[3][index])
                        / (2.0 * step_y)
                    ),
                ),
            ),
        )
        for index in range(count)
    )

moment_equivalent_gaussian_shape(covariance_pixels_squared, transform)

Transform one positive pixel moment covariance into a sky ellipse.

Source code in src/hebog/algorithms/astrometry.py

def moment_equivalent_gaussian_shape(
    covariance_pixels_squared: npt.ArrayLike,
    transform: LocalTangentPlaneTransform,
) -> GaussianShape:
    """Transform one positive pixel moment covariance into a sky ellipse."""
    covariance = np.asarray(covariance_pixels_squared, dtype=np.float64)
    if covariance.shape != (_SHAPE_AXIS_PARAMETER_COUNT,) * 2 or not np.all(
        np.isfinite(covariance)
    ):
        raise ValueError("moment covariance must be one finite 2-by-2 matrix")
    if not np.allclose(covariance, covariance.T, rtol=0.0, atol=1e-14):
        raise ValueError("moment covariance must be symmetric")
    eigenvalues = np.linalg.eigvalsh(covariance)
    if float(np.min(eigenvalues)) <= 0.0:
        raise ValueError("moment covariance must be positive definite")
    jacobian = np.asarray(
        transform.jacobian_degrees_per_pixel,
        dtype=np.float64,
    )
    sky_covariance = jacobian @ covariance @ jacobian.T
    sky_eigenvalues = np.linalg.eigvalsh(sky_covariance)
    if (
        not np.all(np.isfinite(sky_covariance))
        or float(np.min(sky_eigenvalues)) <= 0.0
    ):
        raise ValueError("transformed moment covariance must be positive")
    shape = _shape_from_sky_covariance(sky_covariance)
    position_angle = shape.position_angle_degrees
    if np.isclose(position_angle, 180.0, rtol=0.0, atol=1e-6):
        position_angle = 0.0
    return GaussianShape(
        major_fwhm_degrees=shape.major_fwhm_degrees,
        minor_fwhm_degrees=shape.minor_fwhm_degrees,
        position_angle_degrees=position_angle,
        major_fwhm_error_degrees=None,
        minor_fwhm_error_degrees=None,
        position_angle_error_degrees=None,
    )

restoring_beam_in_icrs(beam, celestial_wcs, position_xy)

Rotate a native equatorial FITS beam PA into local ICRS axes.

FK5 equinox precession rotates local north along with the coordinates. Transform a one-arcsecond directional offset using Astropy, preserving the angular beam axes. ICRS returns the original beam without arithmetic. This boundary supports ICRS/FK5, not frames with non-rotational geometry.

Source code in src/hebog/algorithms/astrometry.py

def restoring_beam_in_icrs(
    beam: RestoringBeam,
    celestial_wcs: WCS,
    position_xy: tuple[float, float],
) -> RestoringBeam:
    """Rotate a native equatorial FITS beam PA into local ICRS axes.

    FK5 equinox precession rotates local north along with the coordinates.
    Transform a one-arcsecond directional offset using Astropy, preserving
    the angular beam axes. ICRS returns the original beam without arithmetic.
    This boundary supports ICRS/FK5, not frames with non-rotational geometry.
    """
    if not celestial_wcs.has_celestial:
        raise ValueError("beam geometry requires a celestial WCS")
    frame = wcs_to_celestial_frame(celestial_wcs)
    if frame.name == "icrs":
        return beam
    if frame.name != "fk5":
        raise ValueError("beam geometry requires an ICRS or FK5 celestial WCS")
    center = celestial_wcs.celestial.pixel_to_world(*position_xy)
    direction = center.directional_offset_by(
        beam.position_angle_degrees * u.deg, 1.0 * u.arcsec
    )
    return replace(
        beam,
        position_angle_degrees=float(
            center.icrs.position_angle(direction.icrs).deg % 180.0
        ),
    )

restoring_beams_in_icrs(beam, celestial_wcs, positions_xy)

Rotate one native beam into local ICRS axes at several positions.

Equivalent to :func:restoring_beam_in_icrs at each position, for the same reason :func:local_tangent_plane_transforms_from_wcs is: the offset and position-angle transforms are element-wise.

Source code in src/hebog/algorithms/astrometry.py

def restoring_beams_in_icrs(
    beam: RestoringBeam,
    celestial_wcs: WCS,
    positions_xy: Sequence[tuple[float, float]],
) -> tuple[RestoringBeam, ...]:
    """Rotate one native beam into local ICRS axes at several positions.

    Equivalent to :func:`restoring_beam_in_icrs` at each position, for the
    same reason :func:`local_tangent_plane_transforms_from_wcs` is: the
    offset and position-angle transforms are element-wise.
    """
    if not celestial_wcs.has_celestial:
        raise ValueError("beam geometry requires a celestial WCS")
    # Validate the frame before the empty batch returns, so an unsupported
    # one fails closed whether or not this batch holds an object.
    frame = wcs_to_celestial_frame(celestial_wcs)
    if frame.name not in {"icrs", "fk5"}:
        raise ValueError("beam geometry requires an ICRS or FK5 celestial WCS")
    if not positions_xy:
        return ()
    if frame.name == "icrs":
        return (beam,) * len(positions_xy)
    centers = celestial_wcs.celestial.pixel_to_world(
        np.asarray([x for x, _ in positions_xy], dtype=np.float64),
        np.asarray([y for _, y in positions_xy], dtype=np.float64),
    )
    directions = centers.directional_offset_by(
        beam.position_angle_degrees * u.deg, 1.0 * u.arcsec
    )
    angles = centers.icrs.position_angle(directions.icrs).deg % 180.0
    return tuple(
        replace(beam, position_angle_degrees=float(angle)) for angle in angles
    )

transform_compact_fit_at_tangent(fit, beam, transform, *, deconvolution_relative_tolerance=1e-10, extension_significance_sigma=5.0, deconvolution_axis_significance_sigma=5.0)

Project a Jy/beam fit using an explicit local celestial transform.

Source code in src/hebog/algorithms/astrometry.py

def transform_compact_fit_at_tangent(  # noqa: PLR0913
    fit: ValidCompactGaussianFit,
    beam: RestoringBeam,
    transform: LocalTangentPlaneTransform,
    *,
    deconvolution_relative_tolerance: float = 1e-10,
    extension_significance_sigma: float = 5.0,
    deconvolution_axis_significance_sigma: float = 5.0,
) -> CelestialCompactGaussianFit:
    """Project a Jy/beam fit using an explicit local celestial transform."""
    if (
        not isfinite(extension_significance_sigma)
        or extension_significance_sigma <= 0
    ):
        raise ValueError(
            "extension_significance_sigma must be finite and positive"
        )
    if (
        not isfinite(deconvolution_axis_significance_sigma)
        or deconvolution_axis_significance_sigma <= 0
    ):
        raise ValueError(
            "deconvolution_axis_significance_sigma must be finite and positive"
        )
    parameters = fit.parameters
    jacobian = np.asarray(transform.jacobian_degrees_per_pixel)
    fitted_covariance = (
        jacobian
        @ _pixel_covariance(
            parameters.major_sigma_pixels,
            parameters.minor_sigma_pixels,
            parameters.major_axis_angle_degrees,
        )
        @ jacobian.T
    )
    fitted_shape = _fitted_shape_with_errors(
        _shape_from_sky_covariance(fitted_covariance), fit, jacobian
    )
    geometric_deconvolution = deconvolve_gaussian_shapes(
        fitted_shape,
        beam,
        relative_tolerance=deconvolution_relative_tolerance,
    )
    geometry = compact_geometry_from_transform(beam, transform)
    uncertainty = fit.uncertainty
    integrated_flux = fitted_gaussian_integrated_flux_jy(
        amplitude_jy_per_beam=parameters.amplitude_jy_per_beam,
        major_sigma_pixels=parameters.major_sigma_pixels,
        minor_sigma_pixels=parameters.minor_sigma_pixels,
        geometry=geometry,
    )
    integrated_flux_error = (
        uncertainty.integrated_flux_error_jy
        * integrated_flux
        / parameters.integrated_flux_jy
        if uncertainty is not None
        else None
    )
    if uncertainty is not None and integrated_flux_error is not None:
        integrated_flux -= (
            uncertainty.integrated_flux_bias_correction_sigma
            * integrated_flux_error
        )
        if integrated_flux <= 0.0:
            raise ValueError(
                "integrated-flux bias correction produced non-positive flux"
            )
    deconvolution = _extension_classification(
        geometric_deconvolution,
        fit,
        integrated_flux_jy=integrated_flux,
        integrated_flux_error_jy=integrated_flux_error,
        significance_sigma=extension_significance_sigma,
    )
    deconvolution = _axis_significance_classification(
        deconvolution,
        fit,
        jacobian,
        beam,
        significance_sigma=deconvolution_axis_significance_sigma,
        relative_tolerance=deconvolution_relative_tolerance,
    )
    fitted_flux = FluxMeasurement(
        peak_flux_jy_per_beam=parameters.amplitude_jy_per_beam,
        peak_flux_error_jy_per_beam=(
            uncertainty.amplitude_error_jy_per_beam
            if uncertainty is not None
            else None
        ),
        integrated_flux_jy=integrated_flux,
        integrated_flux_error_jy=integrated_flux_error,
        local_rms_jy_per_beam=parameters.local_rms_jy_per_beam,
    )
    flags = set(fit.quality_flags)
    if fitted_shape.major_fwhm_error_degrees is None:
        flags.add("shape-uncertainty-unavailable")
    flags.update(deconvolution.quality_flags)
    if uncertainty is None:
        flags.add("position-flux-uncertainty-unavailable")
    elif uncertainty.integrated_flux_bias_correction_sigma > 0.0:
        flags.add("fitted-integrated-flux-bias-corrected")
    return CelestialCompactGaussianFit(
        pixel_fit=fit,
        position=_position_with_errors(transform, fit),
        fitted_flux=fitted_flux,
        fitted_shape=fitted_shape,
        deconvolution_status=deconvolution.status,
        deconvolved_shape=deconvolution.shape,
        deconvolved_major_fwhm_degrees=(deconvolution.major_axis_fwhm_degrees),
        quality_flags=tuple(sorted(flags)),
    )

hebog.science.catalogue_rows

Internal catalogue rows in canonical physical units.

The quick check's PyBDSF reference worker builds these rows, and its cached results are keyed by this module's code. The module therefore holds only the rows, apart from the composition records in :mod:hebog.science.models and the algorithms they import, so that editing those keeps the cached results.

CatalogueEllipse dataclass

One canonical catalogue ellipse and optional one-sigma errors.

Source code in src/hebog/science/catalogue_rows.py

@dataclass(frozen=True, slots=True)
class CatalogueEllipse:
    """One canonical catalogue ellipse and optional one-sigma errors."""

    major_fwhm_degrees: float
    minor_fwhm_degrees: float
    position_angle_degrees: float
    major_fwhm_error_degrees: float | None = None
    minor_fwhm_error_degrees: float | None = None
    position_angle_error_degrees: float | None = None

    def __post_init__(self) -> None:
        """Require a positive ordered ellipse modulo 180 degrees."""
        if (
            not np.isfinite(self.major_fwhm_degrees)
            or not np.isfinite(self.minor_fwhm_degrees)
            or self.major_fwhm_degrees <= 0
            or self.minor_fwhm_degrees <= 0
        ):
            raise ValueError(
                "catalogue ellipse axes must be finite and positive"
            )
        if self.minor_fwhm_degrees > self.major_fwhm_degrees:
            raise ValueError("catalogue ellipse axes must be ordered")
        if not np.isfinite(self.position_angle_degrees):
            raise ValueError("catalogue position angle must be finite")
        object.__setattr__(
            self,
            "position_angle_degrees",
            self.position_angle_degrees % _HALF_CIRCLE_DEGREES,
        )
        _require_optional_positive(
            (
                self.major_fwhm_error_degrees,
                self.minor_fwhm_error_degrees,
                self.position_angle_error_degrees,
            ),
            field_name="catalogue ellipse errors",
        )

__post_init__()

Require a positive ordered ellipse modulo 180 degrees.

Source code in src/hebog/science/catalogue_rows.py

def __post_init__(self) -> None:
    """Require a positive ordered ellipse modulo 180 degrees."""
    if (
        not np.isfinite(self.major_fwhm_degrees)
        or not np.isfinite(self.minor_fwhm_degrees)
        or self.major_fwhm_degrees <= 0
        or self.minor_fwhm_degrees <= 0
    ):
        raise ValueError(
            "catalogue ellipse axes must be finite and positive"
        )
    if self.minor_fwhm_degrees > self.major_fwhm_degrees:
        raise ValueError("catalogue ellipse axes must be ordered")
    if not np.isfinite(self.position_angle_degrees):
        raise ValueError("catalogue position angle must be finite")
    object.__setattr__(
        self,
        "position_angle_degrees",
        self.position_angle_degrees % _HALF_CIRCLE_DEGREES,
    )
    _require_optional_positive(
        (
            self.major_fwhm_error_degrees,
            self.minor_fwhm_error_degrees,
            self.position_angle_error_degrees,
        ),
        field_name="catalogue ellipse errors",
    )

CatalogueSource dataclass

One internal source-catalogue row in canonical physical units.

Source code in src/hebog/science/catalogue_rows.py

@dataclass(frozen=True, slots=True)
class CatalogueSource:
    """One internal source-catalogue row in canonical physical units."""

    identifier: str
    right_ascension_degrees: float
    declination_degrees: float
    peak_flux_jy_per_beam: float
    integrated_flux_jy: float
    right_ascension_error_degrees: float | None = None
    declination_error_degrees: float | None = None
    peak_flux_error_jy_per_beam: float | None = None
    integrated_flux_error_jy: float | None = None
    fitted_shape: CatalogueEllipse | None = None
    deconvolved_shape: CatalogueEllipse | None = None
    deconvolved_major_fwhm_degrees: float | None = None
    deconvolution_status: Literal[
        "resolved",
        "major-axis-only",
        "unresolved",
        "unavailable",
    ] = "unavailable"
    island_identifier: str | None = None
    component_count: int | None = None
    quality_flags: tuple[str, ...] = ()
    association_integrated_flux_jy: float | None = None

    def _validate_optional_metadata(self) -> None:
        """Validate optional errors, associations, and quality flags."""
        _require_optional_positive(
            (
                self.right_ascension_error_degrees,
                self.declination_error_degrees,
                self.peak_flux_error_jy_per_beam,
                self.integrated_flux_error_jy,
            ),
            field_name="source errors",
        )
        if self.island_identifier is not None and not self.island_identifier:
            raise ValueError("island identifier must not be empty")
        if self.component_count is not None and self.component_count <= 0:
            raise ValueError("component count must be positive")
        if self.quality_flags != tuple(sorted(set(self.quality_flags))) or any(
            not flag for flag in self.quality_flags
        ):
            raise ValueError("quality flags must be non-empty and canonical")

    def _validate_deconvolution(self) -> None:
        """Keep full, one-axis, unresolved, and unavailable states clear."""
        if self.deconvolution_status == "resolved":
            if (
                self.deconvolved_shape is None
                or self.deconvolved_major_fwhm_degrees is not None
            ):
                raise ValueError("resolved deconvolution requires a shape")
        elif self.deconvolution_status == "major-axis-only":
            major = self.deconvolved_major_fwhm_degrees
            if (
                self.deconvolved_shape is not None
                or major is None
                or not np.isfinite(major)
                or major <= 0
            ):
                raise ValueError(
                    "major-axis-only deconvolution requires one positive axis"
                )
            if "major-axis-only" not in self.quality_flags:
                raise ValueError(
                    "major-axis-only deconvolution requires its quality flag"
                )
        elif (
            self.deconvolved_shape is not None
            or self.deconvolved_major_fwhm_degrees is not None
        ):
            raise ValueError(
                "only identifiable deconvolution may contain an axis"
            )
        if (
            self.deconvolution_status == "unresolved"
            and "unresolved" not in self.quality_flags
        ):
            raise ValueError(
                "unresolved deconvolution requires its quality flag"
            )

    def __post_init__(self) -> None:
        """Validate identity, coordinates, and positive finite fluxes."""
        if not self.identifier:
            raise ValueError("source identifier must not be empty")
        numeric_values = (
            self.right_ascension_degrees,
            self.declination_degrees,
            self.peak_flux_jy_per_beam,
            self.integrated_flux_jy,
        )
        if not all(np.isfinite(value) for value in numeric_values):
            raise ValueError("source coordinates and fluxes must be finite")
        if not (
            _MINIMUM_DECLINATION_DEGREES
            <= self.declination_degrees
            <= _MAXIMUM_DECLINATION_DEGREES
        ):
            raise ValueError("declination must be within [-90, 90] degrees")
        if self.peak_flux_jy_per_beam <= 0 or self.integrated_flux_jy <= 0:
            raise ValueError("source fluxes must be positive")
        association_flux = self.association_integrated_flux_jy
        if association_flux is not None and (
            not np.isfinite(association_flux) or association_flux <= 0
        ):
            raise ValueError(
                "association integrated flux must be finite and positive"
            )
        self._validate_optional_metadata()
        self._validate_deconvolution()
        normalized_right_ascension = (
            self.right_ascension_degrees % _FULL_CIRCLE_DEGREES
        )
        object.__setattr__(
            self,
            "right_ascension_degrees",
            normalized_right_ascension,
        )

    @classmethod
    def from_units(  # noqa: PLR0913
        cls,
        *,
        identifier: str,
        right_ascension: float,
        declination: float,
        angle_unit: Literal["deg", "arcsec"],
        peak_flux_density: float,
        peak_flux_unit: Literal["Jy/beam", "mJy/beam"],
        integrated_flux_density: float,
        integrated_flux_unit: Literal["Jy", "mJy"],
    ) -> Self:
        """Convert supported angular and flux units into canonical units."""
        angle_scale = _conversion_scale(
            angle_unit,
            {"deg": 1.0, "arcsec": 1.0 / 3600.0},
            field_name="angle_unit",
        )
        peak_flux_scale = _conversion_scale(
            peak_flux_unit,
            {"Jy/beam": 1.0, "mJy/beam": 0.001},
            field_name="peak_flux_unit",
        )
        integrated_flux_scale = _conversion_scale(
            integrated_flux_unit,
            {"Jy": 1.0, "mJy": 0.001},
            field_name="integrated_flux_unit",
        )
        return cls(
            identifier=identifier,
            right_ascension_degrees=right_ascension * angle_scale,
            declination_degrees=declination * angle_scale,
            peak_flux_jy_per_beam=peak_flux_density * peak_flux_scale,
            integrated_flux_jy=(
                integrated_flux_density * integrated_flux_scale
            ),
        )

__post_init__()

Validate identity, coordinates, and positive finite fluxes.

Source code in src/hebog/science/catalogue_rows.py

def __post_init__(self) -> None:
    """Validate identity, coordinates, and positive finite fluxes."""
    if not self.identifier:
        raise ValueError("source identifier must not be empty")
    numeric_values = (
        self.right_ascension_degrees,
        self.declination_degrees,
        self.peak_flux_jy_per_beam,
        self.integrated_flux_jy,
    )
    if not all(np.isfinite(value) for value in numeric_values):
        raise ValueError("source coordinates and fluxes must be finite")
    if not (
        _MINIMUM_DECLINATION_DEGREES
        <= self.declination_degrees
        <= _MAXIMUM_DECLINATION_DEGREES
    ):
        raise ValueError("declination must be within [-90, 90] degrees")
    if self.peak_flux_jy_per_beam <= 0 or self.integrated_flux_jy <= 0:
        raise ValueError("source fluxes must be positive")
    association_flux = self.association_integrated_flux_jy
    if association_flux is not None and (
        not np.isfinite(association_flux) or association_flux <= 0
    ):
        raise ValueError(
            "association integrated flux must be finite and positive"
        )
    self._validate_optional_metadata()
    self._validate_deconvolution()
    normalized_right_ascension = (
        self.right_ascension_degrees % _FULL_CIRCLE_DEGREES
    )
    object.__setattr__(
        self,
        "right_ascension_degrees",
        normalized_right_ascension,
    )

from_units(*, identifier, right_ascension, declination, angle_unit, peak_flux_density, peak_flux_unit, integrated_flux_density, integrated_flux_unit) classmethod

Convert supported angular and flux units into canonical units.

Source code in src/hebog/science/catalogue_rows.py

@classmethod
def from_units(  # noqa: PLR0913
    cls,
    *,
    identifier: str,
    right_ascension: float,
    declination: float,
    angle_unit: Literal["deg", "arcsec"],
    peak_flux_density: float,
    peak_flux_unit: Literal["Jy/beam", "mJy/beam"],
    integrated_flux_density: float,
    integrated_flux_unit: Literal["Jy", "mJy"],
) -> Self:
    """Convert supported angular and flux units into canonical units."""
    angle_scale = _conversion_scale(
        angle_unit,
        {"deg": 1.0, "arcsec": 1.0 / 3600.0},
        field_name="angle_unit",
    )
    peak_flux_scale = _conversion_scale(
        peak_flux_unit,
        {"Jy/beam": 1.0, "mJy/beam": 0.001},
        field_name="peak_flux_unit",
    )
    integrated_flux_scale = _conversion_scale(
        integrated_flux_unit,
        {"Jy": 1.0, "mJy": 0.001},
        field_name="integrated_flux_unit",
    )
    return cls(
        identifier=identifier,
        right_ascension_degrees=right_ascension * angle_scale,
        declination_degrees=declination * angle_scale,
        peak_flux_jy_per_beam=peak_flux_density * peak_flux_scale,
        integrated_flux_jy=(
            integrated_flux_density * integrated_flux_scale
        ),
    )

hebog.science.segment_rows

Segment catalogue rows and segment moments of the installed composition.

A segment is one detection label measured as a whole: its position, fluxes and moment-equivalent shape, before any component is fitted.

SegmentPixels dataclass

The pixels one segment's row reads, in raster order.

A pixel is here when the segment holds it in its seeded ownership (owned) or in its expanded aperture (aperture); a caller may add others. Coordinates are in the image's pixel frame and every array is aligned with y. Each reduction a row makes is over one of those two sets, so any such superset in raster order gives the same row: a window over the segment, or the pieces its cores return.

Source code in src/hebog/science/segment_rows.py

@dataclass(frozen=True, slots=True)
class SegmentPixels:
    """The pixels one segment's row reads, in raster order.

    A pixel is here when the segment holds it in its seeded ownership
    (``owned``) or in its expanded aperture (``aperture``); a caller may add
    others. Coordinates are in the image's pixel frame and every array is
    aligned with ``y``. Each reduction a row makes is over one of those two
    sets, so any such superset in raster order gives the same row: a window
    over the segment, or the pieces its cores return.
    """

    y: npt.NDArray[np.int64]
    x: npt.NDArray[np.int64]
    residual: npt.NDArray[np.float64]
    position_signal: npt.NDArray[np.float64] | None
    background: npt.NDArray[np.float64]
    valid: npt.NDArray[np.bool_]
    owned: npt.NDArray[np.bool_]
    aperture: npt.NDArray[np.bool_]

SegmentRowMeasurement dataclass

One segment's measured row, before its centroid has a sky position.

centroid_xy is where the row's coordinate belongs, in the image's pixel frame, so a caller measuring many segments takes every centroid first and converts them together; see :func:segment_row_at.

Source code in src/hebog/science/segment_rows.py

@dataclass(frozen=True, slots=True)
class SegmentRowMeasurement:
    """One segment's measured row, before its centroid has a sky position.

    ``centroid_xy`` is where the row's coordinate belongs, in the image's
    pixel frame, so a caller measuring many segments takes every centroid
    first and converts them together; see :func:`segment_row_at`.
    """

    label_value: int
    centroid_xy: tuple[float, float]
    peak_flux_jy_per_beam: float
    integrated_flux_jy: float
    quality_flags: tuple[str, ...]

build_hebog_segment_catalogue(image_jy_per_beam, background_jy_per_beam, valid_pixels, component_labels, header, *, beam_major_fwhm_pixels, beam_minor_fwhm_pixels, measurement_aperture_radius_beams=4.0, position_signal_jy_per_beam=None, denoised_position_maximum_peak_to_mean_ratio=3.0, aperture_tie_policy='nearest-support', position_labels=None, position_diagnostics=None)

Measure catalogue rows for physically measurable blind segments.

Labels remain the authoritative record of every accepted detection. Expanded-aperture photometry is preferred. When surrounding negative residuals make that aperture non-positive, the accepted owner's positive exact support provides an explicitly flagged conservative fallback.

Source code in src/hebog/science/segment_rows.py

def build_hebog_segment_catalogue(  # noqa: PLR0913
    image_jy_per_beam: npt.ArrayLike,
    background_jy_per_beam: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    component_labels: npt.ArrayLike,
    header: fits.Header,
    *,
    beam_major_fwhm_pixels: float,
    beam_minor_fwhm_pixels: float,
    measurement_aperture_radius_beams: float = 4.0,
    position_signal_jy_per_beam: npt.ArrayLike | None = None,
    denoised_position_maximum_peak_to_mean_ratio: float = 3.0,
    aperture_tie_policy: Literal[
        "nearest-support", "canonical-source"
    ] = "nearest-support",
    position_labels: npt.ArrayLike | None = None,
    position_diagnostics: dict[int, SourcePositionDiagnostics] | None = None,
) -> tuple[CatalogueSource, ...]:
    """Measure catalogue rows for physically measurable blind segments.

    Labels remain the authoritative record of every accepted detection.
    Expanded-aperture photometry is preferred. When surrounding negative
    residuals make that aperture non-positive, the accepted owner's positive
    exact support provides an explicitly flagged conservative fallback.
    """
    residual, valid, labels = _validated_hebog_segment_planes(
        image_jy_per_beam,
        background_jy_per_beam,
        valid_pixels,
        component_labels,
    )
    if (
        not np.isfinite(beam_major_fwhm_pixels)
        or not np.isfinite(beam_minor_fwhm_pixels)
        or beam_major_fwhm_pixels <= 0.0
        or beam_minor_fwhm_pixels <= 0.0
    ):
        raise ValueError("Hebog segment beam axes must be positive")
    if (
        not np.isfinite(measurement_aperture_radius_beams)
        or measurement_aperture_radius_beams <= 0.0
    ):
        raise ValueError(
            "Hebog segment measurement aperture radius must be positive"
        )
    if (
        not np.isfinite(denoised_position_maximum_peak_to_mean_ratio)
        or denoised_position_maximum_peak_to_mean_ratio <= 1.0
    ):
        raise ValueError(
            "Hebog segment denoised-position peak-to-mean ratio must exceed 1"
        )
    if aperture_tie_policy not in {"nearest-support", "canonical-source"}:
        raise ValueError("Hebog segment aperture tie policy is unsupported")
    position_signal = _validated_position_signal(
        position_signal_jy_per_beam,
        shape=residual.shape,
        valid_pixels=valid,
    )
    centroid_labels = _validated_centroid_labels(position_labels, labels)
    aperture_builder = (
        expand_source_measurement_labels
        if aperture_tie_policy == "canonical-source"
        else expand_detected_segment_labels
    )
    measurement_labels = aperture_builder(
        labels,
        valid & np.isfinite(residual),
        radius_pixels=ceil(
            measurement_aperture_radius_beams * beam_major_fwhm_pixels
        ),
    )
    beam_area_pixels = (
        2.0
        * np.pi
        / (8.0 * np.log(2.0))
        * beam_major_fwhm_pixels
        * beam_minor_fwhm_pixels
    )
    celestial_wcs = WCS(header, relax=True).celestial
    background = np.asarray(background_jy_per_beam)
    # One pass gives every segment its aperture window, so the work below
    # costs each segment's own pixels instead of the whole image.
    aperture_windows = label_windows(measurement_labels)
    segment_windows = label_windows(labels)
    output: list[CatalogueSource] = []
    for label_value in sorted(
        int(item) for item in np.unique(labels) if item > 0
    ):
        crop = _segment_crop(segment_windows, aperture_windows, label_value)
        row = build_segment_row(
            residual[crop],
            None if position_signal is None else position_signal[crop],
            valid[crop],
            centroid_labels[crop],
            measurement_labels[crop],
            background[crop],
            celestial_wcs,
            label_value=label_value,
            window=SegmentWindow(
                origin_yx=(crop[0].start, crop[1].start),
                plane_shape_yx=residual.shape,
            ),
            beam_area_pixels=beam_area_pixels,
            denoised_position_maximum_peak_to_mean_ratio=(
                denoised_position_maximum_peak_to_mean_ratio
            ),
            position_diagnostics=position_diagnostics,
        )
        if row is not None:
            output.append(row)
    return tuple(output)

build_hebog_segment_moment_catalogue(image_jy_per_beam, background_jy_per_beam, valid_pixels, component_labels, header, *, beam_major_fwhm_pixels, beam_minor_fwhm_pixels, measurement_aperture_radius_beams=4.0, position_signal_jy_per_beam=None, denoised_position_maximum_peak_to_mean_ratio=3.0, aperture_tie_policy='nearest-support', position_labels=None, position_diagnostics=None)

Publish exact-support moment shapes without changing photometry.

Source code in src/hebog/science/segment_rows.py

def build_hebog_segment_moment_catalogue(  # noqa: PLR0913
    image_jy_per_beam: npt.ArrayLike,
    background_jy_per_beam: npt.ArrayLike,
    valid_pixels: npt.ArrayLike,
    component_labels: npt.ArrayLike,
    header: fits.Header,
    *,
    beam_major_fwhm_pixels: float,
    beam_minor_fwhm_pixels: float,
    measurement_aperture_radius_beams: float = 4.0,
    position_signal_jy_per_beam: npt.ArrayLike | None = None,
    denoised_position_maximum_peak_to_mean_ratio: float = 3.0,
    aperture_tie_policy: Literal[
        "nearest-support", "canonical-source"
    ] = "nearest-support",
    position_labels: npt.ArrayLike | None = None,
    position_diagnostics: dict[int, SourcePositionDiagnostics] | None = None,
) -> tuple[CatalogueSource, ...]:
    """Publish exact-support moment shapes without changing photometry."""
    sources = build_hebog_segment_catalogue(
        image_jy_per_beam,
        background_jy_per_beam,
        valid_pixels,
        component_labels,
        header,
        beam_major_fwhm_pixels=beam_major_fwhm_pixels,
        beam_minor_fwhm_pixels=beam_minor_fwhm_pixels,
        measurement_aperture_radius_beams=measurement_aperture_radius_beams,
        position_signal_jy_per_beam=position_signal_jy_per_beam,
        denoised_position_maximum_peak_to_mean_ratio=(
            denoised_position_maximum_peak_to_mean_ratio
        ),
        aperture_tie_policy=aperture_tie_policy,
        position_labels=position_labels,
        position_diagnostics=position_diagnostics,
    )
    residual, valid, labels = _validated_hebog_segment_planes(
        image_jy_per_beam,
        background_jy_per_beam,
        valid_pixels,
        component_labels,
    )
    celestial_wcs = WCS(header, relax=True).celestial
    try:
        beam = RestoringBeam(
            major_fwhm_degrees=cast(float, header["BMAJ"]),
            minor_fwhm_degrees=cast(float, header["BMIN"]),
            position_angle_degrees=cast(float, header["BPA"]),
        )
    except (KeyError, TypeError, ValueError):
        return tuple(
            replace(
                source,
                quality_flags=(
                    "segment-moment-equivalent-shape",
                    "shape-unavailable",
                ),
            )
            for source in sources
        )
    by_identifier = {source.identifier: source for source in sources}
    segment_windows = label_windows(labels)
    output: list[CatalogueSource] = []
    for label_value in sorted(
        int(item) for item in np.unique(labels) if item > 0
    ):
        identifier = f"hebog-segment-{label_value}"
        source = by_identifier.get(identifier)
        if source is None:
            continue
        crop = _segment_crop(segment_windows, segment_windows, label_value)
        support = (labels[crop] == label_value) & valid[crop]
        shape_fields = _moment_shape_fields(
            residual[crop],
            support,
            celestial_wcs,
            beam,
            SegmentWindow(
                origin_yx=(crop[0].start, crop[1].start),
                plane_shape_yx=residual.shape,
            ),
        )
        shape_fields["quality_flags"] = tuple(
            sorted(
                {
                    *source.quality_flags,
                    *cast(tuple[str, ...], shape_fields["quality_flags"]),
                }
            )
        )
        output.append(
            replace(
                source,
                **shape_fields,  # type: ignore[arg-type]
            )
        )
    return tuple(output)

build_segment_row(residual_window, position_signal_window, valid_window, centroid_window, aperture_window, background_window, celestial_wcs, *, label_value, window, beam_area_pixels, denoised_position_maximum_peak_to_mean_ratio, position_diagnostics=None)

Measure and place one segment's row, one segment at a time.

This is the readable reference for the batched path in :mod:hebog.stages.catalogue_rows, which converts every centroid of a batch together.

Source code in src/hebog/science/segment_rows.py

def build_segment_row(  # noqa: PLR0913, PLR0917
    residual_window: npt.NDArray[np.float64],
    position_signal_window: npt.NDArray[np.float64] | None,
    valid_window: npt.NDArray[np.bool_],
    centroid_window: npt.NDArray[np.int64],
    aperture_window: npt.NDArray[np.int32],
    background_window: npt.NDArray[np.float64],
    celestial_wcs: WCS,
    *,
    label_value: int,
    window: SegmentWindow,
    beam_area_pixels: float,
    denoised_position_maximum_peak_to_mean_ratio: float,
    position_diagnostics: dict[int, SourcePositionDiagnostics] | None = None,
) -> CatalogueSource | None:
    """Measure and place one segment's row, one segment at a time.

    This is the readable reference for the batched path in
    :mod:`hebog.stages.catalogue_rows`, which converts every centroid of a
    batch together.
    """
    measurement = measure_segment_row(
        residual_window,
        position_signal_window,
        valid_window,
        centroid_window,
        aperture_window,
        background_window,
        label_value=label_value,
        window=window,
        beam_area_pixels=beam_area_pixels,
        denoised_position_maximum_peak_to_mean_ratio=(
            denoised_position_maximum_peak_to_mean_ratio
        ),
        position_diagnostics=position_diagnostics,
    )
    if measurement is None:
        return None
    position = cast(
        Any, celestial_wcs.pixel_to_world(*measurement.centroid_xy)
    ).icrs
    return segment_row_at(
        measurement,
        right_ascension_degrees=float(position.ra.deg),
        declination_degrees=float(position.dec.deg),
    )

catalogue_ellipse(shape)

Translate one validated Gaussian-like shape into catalogue units.

Source code in src/hebog/science/segment_rows.py

def catalogue_ellipse(shape: GaussianShape) -> CatalogueEllipse:
    """Translate one validated Gaussian-like shape into catalogue units."""
    return CatalogueEllipse(
        major_fwhm_degrees=shape.major_fwhm_degrees,
        minor_fwhm_degrees=shape.minor_fwhm_degrees,
        position_angle_degrees=shape.position_angle_degrees,
        major_fwhm_error_degrees=shape.major_fwhm_error_degrees,
        minor_fwhm_error_degrees=shape.minor_fwhm_error_degrees,
        position_angle_error_degrees=shape.position_angle_error_degrees,
    )

measure_segment_row(residual_window, position_signal_window, valid_window, centroid_window, aperture_window, background_window, *, label_value, window, beam_area_pixels, denoised_position_maximum_peak_to_mean_ratio, position_diagnostics=None)

Measure one segment's catalogue row inside its own window.

Every array covers that segment's exact support joined with its expanded aperture, which is the window :func:_segment_crop returns, so this work costs the segment's own pixels rather than the plane around it. The row is :func:measure_segment_row_pixels over the pixels of the window the segment owns or holds in its aperture.

Source code in src/hebog/science/segment_rows.py

def measure_segment_row(  # noqa: PLR0913, PLR0917
    residual_window: npt.NDArray[np.float64],
    position_signal_window: npt.NDArray[np.float64] | None,
    valid_window: npt.NDArray[np.bool_],
    centroid_window: npt.NDArray[np.int64],
    aperture_window: npt.NDArray[np.int32],
    background_window: npt.NDArray[np.float64],
    *,
    label_value: int,
    window: SegmentWindow,
    beam_area_pixels: float,
    denoised_position_maximum_peak_to_mean_ratio: float,
    position_diagnostics: dict[int, SourcePositionDiagnostics] | None = None,
) -> SegmentRowMeasurement | None:
    """Measure one segment's catalogue row inside its own window.

    Every array covers that segment's exact support joined with its expanded
    aperture, which is the window :func:`_segment_crop` returns, so this work
    costs the segment's own pixels rather than the plane around it. The row
    is :func:`measure_segment_row_pixels` over the pixels of the window the
    segment owns or holds in its aperture.
    """
    owned = centroid_window == label_value
    aperture = aperture_window == label_value
    member = owned | aperture
    local_y, local_x = np.nonzero(member)
    return measure_segment_row_pixels(
        SegmentPixels(
            y=local_y + window.origin_yx[0],
            x=local_x + window.origin_yx[1],
            residual=residual_window[member],
            position_signal=None
            if position_signal_window is None
            else position_signal_window[member],
            background=np.asarray(background_window)[member],
            valid=valid_window[member],
            owned=owned[member],
            aperture=aperture[member],
        ),
        label_value=label_value,
        plane_shape_yx=window.plane_shape_yx,
        beam_area_pixels=beam_area_pixels,
        denoised_position_maximum_peak_to_mean_ratio=(
            denoised_position_maximum_peak_to_mean_ratio
        ),
        position_diagnostics=position_diagnostics,
    )

measure_segment_row_pixels(pixels, *, label_value, plane_shape_yx, beam_area_pixels, denoised_position_maximum_peak_to_mean_ratio, position_diagnostics=None)

Measure one segment's catalogue row from its own pixels.

This is the half that reads pixels, and it needs no WCS. Absence means the segment has no measurable row, not an error.

Source code in src/hebog/science/segment_rows.py

def measure_segment_row_pixels(  # noqa: PLR0913
    pixels: SegmentPixels,
    *,
    label_value: int,
    plane_shape_yx: tuple[int, int],
    beam_area_pixels: float,
    denoised_position_maximum_peak_to_mean_ratio: float,
    position_diagnostics: dict[int, SourcePositionDiagnostics] | None = None,
) -> SegmentRowMeasurement | None:
    """Measure one segment's catalogue row from its own pixels.

    This is the half that reads pixels, and it needs no WCS. Absence means
    the segment has no measurable row, not an error.
    """
    support = pixels.owned & pixels.valid & np.isfinite(pixels.residual)
    residual = pixels.residual[support]
    estimate = _segment_position(
        pixels.y[support],
        pixels.x[support],
        residual,
        None
        if pixels.position_signal is None
        else pixels.position_signal[support],
        maximum_peak_to_mean_ratio=(
            denoised_position_maximum_peak_to_mean_ratio
        ),
        plane_shape_yx=plane_shape_yx,
    )
    integrated_weight = float(
        np.sum(pixels.residual[pixels.aperture], dtype=np.float64)
    )
    # Emission this segment owns that no fitted model accounts for. A source
    # whose flux is the sum of its fits would otherwise lose it, because the
    # fits describe the compact parts and nothing describes the rest. This is
    # the residual after the models, not the residual outside their support:
    # a fit's support is the region it measured over, which covers emission
    # its model does not explain.
    if position_diagnostics is not None:
        position_diagnostics[label_value] = _position_attribution(
            pixels,
            support,
            estimate,
            integrated_weight / beam_area_pixels,
            plane_shape_yx=plane_shape_yx,
        )
    if not estimate.available or estimate.centroid_xy is None:
        return None
    quality_flags: tuple[str, ...] = ()
    if not np.isfinite(integrated_weight) or integrated_weight <= 0.0:
        integrated_weight = float(
            np.sum(residual[residual > 0.0], dtype=np.float64)
        )
        quality_flags = (
            "association-aperture-nonpositive",
            "exact-owner-positive-residual-flux",
        )
    if not np.isfinite(integrated_weight) or integrated_weight <= 0.0:
        return None
    return SegmentRowMeasurement(
        label_value=label_value,
        centroid_xy=estimate.centroid_xy,
        peak_flux_jy_per_beam=float(np.max(residual)),
        integrated_flux_jy=integrated_weight / beam_area_pixels,
        quality_flags=tuple(
            sorted(
                {
                    *quality_flags,
                    f"position-{estimate.weighting}",
                    "positive-exact-owner-flux"
                    if "exact-owner-positive-residual-flux" in quality_flags
                    else "source-owned-signed-aperture",
                    "aperture-flux-uncertainty-unavailable",
                    "position-uncertainty-unavailable",
                }
            )
        ),
    )

moment_shape_fields_at(moment, *, transform, beam_icrs)

Return catalogue fields for one moment under its own local geometry.

transform and beam_icrs belong to this moment's centroid. A moment the local geometry cannot describe is reported as unavailable, exactly as an unmeasurable one is.

Source code in src/hebog/science/segment_rows.py

def moment_shape_fields_at(
    moment: tuple[tuple[float, float], npt.NDArray[np.float64]],
    *,
    transform: LocalTangentPlaneTransform,
    beam_icrs: RestoringBeam,
) -> dict[str, object]:
    """Return catalogue fields for one moment under its own local geometry.

    ``transform`` and ``beam_icrs`` belong to this moment's centroid. A
    moment the local geometry cannot describe is reported as unavailable,
    exactly as an unmeasurable one is.
    """
    _, covariance = moment
    try:
        fitted = moment_equivalent_gaussian_shape(covariance, transform)
        deconvolution = deconvolve_gaussian_shapes(
            fitted,
            beam_icrs,
            relative_tolerance=1e-10,
        )
    except (TypeError, ValueError, np.linalg.LinAlgError):
        return unavailable_moment_shape_fields()
    flags = {_MOMENT_SHAPE_PROVENANCE}
    if deconvolution.status in {"major-axis-only", "unresolved"}:
        flags.update(deconvolution.quality_flags)
    return {
        "fitted_shape": catalogue_ellipse(fitted),
        "deconvolved_shape": (
            catalogue_ellipse(deconvolution.shape)
            if deconvolution.shape is not None
            else None
        ),
        "deconvolved_major_fwhm_degrees": (
            deconvolution.major_axis_fwhm_degrees
        ),
        "deconvolution_status": deconvolution.status,
        "quality_flags": tuple(sorted(flags)),
    }

segment_local_rms(rms_jy_per_beam, owner_window, *, label_value)

Return the median local noise over one segment's exact owned support.

owner_window is the segment's seeded ownership, not its expanded aperture: the noise a catalogue row quotes describes the pixels the segment itself owns. Both arrays cover the same window. A pixel whose estimate is not finite and positive carries no usable noise, and None means this segment owns no pixel that does.

Examples:

>>> import numpy as np
>>> rms = np.array([[1.0, 3.0], [np.nan, 5.0]])
>>> owners = np.array([[2, 2], [2, 0]])
>>> segment_local_rms(rms, owners, label_value=2)
2.0
>>> segment_local_rms(rms, owners, label_value=1) is None
True

Source code in src/hebog/science/segment_rows.py

def segment_local_rms(
    rms_jy_per_beam: npt.NDArray[np.float64],
    owner_window: npt.NDArray[np.integer[Any]],
    *,
    label_value: int,
) -> float | None:
    """Return the median local noise over one segment's exact owned support.

    ``owner_window`` is the segment's seeded ownership, not its expanded
    aperture: the noise a catalogue row quotes describes the pixels the
    segment itself owns. Both arrays cover the same window. A pixel whose
    estimate is not finite and positive carries no usable noise, and
    ``None`` means this segment owns no pixel that does.

    Examples:
        >>> import numpy as np
        >>> rms = np.array([[1.0, 3.0], [np.nan, 5.0]])
        >>> owners = np.array([[2, 2], [2, 0]])
        >>> segment_local_rms(rms, owners, label_value=2)
        2.0
        >>> segment_local_rms(rms, owners, label_value=1) is None
        True
    """
    return segment_local_rms_pixels(
        rms_jy_per_beam[owner_window == label_value]
    )

segment_local_rms_pixels(rms_jy_per_beam)

Return the median usable noise over one segment's owned pixels.

The array holds the noise at the pixels the segment owns, so a segment too wide to read at once quotes the noise its window would give.

Source code in src/hebog/science/segment_rows.py

def segment_local_rms_pixels(
    rms_jy_per_beam: npt.NDArray[np.float64],
) -> float | None:
    """Return the median usable noise over one segment's owned pixels.

    The array holds the noise at the pixels the segment owns, so a segment
    too wide to read at once quotes the noise its window would give.
    """
    usable = np.isfinite(rms_jy_per_beam) & (rms_jy_per_beam > 0.0)
    if not bool(np.any(usable)):
        return None
    return float(np.median(rms_jy_per_beam[usable]))

segment_moment(residual_jy_per_beam, support, window=None)

Return one segment's weighted centroid and covariance, or None.

This is the half of a moment shape that reads pixels. Its centroid is where the shape's local geometry belongs, so a caller measuring many segments takes every centroid first and transforms them together; see :func:moment_shape_fields_at.

Source code in src/hebog/science/segment_rows.py

def segment_moment(
    residual_jy_per_beam: npt.NDArray[np.float64],
    support: npt.NDArray[np.bool_],
    window: SegmentWindow | None = None,
) -> tuple[tuple[float, float], npt.NDArray[np.float64]] | None:
    """Return one segment's weighted centroid and covariance, or ``None``.

    This is the half of a moment shape that reads pixels. Its centroid is
    where the shape's local geometry belongs, so a caller measuring many
    segments takes every centroid first and transforms them together; see
    :func:`moment_shape_fields_at`.
    """
    return _segment_pixel_moment_covariance(
        residual_jy_per_beam,
        support,
        window=window,
    )

segment_moment_pixels(y_pixels, x_pixels, residual_jy_per_beam)

Return a positive centroid and covariance from support pixels alone.

The arrays hold the segment's measured support in raster order, with coordinates in the image's pixel frame, so a segment too wide to read at once gives the moment its window would give, bit for bit.

Source code in src/hebog/science/segment_rows.py

def segment_moment_pixels(
    y_pixels: npt.NDArray[np.int64],
    x_pixels: npt.NDArray[np.int64],
    residual_jy_per_beam: npt.NDArray[np.float64],
) -> tuple[tuple[float, float], npt.NDArray[np.float64]] | None:
    """Return a positive centroid and covariance from support pixels alone.

    The arrays hold the segment's measured support in raster order, with
    coordinates in the image's pixel frame, so a segment too wide to read at
    once gives the moment its window would give, bit for bit.
    """
    positive = np.isfinite(residual_jy_per_beam) & (residual_jy_per_beam > 0.0)
    if int(np.count_nonzero(positive)) < _MINIMUM_MOMENT_PIXELS:
        return None
    weights = residual_jy_per_beam[positive]
    weight = float(np.sum(weights, dtype=np.float64))
    if not np.isfinite(weight) or weight <= 0.0:
        return None
    y_positive = y_pixels[positive]
    x_positive = x_pixels[positive]
    centroid_x = float(np.sum(x_positive * weights, dtype=np.float64) / weight)
    centroid_y = float(np.sum(y_positive * weights, dtype=np.float64) / weight)
    delta_x = x_positive - centroid_x
    delta_y = y_positive - centroid_y
    covariance = np.asarray(
        (
            (
                np.sum(weights * delta_x * delta_x, dtype=np.float64) / weight,
                np.sum(weights * delta_x * delta_y, dtype=np.float64) / weight,
            ),
            (
                np.sum(weights * delta_x * delta_y, dtype=np.float64) / weight,
                np.sum(weights * delta_y * delta_y, dtype=np.float64) / weight,
            ),
        ),
        dtype=np.float64,
    )
    if (
        not np.all(np.isfinite(covariance))
        or float(np.linalg.det(covariance)) <= 0.0
    ):
        return None
    return (centroid_x, centroid_y), covariance

segment_row_at(measurement, *, right_ascension_degrees, declination_degrees)

Return one measured row at the sky position its centroid earns.

Source code in src/hebog/science/segment_rows.py

def segment_row_at(
    measurement: SegmentRowMeasurement,
    *,
    right_ascension_degrees: float,
    declination_degrees: float,
) -> CatalogueSource:
    """Return one measured row at the sky position its centroid earns."""
    identifier = f"hebog-segment-{measurement.label_value}"
    return CatalogueSource(
        identifier=identifier,
        right_ascension_degrees=right_ascension_degrees,
        declination_degrees=declination_degrees,
        peak_flux_jy_per_beam=measurement.peak_flux_jy_per_beam,
        integrated_flux_jy=measurement.integrated_flux_jy,
        association_integrated_flux_jy=measurement.integrated_flux_jy,
        deconvolution_status="unavailable",
        island_identifier=identifier,
        component_count=1,
        quality_flags=measurement.quality_flags,
    )

unavailable_moment_shape_fields()

Return the fields of a segment whose shape cannot be measured.

Source code in src/hebog/science/segment_rows.py

def unavailable_moment_shape_fields() -> dict[str, object]:
    """Return the fields of a segment whose shape cannot be measured."""
    return {
        "fitted_shape": None,
        "deconvolved_shape": None,
        "deconvolved_major_fwhm_degrees": None,
        "deconvolution_status": "unavailable",
        "quality_flags": (_MOMENT_SHAPE_PROVENANCE, "shape-unavailable"),
    }

hebog.science.catalogues

Reconstructed source and component catalogues of the composition.

AssociatedMomentCatalogues dataclass

Binding source rows plus retained immutable component diagnostics.

Source code in src/hebog/science/catalogues.py

@dataclass(frozen=True, slots=True)
class AssociatedMomentCatalogues:
    """Binding source rows plus retained immutable component diagnostics."""

    component_catalogue: tuple[CatalogueSource, ...]
    source_catalogue: tuple[CatalogueSource, ...]
    association: SourceAssociationResult
    measurement_dispositions: tuple[MeasurementDisposition, ...] = ()

build_hebog_reconstructed_source_catalogues(header, *, component_measurements=None, association, hierarchy, component_rows, source_rows, source_positions)

Assemble each common-parent catalogue source exactly once.

Direct seed labels define hierarchy identity. Recovered measurement labels define masks and apertures. Immutable component measurements remain diagnostic. Binding source rows are measured from a source-label plane before aperture expansion, so every observable pixel belongs to at most one source aperture.

This assembles records only: every row was measured by the stage that held the window it was measured in, and every plane the rows describe was checked by the core that wrote it — direct ownership against the measurement plane and the validity beside it, source ownership against the memberships the cores were given.

Source code in src/hebog/science/catalogues.py

def build_hebog_reconstructed_source_catalogues(  # noqa: PLR0913
    header: fits.Header,
    *,
    component_measurements: ComponentMeasurements | None = None,
    association: SourceAssociationResult,
    hierarchy: SourceAssociationResult,
    component_rows: tuple[CatalogueSource, ...],
    source_rows: tuple[CatalogueSource, ...],
    source_positions: Mapping[int, SourcePositionDiagnostics],
) -> AssociatedMomentCatalogues:
    """Assemble each common-parent catalogue source exactly once.

    Direct seed labels define hierarchy identity. Recovered measurement labels
    define masks and apertures. Immutable component measurements remain
    diagnostic. Binding source rows are measured from a source-label plane
    before aperture expansion, so every observable pixel belongs to at most one
    source aperture.

    This assembles records only: every row was measured by the stage that held
    the window it was measured in, and every plane the rows describe was
    checked by the core that wrote it — direct ownership against the
    measurement plane and the validity beside it, source ownership against the
    memberships the cores were given.
    """
    component_sources, _ = _apply_component_measurements(
        component_rows,
        component_measurements,
        header,
    )
    stable_components = _stable_component_catalogue(
        component_sources,
        association,
    )
    membership_by_label = dict(enumerate(association.memberships, start=1))
    output = _reconstructed_source_rows(
        source_rows,
        stable_components,
        membership_by_label,
        association,
        require_signed_aperture=component_measurements is not None,
    )
    if component_measurements is None and len(output) != len(
        association.memberships
    ):
        raise ValueError(
            "reconstructed source has no measurable catalogue row"
        )
    return AssociatedMomentCatalogues(
        component_catalogue=stable_components
        if component_measurements is None
        else tuple(
            row
            for row in stable_components
            if "original-pixel-gaussian-model" in row.quality_flags
        ),
        source_catalogue=output,
        association=association,
        measurement_dispositions=()
        if component_measurements is None
        else (
            _measurement_dispositions(
                association,
                component_measurements,
                tuple(output),
                {
                    membership.source_id: source_positions[label]
                    for label, membership in membership_by_label.items()
                    if label in source_positions
                },
                hierarchy,
            )
        ),
    )

local_rms_by_object_id(association, *, component_local_rms, source_local_rms)

Name each measured owner's local noise by its catalogue identity.

The row passes measure noise per label, because that is what the plane they read carries: components by their measurement label and sources by their canonical source label, which is the membership's position here. An owner whose support carries no usable estimate is absent, exactly as it is in the mappings this joins.

Source code in src/hebog/science/catalogues.py

def local_rms_by_object_id(
    association: SourceAssociationResult,
    *,
    component_local_rms: Mapping[int, float],
    source_local_rms: Mapping[int, float],
) -> dict[str, float]:
    """Name each measured owner's local noise by its catalogue identity.

    The row passes measure noise per label, because that is what the plane
    they read carries: components by their measurement label and sources by
    their canonical source label, which is the membership's position here.
    An owner whose support carries no usable estimate is absent, exactly as
    it is in the mappings this joins.
    """
    by_object_id = {
        record.component_id: component_local_rms[record.label_value]
        for record in association.components
        if record.label_value in component_local_rms
    }
    by_object_id.update(
        (membership.source_id, source_local_rms[source_label])
        for source_label, membership in enumerate(
            association.memberships, start=1
        )
        if source_label in source_local_rms
    )
    return by_object_id

source_label_by_owner(association)

Map each immutable component owner to its canonical source label.

Source code in src/hebog/science/catalogues.py

def source_label_by_owner(
    association: SourceAssociationResult,
) -> dict[int, int]:
    """Map each immutable component owner to its canonical source label."""
    records_by_id = {
        item.component_id: item for item in association.components
    }
    return {
        records_by_id[component_id].label_value: source_label
        for source_label, membership in enumerate(
            association.memberships, start=1
        )
        for component_id in membership.component_ids
    }

hebog.stages.catalogue_rows

Scheduler-facing rounds that measure one catalogue row per segment.

ADR-008's source-row round measures each segment inside the window holding its exact support and its expanded aperture. Only the aperture expansion crosses a segment's own bounds, and it reaches no further than the reviewed radius, so the cores write it under that halo and every row after it is bounded by the segment it describes. No round reduces anything globally.

SegmentRowStageConfig dataclass

Reviewed row policy and the bounded task limits.

Source code in src/hebog/stages/catalogue_rows.py

@dataclass(frozen=True, slots=True)
class SegmentRowStageConfig:
    """Reviewed row policy and the bounded task limits."""

    label_product_name: str
    centroid_product_name: str
    aperture_radius_pixels: int
    aperture_tie_policy: Literal["nearest-support", "canonical-source"]
    beam_area_pixels: float
    denoised_position_maximum_peak_to_mean_ratio: float
    with_position_diagnostics: bool
    maximum_tiles_per_batch: int
    maximum_objects_per_batch: int
    maximum_batch_read_pixels: int

    def __post_init__(self) -> None:
        """Reject an unbounded task before any product is initialized."""
        for name, value in (
            ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
            ("maximum_objects_per_batch", self.maximum_objects_per_batch),
            ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
        ):
            if (
                isinstance(value, bool)
                or not isinstance(value, Integral)
                or value < 1
            ):
                raise ValueError(f"{name} must be a positive integer")
        if (
            isinstance(self.aperture_radius_pixels, bool)
            or not isinstance(self.aperture_radius_pixels, Integral)
            or self.aperture_radius_pixels < 0
        ):
            raise ValueError(
                "aperture_radius_pixels must be a non-negative integer"
            )
        if self.aperture_tie_policy not in {
            "nearest-support",
            "canonical-source",
        }:
            raise ValueError("segment aperture tie policy is unsupported")

__post_init__()

Reject an unbounded task before any product is initialized.

Source code in src/hebog/stages/catalogue_rows.py

def __post_init__(self) -> None:
    """Reject an unbounded task before any product is initialized."""
    for name, value in (
        ("maximum_tiles_per_batch", self.maximum_tiles_per_batch),
        ("maximum_objects_per_batch", self.maximum_objects_per_batch),
        ("maximum_batch_read_pixels", self.maximum_batch_read_pixels),
    ):
        if (
            isinstance(value, bool)
            or not isinstance(value, Integral)
            or value < 1
        ):
            raise ValueError(f"{name} must be a positive integer")
    if (
        isinstance(self.aperture_radius_pixels, bool)
        or not isinstance(self.aperture_radius_pixels, Integral)
        or self.aperture_radius_pixels < 0
    ):
        raise ValueError(
            "aperture_radius_pixels must be a non-negative integer"
        )
    if self.aperture_tie_policy not in {
        "nearest-support",
        "canonical-source",
    }:
        raise ValueError("segment aperture tie policy is unsupported")

SegmentRowStageResult dataclass

Published apertures, the rows they measure, and scalar evidence.

local_rms_by_label holds the median local noise over each segment's exact owned support, for every segment the rounds observed rather than only the measurable ones: a catalogue row that no row measured can still be published from a fitted model, and it quotes the same noise.

Source code in src/hebog/stages/catalogue_rows.py

@dataclass(frozen=True, slots=True)
class SegmentRowStageResult:
    """Published apertures, the rows they measure, and scalar evidence.

    ``local_rms_by_label`` holds the median local noise over each segment's
    exact owned support, for every segment the rounds observed rather than
    only the measurable ones: a catalogue row that no row measured can still
    be published from a fitted model, and it quotes the same noise.
    """

    generation: ProductGenerationManifest
    rows: tuple[CatalogueSource, ...]
    local_rms_by_label: Mapping[int, float]
    position_diagnostics: Mapping[int, SourcePositionDiagnostics]
    segment_count: int
    measured_segment_count: int
    wide_segment_count: int
    partition_count: int
    executor_task_count: int
    maximum_graph_width: int
    maximum_segment_read_pixels: int

run_segment_row_stage(source, background_rms_source, detection_source, label_source, centroid_source, position_source, manifest, *, config, wcs_header_text, beam, executor, sink)

Measure one catalogue row per segment, inside the window holding it.

Three rounds and one global reduction fewer than every other object round: the cores write the expanded apertures under the reviewed radius, they then observe the bounds each segment and aperture occupies, and one task per batch of segments measures their rows, moment shapes and the local noise over the support they own.

wcs_header_text is the caller's own header as :meth:astropy.io.fits.Header.tostring writes it, not a WCS; see :func:~hebog.algorithms.astrometry.celestial_wcs_from_header_text.

A segment whose window exceeds maximum_batch_read_pixels is never read whole. The cores holding it return the pixels its row visits, and the row is measured from them restored to raster order, which is the row its window would give.

Source code in src/hebog/stages/catalogue_rows.py

def run_segment_row_stage(  # noqa: PLR0913, PLR0917
    source: WindowReadable,
    background_rms_source: CompletedProductSource,
    detection_source: CompletedProductSource,
    label_source: CompletedProductSource,
    centroid_source: CompletedProductSource,
    position_source: CompletedProductSource,
    manifest: PartitionManifest,
    *,
    config: SegmentRowStageConfig,
    wcs_header_text: str,
    beam: RestoringBeam,
    executor: Executor,
    sink: ZarrProductSink,
) -> SegmentRowStageResult:
    """Measure one catalogue row per segment, inside the window holding it.

    Three rounds and one global reduction fewer than every other object
    round: the cores write the expanded apertures under the reviewed radius,
    they then observe the bounds each segment and aperture occupies, and one
    task per batch of segments measures their rows, moment shapes and the
    local noise over the support they own.

    ``wcs_header_text`` is the caller's own header as
    :meth:`astropy.io.fits.Header.tostring` writes it, not a ``WCS``; see
    :func:`~hebog.algorithms.astrometry.celestial_wcs_from_header_text`.

    A segment whose window exceeds ``maximum_batch_read_pixels`` is never
    read whole. The cores holding it return the pixels its row visits, and
    the row is measured from them restored to raster order, which is the row
    its window would give.
    """
    if sink.manifest != manifest:
        raise ValueError("segment row sink must use the stage manifest")
    _require_row_inputs(
        background_rms_source,
        detection_source,
        label_source,
        centroid_source,
        position_source,
        manifest,
        config,
    )
    core_batches = _core_batches(
        manifest.tiles,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    tile_reads = tuple(partition.read_bounds for partition in manifest.tiles)
    sink.initialize_product(
        product_name="aperture-labels",
        dtype=np.dtype("<i4"),
    )
    aperture_results = tuple(
        executor.map_batches(
            partial(
                _publish_apertures,
                source=source,
                background_rms_source=background_rms_source,
                detection_source=detection_source,
                label_source=label_source,
                config=config,
                image_shape_yx=manifest.image_shape_yx,
                sink=sink,
            ),
            core_batches,
            requirement=read_requirement(
                _APERTURE_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not aperture_results:
        raise ValueError("executor returned no segment aperture results")
    generation = sink.publish_generation(
        product_names=_PRODUCT_NAMES,
        chunks=(
            chunk
            for result in aperture_results
            for chunk in result.product_chunks
        ),
    )
    window_results = tuple(
        executor.map_batches(
            partial(
                _scan_windows,
                label_source=label_source,
                aperture_source=sink,
                label_product_name=config.label_product_name,
            ),
            core_batches,
            requirement=read_requirement(
                _WINDOW_SCAN_BYTES_PER_READ_PIXEL,
                tile_reads,
                chunk_shape_yx=manifest.tile_core_shape_yx,
            ),
        )
    )
    if not window_results:
        raise ValueError("executor returned no segment window results")
    segments = _segments(window_results)
    budget = config.maximum_batch_read_pixels
    row_batches = _row_batches(
        tuple(
            segment
            for segment in segments
            if read_pixels(segment.bounds) <= budget
        ),
        core_shape_yx=manifest.tile_core_shape_yx,
        maximum_objects_per_batch=config.maximum_objects_per_batch,
        maximum_batch_read_pixels=budget,
    )
    row_results = map_round(
        executor,
        partial(
            _row_batch,
            source=source,
            background_rms_source=background_rms_source,
            detection_source=detection_source,
            label_source=label_source,
            centroid_source=centroid_source,
            aperture_source=sink,
            position_source=position_source,
            config=config,
            wcs_header_text=wcs_header_text,
            beam=beam,
            image_shape_yx=manifest.image_shape_yx,
        ),
        row_batches,
        round_name="segment row",
        requirement=read_requirement(
            _ROW_BYTES_PER_READ_PIXEL,
            (batch.read_bounds for batch in row_batches),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    wide_segments = tuple(
        segment for segment in segments if read_pixels(segment.bounds) > budget
    )
    wide_batches = _wide_segment_batches(
        tuple(
            segment
            for segment in wide_segments
            if read_pixels(segment.bounds) > budget
        ),
        window_results,
        manifest,
        maximum_tiles_per_batch=config.maximum_tiles_per_batch,
    )
    wide_results = map_round(
        executor,
        partial(
            _gather_wide_segments,
            source=source,
            background_rms_source=background_rms_source,
            detection_source=detection_source,
            label_source=label_source,
            centroid_source=centroid_source,
            aperture_source=sink,
            position_source=position_source,
            config=config,
            image_width=manifest.image_shape_yx[1],
        ),
        wide_batches,
        round_name="wide segment",
        requirement=read_requirement(
            _WIDE_SEGMENT_BYTES_PER_READ_PIXEL,
            (
                core.partition.read_bounds
                for batch in wide_batches
                for core in batch.cores
            ),
            chunk_shape_yx=manifest.tile_core_shape_yx,
        ),
    )
    row_results = (
        *row_results,
        _wide_rows(
            wide_batches,
            wide_results,
            config=config,
            image_shape_yx=manifest.image_shape_yx,
            wcs_header_text=wcs_header_text,
            beam=beam,
        ),
    )
    rows = tuple(
        row
        for _, row in sorted(
            (item for result in row_results for item in result.rows),
            key=lambda item: item[0],
        )
    )
    return SegmentRowStageResult(
        generation=generation,
        rows=rows,
        local_rms_by_label=dict(
            sorted(item for result in row_results for item in result.local_rms)
        ),
        position_diagnostics=dict(
            sorted(
                item for result in row_results for item in result.diagnostics
            )
        ),
        segment_count=len(segments),
        measured_segment_count=len(rows),
        wide_segment_count=len(wide_segments),
        partition_count=len(manifest.tiles),
        executor_task_count=(
            2 * len(core_batches) + len(row_batches) + len(wide_batches)
        ),
        maximum_graph_width=max(
            len(core_batches), len(row_batches), len(wide_batches)
        ),
        maximum_segment_read_pixels=max(
            (result.maximum_segment_read_pixels for result in row_results),
            default=0,
        ),
    )

segment_row_product_names()

Return the canonical published segment-row product set.

Source code in src/hebog/stages/catalogue_rows.py

def segment_row_product_names() -> tuple[str, ...]:
    """Return the canonical published segment-row product set."""
    return _PRODUCT_NAMES

hebog.science.continuum

Current continuum detection and publication-support composition.

compact_deblend_config(config)

Return the reviewed deblending policy at the caller's threshold.

A parent is deblended only where its peak clears the caller's detection threshold, so the reviewed minimum is lowered to just below it rather than to a second, independent threshold.

Source code in src/hebog/science/continuum.py

def compact_deblend_config(
    config: SourceFinderConfig,
) -> CompactDeblendConfig:
    """Return the reviewed deblending policy at the caller's threshold.

    A parent is deblended only where its peak clears the caller's detection
    threshold, so the reviewed minimum is lowered to just below it rather
    than to a second, independent threshold.
    """
    return replace(
        source_finder_configs()[1],
        minimum_peak_signal_to_noise=float(
            np.nextafter(config.detection_threshold_sigma, -np.inf)
        ),
    )

residual_detection_config(review)

Return the reviewed thresholds the tiled detection pass runs under.

Source code in src/hebog/science/continuum.py

def residual_detection_config(
    review: ContinuumScienceProfile,
) -> ResidualMultiscaleDetectionConfig:
    """Return the reviewed thresholds the tiled detection pass runs under."""
    return ResidualMultiscaleDetectionConfig(
        detection_threshold_sigma=review.matrix.detection_sigma,
        island_threshold_sigma=review.matrix.island_sigma,
        minimum_scale_support_fraction=(
            review.matrix.support_fraction_bounds[0]
        ),
        minimum_island_area_beams=review.corrections.minimum_island_area_beams,
    )

retained_scale_detections(multiscale)

Describe the retained per-scale features from published records.

The detection pass reconciled these features and published their labels, so this step names them without labelling a plane again, and without holding one: the cores that wrote the scale masks already required them to lie inside the domain their own estimate covers.

Source code in src/hebog/science/continuum.py

def retained_scale_detections(
    multiscale: TiledMultiscaleDetection,
) -> tuple[ScaleDetectionRecords, ...]:
    """Describe the retained per-scale features from published records.

    The detection pass reconciled these features and published their labels,
    so this step names them without labelling a plane again, and without
    holding one: the cores that wrote the scale masks already required them to
    lie inside the domain their own estimate covers.
    """
    return tuple(
        scale_detections_from_islands(
            islands,
            scale_order=scale_order,
            nominal_scale_beam_fwhm=nominal_beam_fwhm,
        )
        for scale_order, (islands, nominal_beam_fwhm) in enumerate(
            zip(
                multiscale.scale_islands_by_order,
                multiscale.scale_nominal_beam_fwhms,
                strict=True,
            ),
            start=1,
        )
    )

Source-count estimate

hebog.expected_source_count counts an image's bright local maxima before an analysis, so that a caller can reserve hebog.analysis_memory_bytes for them; these are its kernels.

hebog.algorithms.source_count

Count an image's bright local maxima, an upper estimate of its sources.

A published source has a peak at least the detection threshold times the local noise above the background, so the pixels that are the highest of their 3x3 neighbourhood and that far above a block's median bound the sources a run can publish. The image is taken in blocks of :data:BLOCK_PIXELS a side, so the count needs no image-sized array; each block's noise is its valid pixels' 3-sigma-clipped standard deviation, lowered to the image's typical block noise where the block's is higher.

These kernels estimate memory before an analysis, not sources: a noise peak counts as a source would, so the count errs high.

BLOCK_PIXELS = 512 module-attribute

The side of a noise block, in pixels.

On the quick check's varying-noise case, 150-pixel blocks, the RMS box Rapthor passes to PyBDSF, read faint sources' own emission as noise and counted fewer peaks than the sources published; 512-pixel blocks bounded the sources of every quick-check, agreement and benchmark case.

MINIMUM_BLOCK_SAMPLES = 64 module-attribute

Valid pixels a block needs before its own noise is measured.

BlockNoise dataclass

A block's clipped median and standard deviation, in image units.

Source code in src/hebog/algorithms/source_count.py

@dataclass(frozen=True, slots=True)
class BlockNoise:
    """A block's clipped median and standard deviation, in image units."""

    median: float
    sigma: float

block_noise(values, valid_pixels)

Return a block's 3-sigma-clipped median and standard deviation.

Returns None when fewer than :data:MINIMUM_BLOCK_SAMPLES pixels are valid or the clipped spread is not positive.

Source code in src/hebog/algorithms/source_count.py

def block_noise(
    values: npt.NDArray[np.float64], valid_pixels: npt.NDArray[np.bool_]
) -> BlockNoise | None:
    """Return a block's 3-sigma-clipped median and standard deviation.

    Returns ``None`` when fewer than :data:`MINIMUM_BLOCK_SAMPLES` pixels
    are valid or the clipped spread is not positive.
    """
    samples = values[valid_pixels]
    if samples.size < MINIMUM_BLOCK_SAMPLES:
        return None
    _, median, sigma = sigma_clipped_stats(
        samples, sigma=_CLIPPING_SIGMA, maxiters=_CLIPPING_ITERATIONS
    )
    if not (np.isfinite(sigma) and sigma > 0):
        return None
    return BlockNoise(median=float(median), sigma=float(sigma))

count_peaks(values, valid_pixels, core, threshold)

Count the core's valid pixels that are 3x3 maxima at the threshold.

A pixel counts when it is valid, no valid neighbour is higher and it is at least threshold. The window around the core supplies the neighbours of its edge pixels; invalid pixels are no one's neighbour. Equal neighbours both count.

Source code in src/hebog/algorithms/source_count.py

def count_peaks(
    values: npt.NDArray[np.float64],
    valid_pixels: npt.NDArray[np.bool_],
    core: tuple[slice, slice],
    threshold: float,
) -> int:
    """Count the core's valid pixels that are 3x3 maxima at the threshold.

    A pixel counts when it is valid, no valid neighbour is higher and it is
    at least ``threshold``. The window around the core supplies the
    neighbours of its edge pixels; invalid pixels are no one's neighbour.
    Equal neighbours both count.
    """
    measured = np.where(valid_pixels, values, -np.inf)
    highest = ndimage.maximum_filter(
        measured, size=3, mode="constant", cval=-np.inf
    )
    peaks = valid_pixels & (measured == highest) & (measured >= threshold)
    return int(np.count_nonzero(peaks[core]))

peak_threshold(noise, *, reference_sigma, detection_sigma)

Return the value a block's peak must reach to count.

It is the block's median plus detection_sigma times the lower of the block's noise and the image's typical noise: a block's spread is inflated by a bright source in it, or by a primary-beam correction's noisier edge, and the typical noise is too high for a quiet block.

Source code in src/hebog/algorithms/source_count.py

def peak_threshold(
    noise: BlockNoise, *, reference_sigma: float, detection_sigma: float
) -> float:
    """Return the value a block's peak must reach to count.

    It is the block's median plus ``detection_sigma`` times the lower of
    the block's noise and the image's typical noise: a block's spread is
    inflated by a bright source in it, or by a primary-beam correction's
    noisier edge, and the typical noise is too high for a quiet block.
    """
    return noise.median + detection_sigma * min(noise.sigma, reference_sigma)

reference_sigma(noises)

Return the median of the blocks' noise, the image's typical noise.

Returns None when no block measured its noise.

Source code in src/hebog/algorithms/source_count.py

def reference_sigma(noises: Iterable[BlockNoise]) -> float | None:
    """Return the median of the blocks' noise, the image's typical noise.

    Returns ``None`` when no block measured its noise.
    """
    sigmas = [noise.sigma for noise in noises]
    return float(np.median(sigmas)) if sigmas else None

Workflow adapters

hebog.adapters.rapthor runs one Rapthor sector through find_sources and writes the files Rapthor's filter_skymodel step reads, under Rapthor's names; hebog.adapters.rapthor_catalogue holds the eight-column catalogue codec it writes them with, and hebog.adapters.rapthor_comparison compares one sector's Hebog and PyBDSF products for Rapthor's dual-run mode. Rapthor calls the adapter on Rapthor's Hebog branch, not yet merged.

hebog.adapters.rapthor

Hebog as the source finder of Rapthor's filter_skymodel step.

Rapthor's step runs PyBDSF through LSMTool on one sector's two images, the primary-beam-corrected true-sky image and the flat-noise image, then filters the sector's sky models with LSMTool's filter_sources. :func:find_sector_sources runs Hebog in its place, in-process on the executor Rapthor builds, and writes the files Rapthor and filter_sources read, under the names Rapthor's PyBDSF step gives them. Rapthor calls it and then filter_sources; this module imports no Rapthor, Prefect, LSMTool or scheduler objects.

RapthorBlankedImageError

Bases: RuntimeError

A sector image has no pixel to measure, as PyBDSF reports it.

PyBDSF raises RuntimeError("All pixels in the image are blanked.") when every pixel is NaN, and Rapthor's step recognises that message and writes its empty products, so this error carries the same message and type. Hebog also counts as blank a pixel inside a 3x3 square of one value, as imagers fill blanked regions, so an image of one value is blank too.

Source code in src/hebog/adapters/rapthor.py

class RapthorBlankedImageError(RuntimeError):
    """A sector image has no pixel to measure, as PyBDSF reports it.

    PyBDSF raises ``RuntimeError("All pixels in the image are blanked.")``
    when every pixel is NaN, and Rapthor's step recognises that message and
    writes its empty products, so this error carries the same message and
    type. Hebog also counts as blank a pixel inside a 3x3 square of one
    value, as imagers fill blanked regions, so an image of one value is
    blank too.
    """

    def __init__(self) -> None:
        """Carry PyBDSF's message."""
        super().__init__(_BLANKED_MESSAGE)

__init__()

Carry PyBDSF's message.

Source code in src/hebog/adapters/rapthor.py

def __init__(self) -> None:
    """Carry PyBDSF's message."""
    super().__init__(_BLANKED_MESSAGE)

RapthorCompatibilityConfig dataclass

Rapthor's source-finding settings, under Rapthor's names.

threshpix and threshisl are the detection and island thresholds in sigma that Rapthor's imaging strategy passes (5 and 3, or 5 and 4 in early self-calibration cycles). The minimum island size is not a setting: :func:find_sector_sources applies PyBDSF's rule to the sector's beam. profile and background choose Hebog's science profile and background. Rapthor's are continuum and zero, PyBDSF's zero mean map: continuum passes the truth-based gate under either background, and with a zero background it covers more of the true emission and keeps the mean map Rapthor passes to PyBDSF. The compact profile fails that gate. The other options LSMTool passes to PyBDSF (the RMS boxes, the bright-source threshold and the wavelet scales) are fixed by Hebog's reviewed science, and LSMTool's mask filter is LSMTool's own; the Rapthor source-finding contract maps each to what replaces it.

Source code in src/hebog/adapters/rapthor.py

@dataclass(frozen=True, slots=True)
class RapthorCompatibilityConfig:
    """Rapthor's source-finding settings, under Rapthor's names.

    ``threshpix`` and ``threshisl`` are the detection and island thresholds
    in sigma that Rapthor's imaging strategy passes (5 and 3, or 5 and 4 in
    early self-calibration cycles). The minimum island size is not a
    setting: :func:`find_sector_sources` applies PyBDSF's rule to the
    sector's beam. ``profile`` and ``background`` choose Hebog's science
    profile and background. Rapthor's are ``continuum`` and ``zero``,
    PyBDSF's zero mean map: ``continuum`` passes the truth-based gate under
    either background, and with a zero background it covers more of the
    true emission and keeps the mean map Rapthor passes to PyBDSF. The
    ``compact`` profile fails that gate. The other options LSMTool passes
    to PyBDSF (the RMS boxes, the bright-source threshold and the wavelet
    scales) are fixed by Hebog's reviewed science, and LSMTool's mask filter
    is LSMTool's own; the Rapthor source-finding contract maps each to what
    replaces it.
    """

    threshpix: float
    threshisl: float
    profile: Literal["continuum", "compact"] = "continuum"
    background: Literal["estimated", "zero"] = "zero"

    def __post_init__(self) -> None:
        """Fail where the settings are made, not after the images load."""
        self.source_finder_config(_PYBDSF_MINIMUM_ISLAND_PIXELS)

    def source_finder_config(
        self, minimum_island_pixels: int
    ) -> SourceFinderConfig:
        """Return the finder's configuration for one minimum island size."""
        return SourceFinderConfig(
            detection_threshold_sigma=self.threshpix,
            island_threshold_sigma=self.threshisl,
            minimum_island_pixels=minimum_island_pixels,
            profile=self.profile,
            background=self.background,
        )

__post_init__()

Fail where the settings are made, not after the images load.

Source code in src/hebog/adapters/rapthor.py

def __post_init__(self) -> None:
    """Fail where the settings are made, not after the images load."""
    self.source_finder_config(_PYBDSF_MINIMUM_ISLAND_PIXELS)

source_finder_config(minimum_island_pixels)

Return the finder's configuration for one minimum island size.

Source code in src/hebog/adapters/rapthor.py

def source_finder_config(
    self, minimum_island_pixels: int
) -> SourceFinderConfig:
    """Return the finder's configuration for one minimum island size."""
    return SourceFinderConfig(
        detection_threshold_sigma=self.threshpix,
        island_threshold_sigma=self.threshisl,
        minimum_island_pixels=minimum_island_pixels,
        profile=self.profile,
        background=self.background,
    )

RapthorSourceFindingRequest dataclass

One Rapthor sector's two images, and the root its products are named by.

output_root is Rapthor's <working directory>/<image name>. The products are <root>.source_catalog.fits, <root>.true_sky_rms.fits and <root>.flat_noise_rms.fits, the island mask is <true-sky image name>.mask.fits in the root's directory, and Hebog's own product bundle is the directory <root>.hebog. run_id names the sector's analysis; a retried task passes the same one.

Source code in src/hebog/adapters/rapthor.py

@dataclass(frozen=True, slots=True)
class RapthorSourceFindingRequest:
    """One Rapthor sector's two images, and the root its products are named by.

    ``output_root`` is Rapthor's ``<working directory>/<image name>``. The
    products are ``<root>.source_catalog.fits``, ``<root>.true_sky_rms.fits``
    and ``<root>.flat_noise_rms.fits``, the island mask is
    ``<true-sky image name>.mask.fits`` in the root's directory, and Hebog's
    own product bundle is the directory ``<root>.hebog``. ``run_id`` names
    the sector's analysis; a retried task passes the same one.
    """

    true_sky_image_path: Path
    flat_noise_image_path: Path
    output_root: Path
    run_id: str
    schema_version: Literal[2] = _SCHEMA_VERSION

    def __post_init__(self) -> None:
        """Reject unsupported versions and empty run identifiers."""
        if self.schema_version != _SCHEMA_VERSION:
            raise ValueError("unsupported Rapthor request schema version")
        if not self.run_id:
            raise ValueError("run_id must not be empty")

__post_init__()

Reject unsupported versions and empty run identifiers.

Source code in src/hebog/adapters/rapthor.py

def __post_init__(self) -> None:
    """Reject unsupported versions and empty run identifiers."""
    if self.schema_version != _SCHEMA_VERSION:
        raise ValueError("unsupported Rapthor request schema version")
    if not self.run_id:
        raise ValueError("run_id must not be empty")

RapthorSourceFindingResult dataclass

The files one sector's analysis wrote for Rapthor, and its counts.

mask_path is the island mask LSMTool's filter_sources clips to the sector polygon, in place, and groups the sky model by. With no island, as with PyBDSF's nisl of zero, LSMTool writes a dummy sky model instead. products_directory is Hebog's own bundle, whose diagnostics bind the run to its inputs, configuration and science.

Source code in src/hebog/adapters/rapthor.py

@dataclass(frozen=True, slots=True)
class RapthorSourceFindingResult:
    """The files one sector's analysis wrote for Rapthor, and its counts.

    ``mask_path`` is the island mask LSMTool's ``filter_sources`` clips to
    the sector polygon, in place, and groups the sky model by. With no
    island, as with PyBDSF's ``nisl`` of zero, LSMTool writes a dummy sky
    model instead. ``products_directory`` is Hebog's own bundle, whose
    diagnostics bind the run to its inputs, configuration and science.
    """

    source_catalog_path: Path
    true_sky_rms_path: Path
    flat_noise_rms_path: Path
    mask_path: Path
    products_directory: Path
    source_count: int
    island_count: int
    wall_seconds: float
    schema_version: Literal[2] = _SCHEMA_VERSION

    def __post_init__(self) -> None:
        """Validate the version and scalar result metadata."""
        if self.schema_version != _SCHEMA_VERSION:
            raise ValueError("unsupported Rapthor result schema version")
        if self.source_count < 0:
            raise ValueError("source_count cannot be negative")
        if self.island_count < 0:
            raise ValueError("island_count cannot be negative")
        if not isfinite(self.wall_seconds) or self.wall_seconds < 0:
            raise ValueError("wall_seconds must be finite and non-negative")

__post_init__()

Validate the version and scalar result metadata.

Source code in src/hebog/adapters/rapthor.py

def __post_init__(self) -> None:
    """Validate the version and scalar result metadata."""
    if self.schema_version != _SCHEMA_VERSION:
        raise ValueError("unsupported Rapthor result schema version")
    if self.source_count < 0:
        raise ValueError("source_count cannot be negative")
    if self.island_count < 0:
        raise ValueError("island_count cannot be negative")
    if not isfinite(self.wall_seconds) or self.wall_seconds < 0:
        raise ValueError("wall_seconds must be finite and non-negative")

find_sector_sources(request, config, executor)

Find one sector's sources and write the files Rapthor's step reads.

One :func:hebog.find_sources run analyses the true-sky image and estimates the flat-noise image's RMS, on the executor the caller built, with Rapthor's thresholds and PyBDSF's minimum island for the sector's beam. It publishes Hebog's bundle in <root>.hebog, from which this function writes the eight-column catalogue, the two RMS maps and the island mask in PyBDSF's four-axis layout, each under Rapthor's name.

Files an earlier run left under those four names are removed before the analysis starts, and a failure removes those this call wrote, so after a failed call none exists. A retry of a finished request reuses the bundle without analysing the images again; a bundle of another request is refused and kept.

Raises:

Type	Description
RapthorBlankedImageError	If either image has no pixel Hebog measures, so its RMS is unavailable. Rapthor's step handles it as it handles PyBDSF's error.
SourceFinderError	If Hebog refuses the sector, or <root>.hebog holds another run's products. Rapthor may fall back to PyBDSF on these typed failures.

Source code in src/hebog/adapters/rapthor.py

def find_sector_sources(
    request: RapthorSourceFindingRequest,
    config: RapthorCompatibilityConfig,
    executor: Executor,
) -> RapthorSourceFindingResult:
    """Find one sector's sources and write the files Rapthor's step reads.

    One :func:`hebog.find_sources` run analyses the true-sky image and
    estimates the flat-noise image's RMS, on the executor the caller built,
    with Rapthor's thresholds and PyBDSF's minimum island for the sector's
    beam. It publishes Hebog's bundle in ``<root>.hebog``, from which this
    function writes the eight-column catalogue, the two RMS maps and the
    island mask in PyBDSF's four-axis layout, each under Rapthor's name.

    Files an earlier run left under those four names are removed before the
    analysis starts, and a failure removes those this call wrote, so after
    a failed call none exists. A retry of a finished request reuses the
    bundle without analysing the images again; a bundle of another request
    is refused and kept.

    Raises:
        RapthorBlankedImageError: If either image has no pixel Hebog
            measures, so its RMS is unavailable. Rapthor's step handles it
            as it handles PyBDSF's error.
        hebog.pipeline.SourceFinderError: If Hebog refuses the sector, or
            ``<root>.hebog`` holds another run's products. Rapthor may fall
            back to PyBDSF on these typed failures.
    """
    started = monotonic()
    request = replace(
        request,
        true_sky_image_path=request.true_sky_image_path.absolute(),
        flat_noise_image_path=request.flat_noise_image_path.absolute(),
        output_root=request.output_root.absolute(),
    )
    names = _ProductNames.of(request)
    _remove(names.rapthor_files())
    true_sky = _image_metadata(request.true_sky_image_path)
    minimum_island_pixels = _shared_minimum_island_pixels(
        true_sky,
        _image_metadata(request.flat_noise_image_path),
        request.flat_noise_image_path,
    )
    result = find_sources(
        SourceFinderRequest(
            image_path=request.true_sky_image_path,
            output_directory=names.products_directory,
            run_id=request.run_id,
            flat_noise_image_path=request.flat_noise_image_path,
        ),
        config.source_finder_config(minimum_island_pixels),
        executor,
    )
    flat_noise_rms = _require_unblanked(result)
    try:
        write_rapthor_catalogue_fits(
            names.source_catalog, read_catalogue_fits_product(result.catalogue)
        )
        _copy(result.rms.path, names.true_sky_rms)
        _copy(flat_noise_rms, names.flat_noise_rms)
        _write_island_mask(result.mask.path, true_sky, names.mask)
    except BaseException:
        _remove(names.rapthor_files())
        raise
    return RapthorSourceFindingResult(
        source_catalog_path=names.source_catalog,
        true_sky_rms_path=names.true_sky_rms,
        flat_noise_rms_path=names.flat_noise_rms,
        mask_path=names.mask,
        products_directory=names.products_directory,
        source_count=result.source_count,
        island_count=result.island_count,
        wall_seconds=monotonic() - started,
    )

pybdsf_minimum_island_pixels(metadata)

Return the minimum island size PyBDSF would use, in pixels.

PyBDSF's minpix_isl defaults to max(6, int(beam area / 3)), with the beam area in pixels taken as 1.1331 times the major axis over the pixel width times the minor axis over the pixel height. The widths are the celestial WCS's pixel scales, |CDELT1| and |CDELT2| for the north-up images Rapthor writes, as PyBDSF reads them.

Source code in src/hebog/adapters/rapthor.py

def pybdsf_minimum_island_pixels(metadata: ImageMetadata) -> int:
    """Return the minimum island size PyBDSF would use, in pixels.

    PyBDSF's ``minpix_isl`` defaults to ``max(6, int(beam area / 3))``, with
    the beam area in pixels taken as 1.1331 times the major axis over the
    pixel width times the minor axis over the pixel height. The widths are
    the celestial WCS's pixel scales, ``|CDELT1|`` and ``|CDELT2|`` for the
    north-up images Rapthor writes, as PyBDSF reads them.
    """
    scale_x, scale_y = (
        float(scale)
        for scale in proj_plane_pixel_scales(
            celestial_wcs_from_metadata(metadata)
        )
    )
    beam_area_pixels = (
        _PYBDSF_BEAM_AREA_FACTOR
        * (metadata.beam.major_fwhm_degrees / scale_x)
        * (metadata.beam.minor_fwhm_degrees / scale_y)
    )
    return max(_PYBDSF_MINIMUM_ISLAND_PIXELS, int(beam_area_pixels / 3.0))

hebog.adapters.rapthor_comparison

Compare one sector's Hebog and PyBDSF products in Rapthor's dual-run mode.

In dual-run mode Rapthor runs both finders on one sector, each under its own output root, and keeps both product sets until it selects one. :func:compare_sector_products reads the four files each finder wrote and reports how they differ, bound to the exact bytes of all eight and to Hebog's provenance. It reads both sets and changes neither.

FinderProducts

Bases: BaseModel

One finder's four files for the sector, and its source count.

Source code in src/hebog/adapters/rapthor_comparison.py

class FinderProducts(BaseModel):
    """One finder's four files for the sector, and its source count."""

    model_config = ConfigDict(extra="forbid", frozen=True, strict=True)

    source_catalog: ProductIdentity
    true_sky_rms: ProductIdentity
    flat_noise_rms: ProductIdentity
    mask: ProductIdentity
    source_count: int

ProductIdentity

Bases: BaseModel

One product file's path and the exact bytes it held.

Source code in src/hebog/adapters/rapthor_comparison.py

class ProductIdentity(BaseModel):
    """One product file's path and the exact bytes it held."""

    model_config = ConfigDict(extra="forbid", frozen=True, strict=True)

    path: str
    byte_count: int
    sha256: str

RapthorDualRunReport

Bases: BaseModel

How one sector's Hebog and PyBDSF products differ.

Sources match when each is the other's nearest within match_radius_arcsec, half the restoring beam's major axis. The flux ratio is Hebog's Total_flux over PyBDSF's, the median over matched sources; each RMS ratio is Hebog's over PyBDSF's, the median over pixels of a regular 16-pixel grid where both are finite and positive. A ratio with nothing to compare is None. The mask counts are of nonzero pixels.

Source code in src/hebog/adapters/rapthor_comparison.py

class RapthorDualRunReport(BaseModel):
    """How one sector's Hebog and PyBDSF products differ.

    Sources match when each is the other's nearest within
    ``match_radius_arcsec``, half the restoring beam's major axis. The flux
    ratio is Hebog's ``Total_flux`` over PyBDSF's, the median over matched
    sources; each RMS ratio is Hebog's over PyBDSF's, the median over pixels
    of a regular 16-pixel grid where both are finite and positive. A ratio
    with nothing to compare is ``None``. The mask counts are of nonzero
    pixels.
    """

    model_config = ConfigDict(extra="forbid", frozen=True, strict=True)

    hebog: FinderProducts
    pybdsf: FinderProducts
    hebog_provenance: PublicSourceFindingProvenance
    match_radius_arcsec: float
    matched_source_count: int
    median_total_flux_ratio: float | None
    true_sky_rms_median_ratio: float | None
    flat_noise_rms_median_ratio: float | None
    hebog_mask_pixel_count: int
    pybdsf_mask_pixel_count: int
    shared_mask_pixel_count: int
    schema_version: Literal[1] = 1

RapthorSectorProducts dataclass

The four files Rapthor's filter step reads for one sector.

Source code in src/hebog/adapters/rapthor_comparison.py

@dataclass(frozen=True, slots=True)
class RapthorSectorProducts:
    """The four files Rapthor's filter step reads for one sector."""

    source_catalog_path: Path
    true_sky_rms_path: Path
    flat_noise_rms_path: Path
    mask_path: Path

    @classmethod
    def named(
        cls, output_root: Path, true_sky_image_path: Path
    ) -> RapthorSectorProducts:
        """Name them as Rapthor's PyBDSF step names them from its root."""
        directory = output_root.parent
        stem = output_root.name
        return cls(
            source_catalog_path=directory / f"{stem}.source_catalog.fits",
            true_sky_rms_path=directory / f"{stem}.true_sky_rms.fits",
            flat_noise_rms_path=directory / f"{stem}.flat_noise_rms.fits",
            mask_path=directory / f"{true_sky_image_path.name}.mask.fits",
        )

    @classmethod
    def of(cls, result: RapthorSourceFindingResult) -> RapthorSectorProducts:
        """Return the four files one Hebog run wrote."""
        return cls(
            source_catalog_path=result.source_catalog_path,
            true_sky_rms_path=result.true_sky_rms_path,
            flat_noise_rms_path=result.flat_noise_rms_path,
            mask_path=result.mask_path,
        )

    def paths(self) -> tuple[Path, Path, Path, Path]:
        """Return the catalogue, the two RMS maps and the mask, in order."""
        return (
            self.source_catalog_path,
            self.true_sky_rms_path,
            self.flat_noise_rms_path,
            self.mask_path,
        )

named(output_root, true_sky_image_path) classmethod

Name them as Rapthor's PyBDSF step names them from its root.

Source code in src/hebog/adapters/rapthor_comparison.py

@classmethod
def named(
    cls, output_root: Path, true_sky_image_path: Path
) -> RapthorSectorProducts:
    """Name them as Rapthor's PyBDSF step names them from its root."""
    directory = output_root.parent
    stem = output_root.name
    return cls(
        source_catalog_path=directory / f"{stem}.source_catalog.fits",
        true_sky_rms_path=directory / f"{stem}.true_sky_rms.fits",
        flat_noise_rms_path=directory / f"{stem}.flat_noise_rms.fits",
        mask_path=directory / f"{true_sky_image_path.name}.mask.fits",
    )

of(result) classmethod

Return the four files one Hebog run wrote.

Source code in src/hebog/adapters/rapthor_comparison.py

@classmethod
def of(cls, result: RapthorSourceFindingResult) -> RapthorSectorProducts:
    """Return the four files one Hebog run wrote."""
    return cls(
        source_catalog_path=result.source_catalog_path,
        true_sky_rms_path=result.true_sky_rms_path,
        flat_noise_rms_path=result.flat_noise_rms_path,
        mask_path=result.mask_path,
    )

paths()

Return the catalogue, the two RMS maps and the mask, in order.

Source code in src/hebog/adapters/rapthor_comparison.py

def paths(self) -> tuple[Path, Path, Path, Path]:
    """Return the catalogue, the two RMS maps and the mask, in order."""
    return (
        self.source_catalog_path,
        self.true_sky_rms_path,
        self.flat_noise_rms_path,
        self.mask_path,
    )

compare_sector_products(hebog, pybdsf, report_path)

Report how one sector's Hebog and PyBDSF products differ.

The report is written to report_path in one rename and returned. Both product sets are only read, so Rapthor can still select either.

Raises:

Type	Description
ValueError	If the two finders' RMS maps or masks are on different pixel grids, or Hebog's bundle holds no public diagnostics.

Source code in src/hebog/adapters/rapthor_comparison.py

def compare_sector_products(
    hebog: RapthorSourceFindingResult,
    pybdsf: RapthorSectorProducts,
    report_path: Path,
) -> RapthorDualRunReport:
    """Report how one sector's Hebog and PyBDSF products differ.

    The report is written to ``report_path`` in one rename and returned.
    Both product sets are only read, so Rapthor can still select either.

    Raises:
        ValueError: If the two finders' RMS maps or masks are on different
            pixel grids, or Hebog's bundle holds no public diagnostics.
    """
    hebog_products = RapthorSectorProducts.of(hebog)
    diagnostics = read_diagnostics_product(
        hebog.products_directory / "diagnostics.json"
    )
    if not isinstance(diagnostics, PublicSourceFindingDiagnostics):
        raise ValueError("Hebog's bundle holds no public diagnostics")
    hebog_sources = _sources(
        read_rapthor_catalogue_fits(hebog_products.source_catalog_path)
    )
    pybdsf_sources = _sources(
        Table.read(pybdsf.source_catalog_path, format="fits", hdu=1)
    )
    radius_arcsec = (
        float(fits.getheader(hebog_products.true_sky_rms_path)["BMAJ"])
        * 3600.0
        / 2.0
    )
    matched = _mutual_matches(hebog_sources, pybdsf_sources, radius_arcsec)
    hebog_mask, pybdsf_mask, shared_mask = _mask_pixel_counts(
        hebog_products.mask_path, pybdsf.mask_path
    )
    report = RapthorDualRunReport(
        hebog=_finder_products(hebog_products, len(hebog_sources[0])),
        pybdsf=_finder_products(pybdsf, len(pybdsf_sources[0])),
        hebog_provenance=diagnostics.provenance,
        match_radius_arcsec=radius_arcsec,
        matched_source_count=len(matched[0]),
        median_total_flux_ratio=_median_ratio(
            hebog_sources[2][matched[0]], pybdsf_sources[2][matched[1]]
        ),
        true_sky_rms_median_ratio=_rms_median_ratio(
            hebog_products.true_sky_rms_path, pybdsf.true_sky_rms_path
        ),
        flat_noise_rms_median_ratio=_rms_median_ratio(
            hebog_products.flat_noise_rms_path, pybdsf.flat_noise_rms_path
        ),
        hebog_mask_pixel_count=hebog_mask,
        pybdsf_mask_pixel_count=pybdsf_mask,
        shared_mask_pixel_count=shared_mask,
    )
    temporary = report_path.with_name(f".{report_path.name}.{uuid4().hex}.tmp")
    try:
        temporary.write_text(
            report.model_dump_json(indent=2), encoding="utf-8"
        )
        temporary.replace(report_path)
    finally:
        temporary.unlink(missing_ok=True)
    return report

hebog.adapters.rapthor_catalogue

Minimal deterministic PyBDSF-style catalogue view consumed by Rapthor.

Total_flux carries the source flux the public catalogue publishes. For the continuum profile that is the sum of the source's fitted Gaussian components, which is how PyBDSF defines it and what Rapthor's photometry check compares against; a source with no admitted fit falls back to its signed aperture. Isl_Total_flux remains the island sum, which Rapthor carries only through the astrometry check.

read_rapthor_catalogue_fits(path)

Read and strictly validate the frozen Rapthor catalogue view.

Source code in src/hebog/adapters/rapthor_catalogue.py

def read_rapthor_catalogue_fits(path: Path) -> Table:
    """Read and strictly validate the frozen Rapthor catalogue view."""
    path = Path(path)
    try:
        with fits.open(path, mode="readonly", memmap=False) as hdus:
            if (
                tuple(hdu.name for hdu in hdus) != ("PRIMARY", "SOURCES")
                or hdus[0].header.get("HBGROLE") != "RAPTHOR"
                or hdus[0].header.get("HBGSCHE") != 1
                or hdus[0].header.get("HBGFRAME") != "icrs"
                or hdus[0].header.get("HBGEPCH") != POSITION_EPOCH
                or tuple(hdus[1].columns.names) != RAPTHOR_CATALOGUE_COLUMNS
            ):
                raise ValueError("Rapthor catalogue FITS schema is invalid")
            units = tuple(
                None if column.unit is None else str(column.unit)
                for column in hdus[1].columns
            )
            if units != _COLUMN_UNITS:
                raise ValueError(
                    "Rapthor catalogue FITS schema has invalid units"
                )
            formats = tuple(str(column.format) for column in hdus[1].columns)
            if formats != ("J", "D", "D", "D", "D", "D", "D", "D"):
                raise ValueError(
                    "Rapthor catalogue FITS schema has invalid dtypes"
                )
    except OSError as error:
        raise ValueError(
            f"cannot read Rapthor catalogue FITS: {error}"
        ) from error
    return Table.read(path, format="fits", hdu=1)

write_rapthor_catalogue_fits(path, catalogue)

Atomically publish a deterministic validated Rapthor catalogue view.

Source code in src/hebog/adapters/rapthor_catalogue.py

def write_rapthor_catalogue_fits(
    path: Path,
    catalogue: SourceCatalogue,
) -> MaterializedProduct:
    """Atomically publish a deterministic validated Rapthor catalogue view."""
    path = Path(path)
    path.parent.mkdir(parents=True, exist_ok=True)
    temporary = path.with_name(f".{path.name}.{uuid4().hex}.tmp")
    try:
        with _catalogue_hdus(catalogue) as hdus:
            for hdu in hdus:
                hdu.add_checksum(when=_CHECKSUM_COMMENT)
            hdus.writeto(temporary, overwrite=False, checksum=False)
        read_rapthor_catalogue_fits(temporary)
        candidate = _product(temporary)
        if path.exists():
            current = _product(path)
            if (
                current.byte_count == candidate.byte_count
                and current.content_sha256 == candidate.content_sha256
            ):
                return current
            raise MaterializedProductConflictError(
                f"destination contains different product bytes: {path}"
            )
        temporary.replace(path)
        return _product(path)
    finally:
        temporary.unlink(missing_ok=True)