Parallel orchestration

Multi-filter parallel scalefit dispatch, Rich live-progress UI, and the worker/log plumbing used by run_sphot.

Parallel execution of run_scalefit across filters.

Architecture

  • One worker process per filter (multiprocessing.spawn).

  • Each worker reads the galaxy from a parent-saved temp h5, calls sphot.core.run_scalefit on its assigned filter, and returns the modified CutoutData. The parent merges results back into the galaxy.

  • Per-filter progress (Rich) is rendered in the parent via:

    • one PRE-ALLOCATED Progress row per filter (the “pre-spaced” layout the user asked for — rows do not jump as filters start),

    • dynamically-added child rows for nested tasks (NNLS refit, iPSF ladder, calibrate_psf_step, …).

  • Workers communicate progress events via a multiprocessing.Manager Queue. A listener thread in the parent translates events into real Progress.update / add_task / remove_task calls.

  • Workers redirect stdout/stderr to per-filter log files so the parent’s Live display stays clean.

Public API

parallel_scalefit(galaxy, base_params, filters, blur_psf, …)

class sphot.parallel.QueueProgressProxy(queue, filter_label)[source]

Bases: object

Pickleable proxy for rich.Progress.

Mirrors the subset of Progress’s API that sphot’s core / psf code uses: add_task, update, remove_task.

add_task(description, total=None, **kw)[source]
update(task_id, advance=None, completed=None, total=None, refresh=None, description=None, **kw)[source]
remove_task(task_id)[source]
sphot.parallel.parallel_scalefit(galaxy, base_params, filters: Sequence[str], blur_psf, *, allow_refit: bool = False, fit_complex_model: bool = False, N_mainloop_iter: int = 5, n_workers: int | None = None, working_dir: str | None = None, log_dir: str | None = None, console=None)[source]

Run run_scalefit on each filter in filters in parallel.

Returns the list of (filter, CutoutData) tuples returned by workers. Modifies galaxy.images[filter] in-place with the worker results.

working_dir should be the directory where sphot_config.toml lives (workers chdir to it before importing sphot.config). log_dir is where each filter’s stdout/stderr is captured.