cesm_hawc.calibration¶

source

Workarounds for cache races when many simulations run in parallel.

hawcsimulator and aliprocessing build cached files the first time a simulator is constructed. When many worker processes start at once, they race to write those files. The functions here pre-build the caches once in the main process and stop hawcsimulator rewriting its calibration file on every call. They do nothing if the [sim] extra isn’t installed.

patch_calibration_database_race¶

def patch_calibration_database_race() -> None

source

Stop hawcsimulator rewriting its calibration cache on every call.

hawcsimulator’s calibration_database() rewrites its cached NetCDF file every time it is called, including whenever an IdealALISimulator is constructed. With many worker processes using the same (often network-mounted) file, this causes PermissionError and KeyError races. After patching, an existing cache file is reused instead. This is safe because the file’s contents depend only on its (name, version).

Safe to call more than once; does nothing if hawcsimulator isn’t installed. cesm_hawc.configure_environment() calls it for you.

warm_calibration_database¶

def warm_calibration_database(
    name: str = 'ideal_spectrograph',
    version: str = 'v1',
) -> None

source

Build hawcsimulator’s calibration cache file once.

Call this in the main process before starting worker processes, so they don’t all race to create the file. Failures are logged as warnings, not raised.

Parameters¶

• name, version (str, optional) — Calibration dataset to build. The defaults are the dataset the ideal_dolp_imager simulator uses.

warm_retrieval_optical_database¶

def warm_retrieval_optical_database() -> None

source

Build the retrieval’s Mie database cache once.

Every IdealALISimulator builds aliprocessing’s aerosol_median_radius_db() when constructed. Call this in the main process before starting worker processes, so they don’t race to write it (a worker reading a half-written file fails with an xarray “did not find a match in any of xarray’s currently installed IO backends” error). Failures are logged as warnings, not raised.