cesm_hawc.resume¶

source

Helpers that let long batch runs resume after being interrupted.

They work at two levels, used together:

• Job level: outputs_already_exist() lets a dispatcher skip a whole job (e.g. a day) whose output files already exist.

• Item level: load_completed_keys() and append_csv_row() let a job that processes many items (e.g. one day’s observations) record each finished item in a CSV as it goes, and skip those items when re-run.

outputs_already_exist¶

def outputs_already_exist(expected_paths: list[str]) -> bool

source

Check whether every expected output file exists.

Parameters¶

• expected_paths (list of str) — Paths a finished job would have written.

Returns¶

• bool — True if all of them exist.

load_completed_keys¶

def load_completed_keys(
    csv_path: str,
    key_columns: list[str],
    expected_fieldnames: list[str],
) -> set[tuple]

source

Read which items a progress CSV already records as done.

Parameters¶

• csv_path (str) — Progress CSV written by append_csv_row().

• key_columns (list of str) — Columns that identify an item.

• expected_fieldnames (list of str) — The CSV’s full expected header.

Returns¶

• set of tuple — One tuple of key_columns values (as strings) per recorded row. Empty if the file doesn’t exist, is empty, or can’t be read.

Notes¶

If the file’s header doesn’t match expected_fieldnames (for example, a file left by an older version), it is renamed to <csv_path>.schema_mismatch.bak and the run starts fresh, rather than appending inconsistent rows or misreading columns.

append_csv_row¶

def append_csv_row(csv_path: str, row: dict, fieldnames: list[str]) -> None

source

Append one row to a progress CSV, writing it to disk immediately.

Writing each row as it finishes, rather than once at the end of a job, means a killed job loses at most the item in progress.

Parameters¶

• csv_path (str) — Progress CSV; it and its directory are created if needed, with a header row for a new file.

• row (dict) — Values for this row, keyed by column name.

• fieldnames (list of str) — Column order.