Jobs¶

A job is a named, schedulable workload: it declares what to materialize and, for a cron job, when. The core carries the declaration and compiles it to the same DAG every other entry point runs. Acting on the schedule is the job of the platform scheduler, outside this site.

Job¶

import interloper as il

job = il.Job(
    targets=[Shop(account_id="act_1"), Finance()],
    destinations=[il.CSVDestination(base_path="./data")],
    tags=["daily"],
)
il.DAG(job).materialize(partition)

A job's destinations are the default for every target that resolves none: a source with its own destinations, or an asset whose source has some, keeps them. The default applies where the job runs (job.operations(), il.DAG(job)), on a copy of the target, so the target itself is never changed and two jobs sharing it each run it with their own. A job subclass that declares a further relation (a connection, say) is a default for its targets the same way.

JobState (next_run_at, last_run_at) is the job's machine-owned state, written by the scheduler. Editing a job's config clears next_run_at, so the next tick re-derives the schedule from the new spec rather than firing once more at the slot the old one produced.

CronJob¶

CronJob adds the trigger:

job = il.CronJob(
    cron="0 6 * * *",
    timezone="Europe/Berlin",
    lookback=3,
    offset=1,
    targets=[Shop(account_id="act_1")],
)

For daily targets the defaults mean "yesterday, in the job's timezone". offset=3, lookback=3 covers the three days ending three days ago, for a vendor whose numbers settle late. The window itself is computed with TimePartitionWindow.lookback; hourly targets always use UTC windows because hour ids are UTC labels. Whether a job is partitioned is derived from its targets, never stored. Each firing is a backfill over the window, dispatched newest partition first and gated by concurrency: at 1 the partitions run one at a time.

On the platform, a firing is quota-checked like a manual backfill: its window counts against the organisation's max_backfill_partitions, and it needs the monthly run quota. A firing the quotas reject is skipped, logged as a warning event on the job, and the job moves on to its next scheduled time.

Jobs as specs¶

A job is the natural unit for a declarative run. Its spec lists targets with their configuration and the workload-level defaults. A connection a target needs is not a job field, so it is nested under that target's own relation name, in the target's own init:

path: interloper.job.base.Job
init:
  destinations:
    - key: bigquery_destination
      init:
        project: dwh
        location: EU
        connection:
          key: google_cloud_connection
          init:
            service_account_key: ${GCP_KEY}
  targets:
    - key: facebook_ads
      init:
        account_id: act_1
        connection:
          key: facebook_ads_connection
          init:
            access_token: ${FB_TOKEN}
            app_id: "1"
            app_secret: ${FB_SECRET}
        assets: {campaigns: {}, ads_stats: {}}

interloper run -f job.yaml --date 2026-01-15 reconstructs it and runs its DAG. See Specs and serialization and CLI. A scheduled job is the same document with key: cron_job and the trigger fields.

Relations¶

A job's relation vocabulary is targets (sources and assets) and destinations. A target is an orchestration pointer, not an input: deleting a target shrinks the job rather than blocking the deletion. Relation semantics are described in the component model.