CLI Commands

Reference for Aether CLI commands.

Global Options

aether [global-options] <command> ...

• --verbose, -v - Enable verbose logging
• --help, -h - Show help
• --version - Show version

Every subcommand that needs configuration takes the path to aether.yaml as its first positional argument. There is no --config flag and no implicit discovery of ./aether.yaml or ~/.config/aether/aether.yaml.

Pipeline Commands

aether pipeline start

Start a new pipeline job.

aether pipeline start <config> <crtdl> [input] [options]

Arguments:

• <config> - Path to aether.yaml (required, first positional)
• <crtdl> - CRTDL JSON file (required, second positional)
• [input] - Optional import-step input:
  • omitted: torch_import submits the CRTDL; local_import uses --dir/config
  • local directory (local_import)
  • HTTP(S) URL (http_import)
  • TORCH result URL (torch_import; auto-detected from URL shape)

Options:

• --no-progress - Disable progress indicators
• --dir PATH - Directory for local import (overrides config)
• --allow-http-crtdl - Acknowledge that HTTP data may not match the CRTDL query (required when http_import is enabled)

Examples:

# TORCH extraction with CRTDL
aether pipeline start aether.yaml crtdl.json

# Direct TORCH URL (skip extraction, poll and download results)
aether pipeline start aether.yaml crtdl.json "https://torch.example.com/fhir/extraction/result-123"

# Local import with CRTDL for flattening (dir from flag)
aether pipeline start aether.yaml crtdl.json --dir /path/to/data

# Local import with CRTDL for flattening (dir as positional)
aether pipeline start aether.yaml crtdl.json /path/to/data

# HTTP download piped through flattening
aether pipeline start aether.yaml crtdl.json https://example.com/fhir/data.ndjson --allow-http-crtdl

# Disable progress bars
aether pipeline start aether.yaml crtdl.json --no-progress

aether pipeline status

Check pipeline job status.

aether pipeline status <config> <job-id>

Output:

• Job ID and status
• Current step
• Progress per step (files, bytes)
• Error messages if failed

Example:

aether pipeline status aether.yaml abc-123-def

aether pipeline continue

Resume a paused or failed pipeline.

aether pipeline continue <config> <job-id> [options]

Use cases:

• Resume after terminal close
• Continue after fixing errors
• Resume from wait step after placing files in wait directory

Options:

• --no-progress - Disable progress indicators

Examples:

# Resume failed job
aether pipeline continue aether.yaml abc-123-def

# Resume from wait step (after placing files)
aether pipeline continue aether.yaml abc-123-def

# Resume without progress bars
aether pipeline continue aether.yaml abc-123-def --no-progress

Job Commands

aether job list

List all pipeline jobs.

aether job list <config>

Output columns:

• JOB ID - Job ID (YYYYMMDD_HHMM_UUID)
• STATUS - completed/in_progress/failed/pending
• STEP - Current step
• FILES - Total files processed
• AGE - Time since creation

Status symbols:

• ✓ - Completed
• → - In progress
• ✗ - Failed
• ○ - Pending

Example:

aether job list aether.yaml

aether job run

Execute a specific pipeline step manually.

aether job run <config> <job-id> --step <step-name>

Options:

• --step - Step to execute (required)
• --force - Re-run the step even if it has already completed

Valid steps: torch, local_import, http_import, dimp, validation, flattening, send

Re-running completed steps: An already-completed step is not re-run unless --force is passed. Without --force, the command fails fast with step '<step>' already completed; pass --force to re-run and a non-zero exit. This is a pre-execution guard — nothing runs, so job state is left untouched.

With --force, the step is reset to pending and re-run. Every step ordered after it is also reset to pending, because re-running a step invalidates the output the later steps consumed (outputs are not diffed, so this is conservative). The command itself re-runs only the target step; the invalidated downstream steps stay pending until you re-run them — either with another job run --step or by resuming the pipeline.

Job status after a manual run: The job-level status is always derived from its steps. While a --force re-run runs, the job reads in_progress (visible in aether pipeline status). After it finishes, the status is re-derived: the job returns to completed only if every step is completed — so re-running the last step completes the job, while re-running an earlier step leaves it in_progress until the invalidated downstream steps re-run. Likewise, a manual run that finishes the last outstanding step sets the job to completed.

Examples:

# Run DIMP step manually
aether job run aether.yaml abc-123-def --step dimp

# Run import step manually
aether job run aether.yaml abc-123-def --step local_import

# Re-run an already-completed step
aether job run aether.yaml abc-123-def --step dimp --force

Other Commands

aether completion

Generate shell completion scripts.

aether completion <shell>

Shells: bash, zsh, fish, powershell

Examples:

# Generate bash completions
aether completion bash > /etc/bash_completion.d/aether

# Generate zsh completions
aether completion zsh > "${fpath[1]}/_aether"

aether help

Show help information.

aether help [command]

Examples:

aether help
aether help pipeline
aether help pipeline start

Configuration Loading

The path to aether.yaml is the first positional argument of every command that needs configuration; aether does not auto-discover it. CLI flags and AETHER_* environment variables still override individual values from the file.

Environment Variables

Configuration values support environment variable substitution:

services:
  torch:
    username: "${TORCH_USERNAME}"
    password: "${TORCH_PASSWORD}"

Next Steps

• Configuration Reference - All configuration options
• Pipeline Steps - Step details