# SSH access to workload VMs

> Most agents never need this page. If you have a git repo, [`deploy`](./deploy.md) ships your code without you ever SSHing in. SSH is for the `provision` path (bare VM, you bring your own runtime) and for one-off operator debugging on dynamic deploys.

## What gets baked into a VM

Every VM h4a creates — `provision`, `deploy` (dynamic), and `attach` (Postgres) — has the public keys listed in the operator's `HCLOUD_SSH_KEY_IDS` env var injected into `/root/.ssh/authorized_keys` at boot. There is no per-tenant or per-workload key separation in v0; whichever public keys the operator registered with their Hetzner project go onto every VM.

Hetzner does not bake a password — only the keys. SSH-by-password is disabled.

## Why the first SSH attempt sometimes fails

The `ssh_command` returned by `provision` and `info` is `ssh root@<ipv4>` by default. That asks SSH to use whichever default key your client picks (typically `~/.ssh/id_rsa` or `~/.ssh/id_ed25519`), which usually doesn't match what the operator registered with Hetzner.

Symptom an agent will hit:

```
$ ssh root@178.105.61.102
root@178.105.61.102: Permission denied (publickey,password).
```

There are three fixes, in order of preference:

### 1. Operator sets `H4A_SSH_KEY_HINT` (recommended, one-time)

Add to the control plane's `.env`:

```
H4A_SSH_KEY_HINT=~/.ssh/id_h4a
```

Restart the control plane. From that point on, `ssh_command` becomes `ssh -i ~/.ssh/id_h4a root@<ipv4>` — agents copy-paste-and-run, no guessing.

The hint is a string the control plane returns verbatim; it's expanded by the agent's local shell. Use whatever path makes sense on the operator's machine. For multiple operators, point it at the most common key — others can override with their own `-i` flag.

### 2. Agent passes `-i` explicitly

If `H4A_SSH_KEY_HINT` isn't set, the agent's caller (the human) usually knows which local private key matches their Hetzner-registered key. Convention in this project: `~/.ssh/id_h4a`. Try:

```
ssh -i ~/.ssh/id_h4a root@<ipv4>
```

If that's the wrong filename, list local keys (`ls ~/.ssh/*.pub`) and try each — the right one will succeed without prompting for a password.

### 3. Operator adds the agent's public key to Hetzner

Long path: register a new SSH key in the Hetzner Cloud Console, append its ID to `HCLOUD_SSH_KEY_IDS`, restart the control plane, then `destroy` + re-provision the workload (existing VMs were created before the new key existed and won't trust it).

## Verifying which keys are on a running VM

Once you're in (via any working key):

```
cat /root/.ssh/authorized_keys
```

That's the ground truth — what Hetzner injected at boot.

## Rotating keys

There is no rotation primitive in v0. To rotate:

1. Add the new public key to Hetzner Cloud Console.
2. Update `HCLOUD_SSH_KEY_IDS` and restart the control plane.
3. For existing VMs: append the new pubkey to `/root/.ssh/authorized_keys` manually. New VMs get it automatically.
4. Once every VM has the new key, remove the old key from Hetzner (this does NOT remove it from existing VMs — `authorized_keys` is captured at boot, not synced).

## Why h4a doesn't manage SSH keys for you

It could, but it would mean the control plane holding agent-callers' private keys, and that breaks the "operator controls the trust root" property. Letting Hetzner's key store stay authoritative keeps key custody where it already lives.