Console authentication
How you get into a gateway’s own web console, and why it works the way it does.
The problem
Section titled “The problem”A gateway lives on a pole or in a cabinet. It is worth stealing. And it has to stay administrable when its backhaul is down — which is exactly when you most need to get into it and exactly when central identity is unavailable.
Those pull in opposite directions:
-
If offline local login always works with a stored credential, then whoever physically holds a stolen gateway can extract that credential from flash at their leisure. Flash is not a secret store. Note that physical presence proves nothing here: a thief has more physical presence than we do, so a reset-button or serial-console gate is not a control, it is a speed bump.
-
If offline login never works, then a site with a dead SIM and a dead uplink is a truck roll with a laptop that cannot do anything when it arrives.
Three states, three answers
Section titled “Three states, three answers”Rather than stretch one mechanism over all of it:
1. Unadopted
Section titled “1. Unadopted”Nobody owns this gateway yet. The console asks for the local password, and
the claim screen is what opens after it. Not before: the claim code is a
key to the gateway, and the console is on the site’s network. See
provisioning.md.
The password is unique per unit, generated on first boot from
/dev/urandom, and on the label, the boot partition of a Pi
(pulzr-initial-password.txt) and the root login banner. It is not
pulzr/pulzr — which is what the previous PulzarOS shipped on every
image, and which means one published README is root on the entire fleet. It is
the console’s password and nothing else: root has none (below).
The cleartext copy lives at /etc/pulzr/console/initial-password, mode 0600,
so an installer can read it off the device. The console nags on every page
until it is changed, and deletes the cleartext the moment it is.
2. Adopted and online
Section titled “2. Adopted and online”Pulzar Cloud is the identity provider. The console accepts a signed one-time
handoff token minted by recovery-service, so an operator already
authenticated centrally lands signed in without a second password.
Local password login is disabled on adoption — pulzr.console.local_login
goes to 0 — not merely discouraged.
The terminal UI has no browser to receive that handoff, so it collects the same
token itself: it shows a code, somebody approves it on Pulzar Cloud from a
phone, and the gateway picks the token up. The token is marked for the terminal
and for that one request, and the web console refuses tokens marked for
anything. See tui.md.
3. Adopted and offline
Section titled “3. Adopted and offline”The interesting case. Local login stays refused, because the credential a thief could extract must not be a way in. Both ways in end in a signature the gateway checks with a key it only ever holds the public half of, so neither needs the gateway to reach anything:
| The browser | Way in |
|---|---|
| reaches the platform | Sign in with Pulzar, exactly as online. The browser carries the token. |
| does not | Recovery access, with the code carried on a second device. |
The sign-in button asks first whether the browser can reach the platform, and
goes to recovery with a notice if it cannot. The platform returns a token only to
a private IP, so a browser that came by pulzr.local is returned to the
address the request reached the gateway on (address in /api/auth/methods).
The same holds from a LAN port and from the setup AP. See
access.md for which of them leaves a phone its own data.
On the bench: a LAN window
Section titled “On the bench: a LAN window”The ethernet port’s firewall admits only 169.254.0.0/16 and the console
subnet, so a gateway on a bench LAN is reachable only through Pulzar Cloud.
From a root shell (SSH as root, or the serial console of a development image):
pulzr-local-access open [minutes] # default 60, at most 1440pulzr-local-access closepulzr-local-access statusopen admits the subnet of the port’s DHCP lease on 22, 80 and 443, and prints
a one-time sign-in link. Running it again extends the window and prints a new
link. The session the link opens ends when that window closes, and does not come
back if another window is opened later.
None of this is a way past the rules above. Opening a window takes root, and
root can already read everything on the box. There is no API for it. The rule,
the window and the link live in /var/run, so a reboot closes all of it.
Offline recovery, step by step
Section titled “Offline recovery, step by step”-
The gateway prints a challenge — its EUI, a fresh 12-byte random nonce, and the time — as one string:
a84041ffff1234ab.9f3c2b1a8e7d6c5b4a39281f.1789330979 -
The operator carries it to Pulzar Cloud on a second device. A phone has cellular even when the gateway’s uplink is dead.
-
Pulzar Cloud checks they are a net-admin, claims the nonce in
gateway_recovery_events(a unique index, so a nonce can be signed exactly once), writes an audit row, and hasrecovery-servicesign those exact bytes with the fleet Ed25519 key. -
The operator carries the base64 signature back and types or pastes it in.
-
The gateway verifies it with
openssl pkeyutl -verify -pubin -rawinagainst/etc/pulzr/recovery.pub, spends the challenge, and grants a 30-minute session that the UI banners as a recovery session with a live countdown.
What this buys
Section titled “What this buys”- The gateway stores no secret that grants access to it. Dump the flash of a stolen gateway and you get a public key, which is worth nothing.
- Every recovery is authorized and audited centrally, even though the gateway was never online.
- The nonce makes each signature good exactly once, for one device. A recovery code left in a chat window is not a standing key.
What it costs
Section titled “What it costs”An operator needs a second device with connectivity. That is the trade, and it is acceptable: the alternative is an extractable skeleton key. It is the same trade a car key fob makes.
The failure mode to know about
Section titled “The failure mode to know about”A gateway adopted without a recovery key installed can only be recovered by reflashing it. Two things prevent that:
build.shbakes the fleet public key into the image, so a gateway has an anchor from first boot — this is also what makes offline adoption possible at all. An image built without one warns loudly.pulzr-adoptrefuses to complete if neither the bundle nor the image supplies an anchor, and leaves local sign-in enabled rather than locking the gateway into a state with no way back in.
The key itself
Section titled “The key itself”One Ed25519 keypair for the whole fleet.
- Public half — installed on every gateway at adoption, at
/etc/pulzr/recovery.pub. - Private half — exists only in
recovery-service, mounted read-only from the secret store. Never in an environment variable in production: those show up indocker inspectand in whatever shipped your logs.
Generate the pair with whatever key tooling the platform operator uses; an image build needs only the public half.
If the private key leaks, the security property above is gone for every gateway already deployed, retroactively — not just future ones. Rotating means re-adopting the entire fleet, because the public key is installed at adoption. Keep the retired private key in the secret store rather than destroying it, so gateways still holding the old public key remain recoverable until they are re-adopted.
Everything else about the session layer
Section titled “Everything else about the session layer”- Password hashing — sha512crypt at 100k rounds via
busybox mkpasswd. ucode has no KDF anducode-mod-digestoffers plain hashes only; libcrypt is already on the device and produces a standard$6$rounds=N$salt$hashstring. The password goes tomkpasswdover a pipe, so it never touches disk and never appears in the process table. - Sessions — 32 random bytes, stored under
/var/runso a reboot logs everybody out. Looked up bysha256(token)so a token containing..cannot traverse the filesystem. - Cookies — HttpOnly, Secure, SameSite=Strict, Path=/.
- CSRF — every non-GET must echo the session’s CSRF token in
X-Pulzar-CSRF. Belt-and-braces over SameSite, because the console is also reachable through the Pulzar Cloud proxy under a different origin where SameSite reasoning gets subtle. - Lockout — 5 failures per source address, then 15 minutes. Per-address and not per-account: there is one account, so a per-account counter is a denial of service anybody on the LAN can aim at the legitimate operator.
- Timing — username and password are always both checked and the results
combined at the end, so a wrong username and a wrong password take the same
time. Token and hash comparison is constant-time over
sha256of both sides, so length does not leak either.
One thing to be careful about
Section titled “One thing to be careful about”loginMethods() reports whether the backhaul is up, but the decision about
whether local login is permitted does not consult it. It keys on adoption
state.
That is deliberate. An attacker can cut a gateway’s uplink — that is what snipping a cable does. If losing the uplink re-enabled password login, cutting the cable would be step one of every attack. Adoption state is something they cannot change from outside.
OpenSSH’s sshd on port 22, on every board. dropbear, which every image
before it ran, cannot check a certificate, and certificates are how
Pulzar Cloud reaches a gateway’s shell: one signed for this gateway’s EUI and
nothing else, valid for five minutes, permit-pty only for the browser
terminal. A certificate leaked from one session opens one gateway for five
minutes. Pulzar Cloud’s docs/gateway-ssh-certificates.md is the other half.
What sshd trusts is written at every boot by pulzr-ssh-setup:
| File | From |
|---|---|
/etc/ssh/sshd_config |
/usr/share/pulzr/ssh/sshd_config in the image |
/etc/ssh/pulzr_user_ca.pub |
boards/common/ssh/pulzr_user_ca.pub in this repo, via the image |
/etc/ssh/sshd_config.d/pulzr-ca.conf |
TrustedUserCAKeys and AuthorizedPrincipalsFile |
/etc/ssh/auth_principals/root |
the gateway’s EUI, 16 lowercase hex |
/root/.ssh/authorized_keys |
the fleet key; on a development image also boards/common/ssh/dev/ and site keys (pulzr-fleet-keys) |
Rewritten from the image each boot rather than kept, because sysupgrade keeps
conffiles and /etc/ssh/sshd_config is one: a change shipped in a release
would otherwise never reach a gateway that had been upgraded once. The fleet
key stays in authorized_keys until every gateway trusts the CA. Pulzar Cloud
uses a certificate only for a gateway marked as trusting the CA, and only with
GATEWAY_SSH_CERTIFICATES set. Staff mark it today; the report carries
sshCa: true once the running sshd trusts the CA, for the cloud to mark it
itself.
Host keys. Pulzar Cloud pins the key a gateway reports, so the first boot
without dropbear converts dropbear’s keys (dropbearconvert) and sshd presents
the key already pinned. dropbear’s files stay, and keep.d carries both, so a
rollback to a dropbear image presents the same key again.
Checking one. On the gateway:
sshd -T | grep -Ei 'trustedusercakeys|authorizedprincipalsfile'cat /etc/ssh/auth_principals/rootlogread | grep pulzr-net # the key ID of every certificate loginAn image built without boards/common/ssh/pulzr_user_ca.pub gets no CA
configuration and is reached with the fleet key, as before.
Rotating the CA is a firmware release: put the new public line in
boards/common/ssh/, ship it, and only then switch recovery-service to the
new private key. A gateway trusts exactly one CA, so a gateway on the old
release refuses the new CA’s certificates until it upgrades.
The keys in this repo
Section titled “The keys in this repo”All public; nothing in boards/common/ssh/ is secret.
| Path | What it is |
|---|---|
fleet/*.pub |
baked into root’s authorized_keys (install_ops_keys in build.sh) |
pulzr_user_ca.pub |
the CA above; one ssh-ed25519 line, the operator’s SSH user CA, public half |
dev/*.pub |
people’s keys for a development image, ignored by git |
The CA is outside the fleet/ glob on purpose: in authorized_keys its
private half would open every gateway directly, and build.sh refuses an
image that puts it there.
fleet/pulzr-fleet.pub’s private half belongs to the platform operator, not a
person: the cloud holds it. Access follows whoever runs the platform, and no
laptop key ships in an image. It is RSA because gateways still on a dropbear
image accept nothing else. To use it locally:
ssh-keygen -t rsa -b 4096 -f fleet_ssh_rsa -N ''cp fleet_ssh_rsa.pub boards/common/ssh/fleet/pulzr-fleet.pub # your own deployment's keyssh -i fleet_ssh_rsa root@192.168.61.1Rotating it is a rebuild: a new pair, then reflash or push the new
authorized_keys over the management tunnel with the old key. No re-adoption.
PULZR_OPS_KEYS=0 ./build.sh <board> builds an image with no fleet key at
all, reached only by certificate.
Who gets root
Section titled “Who gets root”A root shell on PulzarOS is a copy of PulzarOS, so on a production image it
is Pulzar’s: the fleet key, the certificates Pulzar Cloud signs for staff, and
a console session Pulzar Cloud has signed as staff’s. An organization’s admin
runs their gateway through the console — the web console, the TUI on a port,
and Pulzar Cloud’s Console (pulzr-tui --remote, see
tui.md) — and never gets the shell under it. Before 0.3.1 root’s
password was the console’s, so anybody with the label was root.
| Production image | Development image | |
|---|---|---|
| Root’s password | none (*, pulzr-root-keys-only) |
none |
| SSH as root | fleet key, staff certificate | also boards/common/ssh/dev/*.pub, site keys |
| SSH with a password | refused | refused |
ssh pulzr@<gateway> |
the TUI as uid 1000, behind the console’s sign-in; no shell | the same |
| Serial console | the TUI, behind the console’s sign-in | the stock root shell |
| HDMI, USB serial | the TUI, behind the console’s sign-in | the same |
The TUI’s ! |
a staff session, or a TUI started from a root shell | any signed-in session |
| Failsafe mode | off | on |
| Site keys on the card | ignored, and removed from a gateway that has them | accepted until adopted |
ssh pulzr@<gateway> is how a technician on the site network reaches
the console, where the label’s password used to open a root shell. pulzr
is uid 1000 with no password of its own (pulzr-ssh-setup writes it every
boot), and sshd’s Match User pulzr block runs pulzr-tui --console
whatever the client asks for — no command, sftp, forwarding or ~/.ssh/rc.
So the account is the console’s sign-in and nothing else: the local password
while it is allowed, Pulzar or recovery once adopted. Sign-in failures count
against the client’s address, and Ctrl-C ends the session.
It is not root, because anybody on the site’s network reaches it before
signing in. Everything the TUI reads or changes there it asks
pulzr-console-broker for, over ubus (pulzr.console, which only root and
pulzr may call): before sign-in only what getting a session takes and the
claim screen of an unadopted gateway, after it the session’s own reads and
actions, each looked up by the broker from the session’s token. There is no
! shell on it; staff reach one with ssh root@ and a certificate, which
works offline too, and ! gives everybody else the diagnostic prompt.
Staff sessions. Organization admins sign in to a gateway’s console with
Pulzar and with recovery as well as staff do, so the kind of session cannot
say who it is. Pulzar Cloud says it, signed with the recovery key the gateway
already trusts: a terminal sign-in approved by staff carries shell: true in
its token, and staff recovering a gateway are given a signature over
<challenge>.shell instead of over the challenge. auth.uc records either as
the session’s shell; nothing a client sends can set it. The web console has
no shell, and a token for it cannot carry the mark.
Upgrading. pulzr-root-keys-only runs at every boot, so a gateway from
an older release loses root’s password on its first boot of this one. The
serial console moves to the TUI from the boot after that, because procd reads
/etc/inittab once; on the first boot system.ttylogin makes the stock login
ask for root’s password, which no longer exists.
A development image is PULZR_DEV=1 ./build.sh <board>. Its version
ends in -dev, which Pulzar Cloud shows, and CI refuses to build one, so a
release never is. Put your public key in boards/common/ssh/dev/ (ignored by git).
What this does not cover is the hardware. A Pi’s root filesystem is a plain squashfs on its SD card, and a RAK’s is in flash its U-Boot will hand to anybody on the serial header; either can be read by somebody holding the gateway. Closing that is secure boot and encrypted storage, which a Pi 4 or 5 can be made to do and the MT7628 cannot.