Why Kitty's SSH kitten fails in multi-client zmx sessions

I found two related failures while running Kitty’s SSH kitten from a persistent zmx session. Reattaching through another Kitty window produced a request-ID rejection. Attaching two Kitty windows to the same zmx session also corrupted the bootstrap archive sent to the remote host.

The observed failures

A normal connection sometimes worked:

[t] fran@mamini:~ ssh hetzner
deploy@debian-4gb-fsn1-1:~$ exit
logout
Shared connection to 116.202.105.36 closed.

Later attempts from the same zmx session failed in two ways:

base64: invalid input
bootstrap-utils.sh: Syntax error: end of file unexpected (expecting ";;")

Incorrect request id_ _39455_11_ expecting the
KITTY_PID_KITTY_WINDOW_ID for the current kitty window
Shared connection to 116.202.105.36 closed.

Direct OpenSSH remained reliable:

command ssh hetzner

The local alias starts the kitten

My shell configuration explicitly wraps interactive SSH inside Kitty:

if [[ -n "$KITTY_PID" ]] && _have kitten; then
  alias ssh='kitten ssh'
  alias sshraw='command ssh'
fi

Kitty’s shell integration does not create this alias. My configuration makes ssh hetzner run kitten ssh hetzner.

The SSH kitten copies shell integration, terminfo, and selected environment data to the remote host. It sends that data through a DCS escape-sequence handshake with the Kitty window.

The request belongs to one Kitty window

The Kitty 0.48.2 SSH source constructs a request ID from two environment variables:

cd.request_id = os.Getenv("KITTY_PID") + "-" + os.Getenv("KITTY_WINDOW_ID")

The receiving Kitty window validates the request against its actual process and window IDs:

for line in get_ssh_data(msg, f'{os.getpid()}-{self.id}'):
    self.write_to_child(line)

A zmx session keeps the environment inherited when its persistent shell starts. A session created in Kitty process 39455, window 11, therefore keeps this identity:

KITTY_PID=39455
KITTY_WINDOW_ID=11

Reattaching that session from another Kitty window does not replace the persistent shell’s environment. The kitten still sends 39455-11, while the newly attached window expects its own window ID. Kitty rejects the request intentionally.

Kitty sanitizes punctuation before writing the error through the terminal. The original request ID '39455-11' consequently appears as _39455_11_.

Multiple clients answer the same request

zmx supports multiple clients per session. Its daemon reads live output from the persistent PTY and broadcasts the same payload to every attached client.

That behavior creates this path when the PTY emits Kitty’s SSH DCS request:

persistent zmx shell: request id 39455-11
                         |
                  zmx broadcasts
                  /            \
       Kitty window 11      Kitty window N
       archive response     rejection response
                  \            /
                    shared PTY

Each Kitty window handles the request and writes its response to its child, which is that window’s zmx client. Those clients send the responses back to the same persistent PTY.

zmx normally routes complete input from one leader client. A non-leader client can take leadership when isUserInput sees printable text, carriage returns, or line feeds. Kitty’s automated response contains all three. The input handler therefore changes leaders while the handshake is in progress and accepts data from competing clients.

The local zmx log captured that sequence after the second terminal attached:

client connected fd=11 total=3
client disconnected fd=10 remaining=2
KITTY_DATA_START
Incorrect request id_ _39455_11_ expecting the current kitty window
setting new leader client_fd=9
KITTY_DATA_START
_Errno 2_ No such file or directory_ _kssh_..._

The temporary fd=10 connection was zmx’s session probe. Two terminal clients remained as fd=9 and fd=11, and leadership moved between them during the Kitty response.

Kitty stores the archive in one-time shared memory. Its reader unlinks that object when opening it. One responding window can consume the object before another window opens it, producing the observed Errno 2 alongside the request-ID failure.

Why the secondary errors differ

The remote bootstrap expects one stream with this shape:

KITTY_DATA_START
OK
<base64-encoded archive>
KITTY_DATA_END

Competing responses can insert another start marker, an error message, or archive chunks into that stream. The downstream symptoms depend on where the streams overlap:

• base64: invalid input means the decoder received text outside the archive alphabet or an incomplete archive.
• expecting ";;" means the extracted bootstrap-utils.sh ended inside a shell case branch.
• tic: cannot open ... kitty.terminfo means the partial archive lacked the terminfo source file.

These are consequences of the damaged handshake stream. They do not identify independent problems with Debian, base64, or tic.

The source trace also points away from ghostty-vt as the byte-corruption mechanism. zmx feeds PTY output to ghostty-vt for state tracking, applies a targeted rewrite to OSC 133 prompt markers, then broadcasts the live output. Incremental 4096-byte PTY reads preserve byte order. The demonstrated timing sensitivity comes from multiple Kitty windows racing to answer one request.

The successful attempts fit the same model. A newly created zmx session has a matching window ID, and one attached client produces one clean response. Reattaching from another window makes the stored ID stale. Adding a second client introduces competing responders.

Shared connection to ... closed is informational. Kitty’s SSH kitten enables connection sharing by default through OpenSSH ControlMaster. Closing one remote shell closes its logical channel while the shared master can remain alive.

Bypassing the handshake inside zmx

The shell’s command builtin skips the ssh alias and runs OpenSSH directly:

command ssh hetzner
# Equivalent alias already present in my config:
sshraw hetzner

This removes Kitty’s remote shell integration and automatic terminfo transfer for that connection. It also removes the window-scoped DCS handshake that conflicts with persistent, multi-client zmx sessions.

I can make that distinction automatic by enabling the kitten alias only outside zmx:

if [[ -n "$KITTY_PID" ]] && _have kitten; then
  alias sshraw='command ssh'
  [[ -z "$ZMX_SESSION" ]] && alias ssh='kitten ssh'
fi

Existing zmx shells retain aliases loaded at startup. They need unalias ssh or recreation after this configuration change.

Upstream direction and my current decision

After tracing the failure, I found zmx issue #225. It reports the same persistent-environment problem for KITTY_PID, KITTY_WINDOW_ID, KITTY_LISTEN_ON, and SSH_AUTH_SOCK. The maintainer described per-client environment handling as the project’s most important missing feature and preferred exposing the leader client’s environment through get-env and print-env commands.

That proposal would address the stale-identity half of this incident. The shell could refresh Kitty’s variables from the client that most recently supplied user input before starting the SSH kitten.

The response fan-out requires a separate change. Current zmx main still broadcasts PTY output to every client. Reliable handling would need to recognize Kitty’s private SSH DCS on the output path and deliver it only to the leader. Filtering input occurs after every attached Kitty has already seen the request and attempted to consume its one-time shared memory.

A maintained fork could combine per-client environments with leader-only DCS routing. I decided against carrying that fork for now. zmx is active, its current main branch has moved to Zig 0.16, and the simple OpenSSH workaround avoids both failure modes. The project’s documented SSH workflow also places SSH outside a remote zmx session, while my failing path placed the SSH kitten inside a local zmx session.

For now, I use direct OpenSSH inside zmx and keep the SSH kitten available in ordinary Kitty shells. I can revisit the choice if upstream per-client environment handling lands and the DCS routing requirement changes.

References

• Kitty SSH kitten documentation. Describes shell integration, terminfo transfer, connection sharing, and delegated hosts.
• Kitty 0.48.2 request construction. Builds the window-scoped request and shared-memory payload.
• Kitty 0.48.2 request validation. Validates the request ID and emits the archive stream.
• zmx 0.7.0 daemon loop. Reads PTY output and broadcasts it to attached clients.
• zmx 0.7.0 client leadership. Routes input and changes the leader when another client sends user-like input.
• zmx issue #225. Tracks per-client environment updates and records the maintainer’s preferred get-env and print-env direction.
• zmx current main output broadcast. Confirms that PTY output still reaches every attached client after the v0.7.0 refactor.
• zmx SSH workflow. Documents SSH into a remote zmx session with OpenSSH connection sharing.
• POSIX command. Defines command lookup without shell functions on the normal utility search path.