TLS internals

ioxide.tls terminates TLS with OpenSSL in both directions by default: the handshake runs over memory BIOs, inbound records decrypt in userspace, and responses are encrypted before they reach the write slab. Kernel TLS is a per-direction opt-in (TlsOptions.KernelTx / KernelRx) for the deployments where sendfile or NIC offload pays for its constraints.

TlsService · TlsService.cs

Per-reactor TLS termination, holding one SSL_CTX shared across that reactor's connections (created in OnStart). By default the context keeps OpenSSL's own posture - TLS 1.2 and 1.3, any ciphersuite, session tickets on. Only when KernelTx opts into kTLS is it pinned to TLS 1.3 with TLS_AES_128_GCM_SHA256 and tickets disabled: the kernel needs a known key layout, and a ticket would advance the record sequence after the handshake and break the handoff. Asking for KernelRx without KernelTx is refused at Start - RX shares the ULP the TX handoff installs.

The handshake

AcceptAsync drives a standard memory-BIO server handshake as an ordinary reactor await: SSL_accept → read its error immediately → drain the write BIO into the connection's write slab and FlushAsync → on WANT_READ, recv more ciphertext and feed the read BIO - loop until done. Handshake bytes ride the same ring recv/send as application data, so there's no separate I/O path.

TlsService.cs · the handshake as an ordinary reactor await loop

while (true)
{
    int ret = OpenSsl.SSL_accept(ssl);
    int err = ret == 1 ? 0 : OpenSsl.SSL_get_error(ssl, ret);   // read the error immediately

    await FlushOutbound(conn, wbio);    // drain server flights into the slab, then io_uring send

    if (ret == 1) break;
    if (err != OpenSsl.SSL_ERROR_WANT_READ)
        throw new IOException($"TLS handshake failed: {OpenSsl.LastError()}");

    RecvSnapshot snapshot = await conn.ReadAsync();   // WANT_READ: pull more ciphertext
    bool fed = FeedInbound(conn, rbio, snapshot);
    conn.ResetRead();

    if (snapshot.IsClosed && !fed)
        throw new IOException("connection closed during TLS handshake");
}

Per-session secret via ex_data (no global map)

kTLS keys are derived from the TLS 1.3 traffic secrets (server for TX, client for RX), which OpenSSL surfaces through a keylog callback. The callback is a static function pointer with no instance, so to find the right session it reads a GCHandle to the TlsSession stored on the SSL via SSL_set_ex_data / SSL_get_ex_data (the index allocated once with CRYPTO_get_ex_new_index), and stores the secret on that session directly - no process-global, recycled-pointer-keyed map, so two reactors that get the same recycled SSL address can't collide. On every path that does not program the kernel the secrets are zeroed at the handoff decision, and again on TlsSession.Dispose, which also frees the handle.

TlsService.cs · one ex_data index, allocated once; each SSL* carries a GCHandle to its session

private static readonly int SslSessionIndex =
    OpenSsl.CRYPTO_get_ex_new_index(OpenSsl.CRYPTO_EX_INDEX_SSL, 0, 0, 0, 0, 0);

// In AcceptAsync, bind this SSL* to its managed session:
GCHandle handle = GCHandle.Alloc(session);
OpenSsl.SSL_set_ex_data(ssl, SslSessionIndex, GCHandle.ToIntPtr(handle));
session.AttachHandle(handle);

TlsService.cs · the static keylog callback resolves its session through ex_data - no global map

[UnmanagedCallersOnly]
private static void KeylogCallback(nint ssl, nint line)
{
    string? text = Marshal.PtrToStringUTF8(line);
    if (text == null)
        return;

    bool server = text.StartsWith("SERVER_TRAFFIC_SECRET_0 ", StringComparison.Ordinal);
    bool client = text.StartsWith("CLIENT_TRAFFIC_SECRET_0 ", StringComparison.Ordinal);
    if (!server && !client)
        return;

    nint data = OpenSsl.SSL_get_ex_data(ssl, SslSessionIndex);
    if (data == 0 || GCHandle.FromIntPtr(data).Target is not TlsSession session)
        return;

    int lastSpace = text.LastIndexOf(' ');
    if (lastSpace > 0)
    {
        session.ServerSecret = Convert.FromHexString(text.AsSpan(lastSpace + 1).TrimEnd());
    }
}

ALPN

The configured TlsOptions.Alpn is honored: the wire bytes are passed to the (static) select callback via its arg (a GCHandle), so the protocol isn't hard-coded. The callback finds it in the client's offer list and points *out into the client buffer.

The kTLS handoff (opt-in)

With KernelTx on, once the handshake is done and all server flights are flushed, Ktls.EnableTx programs the TX keys into the socket and conn.SendOpFlags = 0 clears MSG_WAITALL (kTLS rejects it; the reactor's partial-send loop preserves correctness). From the next write on the kernel produces the records - which is what makes the kTLS samples' bare conn.Write legal there and nowhere else. TlsSession.Write is correct in either mode, which is why every other sample goes through it. With KernelRx also on, the RX keys are programmed at the same moment, with the record sequence advanced past whatever the handshake already consumed - and a connection whose handshake left a partial record behind silently keeps the userspace reader, because those bytes are gone for the kernel.

TlsService.cs · the handoff - program TX keys, then drop MSG_WAITALL

if (_kernelTx)
{
    Ktls.EnableTx(conn.ClientFd, secret);
    conn.SendOpFlags = 0;                // clear MSG_WAITALL - kTLS rejects it (EOPNOTSUPP)
    session.MarkTxEnabled(conn.ClientFd);
}

TlsSession · TlsSession.cs

The per-connection TLS state after the handshake (reactor-thread only). Decrypt(ciphertext, length) feeds the read BIO and drains SSL_read into a reusable plaintext buffer, returning the span (valid until the next call); under kTLS RX it is a pass-through, since recv already delivered plaintext. Write is the mode-aware outbound: plaintext straight to the slab under kTLS TX, encrypted through the write BIO otherwise. SSL_ERROR_ZERO_RETURN sets Closed (the peer's close_notify). On a clean server-side teardown, Dispose sends close_notify in whichever mode applies - a kTLS control send, or SSL_shutdown's record delivered with a best-effort raw send - zeroes any remaining secrets, then SSL_frees (which frees both BIOs) and releases the ex_data handle.

Ktls · Ktls.cs

The kernel-TLS transmit offload. EnableTx derives the record key (16B) and IV (12B) from the server traffic secret with RFC 8446 HKDF-Expand-Label (the 12-byte TLS 1.3 nonce is split into the kernel's salt[4] + iv[8]; the record sequence stays 0, valid because tickets are off), then setsockopt(SOL_TCP, TCP_ULP, "tls") attaches the ULP and setsockopt(SOL_TLS, TLS_TX, &cryptoInfo) installs the keys. All key material - the derived key/iv, the crypto-info struct, and the captured secret - is zeroed in a finally with CryptographicOperations.ZeroMemory once programmed. EnableRx is the same dance for the receive keys, taking the record sequence the handshake already consumed (the sequence is part of the AEAD nonce, so it cannot be guessed). SendCloseNotify sends a {warning, close_notify} alert as a control message (sendmsg with a SOL_TLS / TLS_SET_RECORD_TYPE = alert cmsg), best-effort.

Ktls.cs · derive the AES-128-GCM key/iv, then attach the ULP and install the TX keys

public static unsafe void EnableTx(int fd, byte[] serverTrafficSecret)
{
    byte[] key   = ExpandLabel(serverTrafficSecret, "key", 16);   // RFC 8446 HKDF-Expand-Label
    byte[] nonce = ExpandLabel(serverTrafficSecret, "iv", 12);

    var info = new CryptoInfoAesGcm128
    {
        Version = TLS_1_3_VERSION,
        CipherType = TLS_CIPHER_AES_GCM_128,
    };
    try
    {
        for (int i = 0; i < 16; i++) info.Key[i]  = key[i];
        for (int i = 0; i < 4;  i++) info.Salt[i] = nonce[i];      // 12-byte nonce → salt[4] + iv[8]
        for (int i = 0; i < 8;  i++) info.Iv[i]   = nonce[4 + i];

        ReadOnlySpan<byte> ulp = "tls"u8;
        fixed (byte* p = ulp)
        {
            if (setsockopt(fd, SOL_TCP, TCP_ULP, p, 3) != 0)
                throw new IOException($"kTLS: TCP_ULP failed (errno {Marshal.GetLastPInvokeError()})");
        }
        if (setsockopt(fd, SOL_TLS, TLS_TX, &info, (uint)sizeof(CryptoInfoAesGcm128)) != 0)
            throw new IOException($"kTLS: TLS_TX failed (errno {Marshal.GetLastPInvokeError()})");
    }
    finally
    {
        CryptographicOperations.ZeroMemory(key);                  // wipe key material once programmed
        CryptographicOperations.ZeroMemory(nonce);
        CryptographicOperations.ZeroMemory(serverTrafficSecret);
    }
}

OpenSsl · OpenSsl.cs

The minimal OpenSSL 3 P/Invoke surface (via [LibraryImport] source-gen): context setup (SSL_CTX_new, protocol/ciphersuite/ticket controls, cert/key file loads, the keylog and ALPN callback setters), the per-connection objects (SSL_new/_free, SSL_set_bio/_accept_state/_accept/_read/_write/_shutdown/_get_error), the ex_data accessors, and the memory BIOs (BIO_new/_s_mem/_read/_write/_ctrl_pending). LastError formats the first queued error and drains the rest so the next report isn't stale.

TlsOptions · TlsOptions.cs

The PEM certificate-chain path, the private-key path, the ALPN protocols to advertise (most preferred first), and the two kernel-offload opt-ins: KernelTx (kTLS transmit - constrains the context to TLS 1.3, one suite, no tickets) and KernelRx (experimental kernel receive; requires KernelTx).

An alternative, fully-managed path exists with no native dependency: a BCL SslStream over TcpConnectionStream (~0.65x the ring-native path, but the only one that does client certificates) - see the tls-sslstream example.

ioxide.file internals →