--- name: udx-go description: Use when writing or debugging Go code on go-udx (import github.com/stephanfeb/go-udx, package udx): Listen, Dial, Multiplexer, Connection and Stream over UDP, stream flow control, half-close, deadlines, idle timeouts, MockClock tests, or interop with dart_udx and js-udx over UDX wire v3. triggers: go-udx, github.com/stephanfeb/go-udx, udx.Listen, udx.Dial, udx.Multiplexer, NewMultiplexer, AcceptStream, OpenStream, ErrStreamReset, MaxDatagram, MockClock --- # go-udx ## What it is go-udx is the Go implementation of UDX, a QUIC-inspired reliable transport over UDP: independent ordered byte streams per connection, CUBIC congestion control with pacing, RFC 9002 loss recovery, per-stream flow control and path MTU discovery. It is the wire authority for UDX v3: dart_udx and js-udx are checked against it. It has no encryption. To run libp2p over UDX in Go, use go-libp2p-udx-transport (skill `go-libp2p-udx`) instead of using go-udx directly. ## Install ```sh go get github.com/stephanfeb/go-udx@v0.3.0 ``` ```go import udx "github.com/stephanfeb/go-udx" // package name is udx ``` go.mod declares `go 1.23.0`; the only direct dependency is `golang.org/x/net` (batched `recvmmsg`/`sendmmsg` on Linux). ## Core workflow `Multiplexer` owns one `net.PacketConn` and routes datagrams by destination connection ID. `Connection` is one peer. `Stream` is an `io.ReadWriteCloser`. ### Echo server ```go mux, err := udx.Listen("127.0.0.1:9000") if err != nil { return err } defer mux.Close() for { conn, err := mux.Accept(ctx) if err != nil { return err } // udx.ErrConnectionClosed after mux.Close go func(c *udx.Connection) { for { s, err := c.AcceptStream(ctx) if err != nil { return } go func(s *udx.Stream) { io.Copy(s, s) // returns at the peer's FIN (io.EOF) s.Close() // our FIN }(s) } }(conn) } ``` ### Client on a multiplexer you own ```go mux, err := udx.Listen("127.0.0.1:0") // also accepts, if you want if err != nil { return err } defer mux.Close() raddr, _ := net.ResolveUDPAddr("udp4", "127.0.0.1:9000") conn, err := mux.Dial(ctx, raddr) // sends the SYN, does not wait for a reply if err != nil { return err } defer conn.Close() s, err := conn.OpenStream(ctx) if err != nil { return err } if _, err := s.Write([]byte("hello")); err != nil { return err } // first Write carries the stream SYN s.CloseWrite() // FIN; reading continues reply, err := io.ReadAll(s) // ends at the peer's FIN ``` `udx.Dial(ctx, "host:port")` is a shortcut that binds a fresh socket and its own `Multiplexer`; see Gotchas before using it in long-lived code. ### Deadlines ```go s.SetReadDeadline(time.Now().Add(5 * time.Second)) n, err := s.Read(buf) if errors.Is(err, udx.ErrDeadlineExceeded) { /* retry or give up */ } s.SetWriteDeadline(time.Now().Add(5 * time.Second)) // Write blocks on flow control ``` ### Abort ```go s.Reset(42) // RESET_STREAM; peer's Read/Write get udx.ErrStreamReset conn.CloseWithError(0x100, "bye") // CONNECTION_CLOSE with an application code ``` ### Inspecting a connection ```go conn.MaxDatagram() // 1200 until PMTUD validates more (up to 1472) conn.StreamCount() conn.State() // udx.ConnStateEstablished, ... conn.RemoteAddr() // follows a peer that a NAT moved, once it answers PATH_CHALLENGE mux.ConnectionCount() ``` ## Key concepts / API map - `Listen(addr string, opts ...Option) (*Multiplexer, error)`: binds `"udp"` on addr. - `Dial(ctx, addr string, opts ...Option) (*Connection, error)`: new socket + multiplexer per call. - `NewMultiplexer(conn net.PacketConn, clk Clock) *Multiplexer`: wrap your own socket. - `WithClock(c Clock) Option`: inject a clock (tests). - `(*Multiplexer).Accept(ctx)`, `Dial(ctx, net.Addr)`, `Close()`, `Addr()`, `ConnectionCount()`. - `(*Connection).OpenStream(ctx)`, `AcceptStream(ctx)`, `Close()`, `CloseWithError(code, reason)`. - `(*Connection).Ping(ctx)`: sends a PING, fire-and-forget (does not wait for the ACK). - `(*Connection).InitiatePathMigration(ctx, newAddr)`: PATH_CHALLENGE, 5 s timeout. - `(*Connection).LocalAddr()`, `RemoteAddr()`, `State()`, `MaxDatagram()`, `StreamCount()`, `InboundDropped()`. - `(*Stream).Read`, `Write`, `Close`, `CloseWrite`, `Reset(code)`, `SetReadDeadline`, `SetWriteDeadline`, `State()`, `BufferedBytes()`, `RecvWindow()`. - Errors: `ErrConnectionClosed`, `ErrStreamReset`, `ErrStreamClosed`, `ErrWriteAfterClose`, `ErrDeadlineExceeded`, `ErrMaxStreams`, `ErrUnsupportedVersion`; protocol errors are `*UDXError{Code, Reason}`. - Codes: `ErrorNoError` 0, `ErrorInternalError` 1, `ErrorStreamLimitError` 2, `ErrorFlowControlError` 3, `ErrorProtocolViolation` 4, `ErrorInvalidMigration` 5, `ErrorConnectionTimeout` 6. - Constants: `SafeDatagramSize` 1200, `MaxDatagramSize` 1472, `MaxIdleTimeout` 30 s, `InitialMaxStreamData` 64 KiB, `MaxStreamRecvWindow` 4 MiB, `InitialMaxStreams` 100, `DefaultCIDLength` 8. - `VersionV3`, `SupportedVersions`, `IsVersionSupported`. - `Clock`, `RealClock{}`, `NewMockClock(start time.Time)` with `Advance(d)`. - `MetricsObserver`: optional hooks for cwnd, RTT and loss. ## Gotchas 1. **v3 is a flag day.** v3 STREAM frames carry an 8-byte offset where v2 had the length, so a v2 peer would accept garbage. Packets of any other version are dropped, so a mismatch looks like an unreachable peer. v1/v2 in `SupportedVersions` are for negotiation reports only. 2. **`OpenStream` sends nothing.** The stream's SYN rides its first `Write`. A peer blocked in `AcceptStream` waits until you write, and closing a stream you never wrote to sends a bare FIN that the peer drops. In a protocol where the acceptor speaks first, write something first. 3. **`Close()` on a Stream is a half-close.** It sends FIN with the final size and leaves reading open; `CloseWrite()` is the same call. The stream is done when both sides have sent FIN. Dart's `UDXStream.close()` is a full close and JS's is `end()`; do not assume the names match. 4. **`udx.Dial` leaks its socket.** It creates a `Multiplexer` per call, and `Connection.Close` only removes the connection from it: the UDP socket and its read and idle goroutines stay. In servers, tools that dial often, and tests, use `Listen` (or `NewMultiplexer`) once, `mux.Dial` per peer, and `mux.Close()` at the end. 5. **Dials do not wait for the peer.** `Dial` returns once the SYN is sent; a peer that never answers is detected only by the idle timeout (30 s), when the connection closes and blocked calls return `ErrStreamReset` or `ErrConnectionClosed`. Put your own timeout on the first read. 6. **Idle connections close after `max(30 s, 3 x PTO)` without a packet received.** The close is silent (no CONNECTION_CLOSE) and resets every stream. go-udx sends no keep-alives; `Ping` is fire-and-forget. Keep connections busy from above (yamux keep-alive) or write periodically. 7. **The stream limit is per connection lifetime.** Closed streams are never removed from the connection's table, and `OpenStream` fails with `ErrMaxStreams` once `len(streams)` reaches the peer's MAX_STREAMS (100 by default). Open a new connection for workloads with many short streams. 8. **Accept queues drop when full.** `Accept` and `AcceptStream` sit on 16-slot channels; when one is full a new connection is dropped (logged) and a new stream is silently not delivered. Keep an accept loop running. 9. **Connection-level flow control is accounted, not enforced** (`flow_control.go`, doc/PENDING_WORK.md section 3). Only per-stream windows bind. Do not "fix" this without also sending MAX_DATA, or the stall moves from 64 KiB to 1 MiB. 10. **Stream windows are absolute offsets.** WINDOW_UPDATE carries `consumed + window` modulo 2^32 and the sender reconstructs the 64-bit value. A receiver that does not `Read` stops the sender after one window; that is the back-pressure. A blocked writer sends STREAM_DATA_BLOCKED every 500 ms because WINDOW_UPDATE is never retransmitted. 11. **Datagrams start at 1200 bytes** (v0.1.4). Carrier tunnels (464XLAT, GTP) drop larger unfragmentable datagrams, which stalled every multi-packet reply. PMTUD (v0.3.0) probes up to 1472 and drops back to 1200 on a black hole (three full-size losses in a row, or three PTOs). Do not raise the base. 12. **Retransmits use fresh sequence numbers and never give up** (v0.2+). Recovery is RFC 9002: ACK-driven loss, one PTO per connection, persistent congestion collapses the window. A dead path ends by idle timeout, not by a retry count resetting a stream. 13. **Only data-bearing packets are numbered.** STREAM with data, SYN or FIN take a sequence number and are ACKed. go-udx's own control packets go out with sequence 0 and are not ACKed. A numbered PING or MTU_PROBE from dart or js is ACKed at once (fixed in v0.1.3, `fix/ack-pings`); older go-udx made their `ping()` always fail and capped their PMTUD. 14. **Dual-stack sockets and batching.** `udx.Dial` binds an unspecified `"udp"` socket; batched writes from a dual-stack socket to IPv4 peers were silently lost before `fix/dual-stack-batch-writes`, and batching is now off on IPv6/dual-stack sockets. Bind `udp4` or `udp6` explicitly when you control the socket. 15. **Stream ids are local.** The dialer opens odd ids, the acceptor even ids. An opener never learns the peer's id and addresses its stream as 0; a bare FIN or RESET for an unknown stream does not open one, data or SYN does. 16. **Several connections may share one peer address.** Routing is by CID, so a hole punch (both sides dialling at once) yields two connections, one each way. dart_udx before 4.0.0 folded them into one and broke the punch. 17. **`cmd/interop-server` is a raw packet echo**, not a UDX endpoint. Its header says v2; do not use it as a reference for the current wire format. 18. **Reordering costs throughput.** Heavy reordering (25% of packets late) makes loss detection cut the window about twice as often as needed in all three stacks. netem retransmit percentages are noisy; use the seeded `loss_detection_test.go` to measure. ## Testing - Loopback pair (from `transfer_test.go`): ```go func testPair(t *testing.T) (*udx.Connection, *udx.Connection, context.Context) { a, _ := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}) b, _ := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}) server := udx.NewMultiplexer(a, udx.RealClock{}) client := udx.NewMultiplexer(b, udx.RealClock{}) t.Cleanup(func() { client.Close(); server.Close() }) ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) t.Cleanup(cancel) c, err := client.Dial(ctx, server.Addr()) if err != nil { t.Fatal(err) } s, err := server.Accept(ctx) // returns once the SYN arrives if err != nil { t.Fatal(err) } return c, s, ctx } ``` - Move enough data to matter (megabytes, with a slow reader): a 256 KB per-stream stall once reached production because every test moved a few bytes. - Always run with `-race`: `go test -race ./...`. - `udx.NewMockClock(time.Now())` with `Advance` drives timers in unit tests; `WithClock` passes a clock to `Listen` and `Dial`. - The package tests use a lossy/rebinding UDP proxy (`rebind_test.go`, `pmtud_test.go`) and a seeded loss simulator (`loss_detection_test.go`). - Interop: `go test -v ./interop/` runs against a Dart peer in `interop/dartpeer` (skips if `dart` is not on PATH). Set `UDX_BULK_PEER="node /tools/bulk-peer/bulk-peer.mjs"` to run the bulk and multi-stream suites against js-udx instead. ## Red lines - Never change the wire format, frame type codes or version in go-udx without the matching dart_udx and js-udx change, and regenerate js-udx's conformance vectors (`npm run vectors` in js-udx). - Never reuse a packet sequence number, and never add a retry cap that resets streams; the idle timeout is the backstop. - Never send stream data in datagrams above `SafeDatagramSize` before PMTUD validates the path. - Never block the read loop or hold a stream lock while calling into the connection; the code releases `s.mu` before sending for that reason. - Close every `Multiplexer` you create, including the hidden one behind `udx.Dial`. - UDX is unauthenticated plaintext. Put Noise (or another secure channel) on top before carrying anything sensitive. ## Reference - Docs: https://cryptopeer.net/udx/go/ and https://cryptopeer.net/udx/ - Repo: https://github.com/stephanfeb/go-udx (doc/PENDING_WORK.md lists the known gaps) - Related skills: udx-dart, udx-js, go-libp2p-udx