--- name: dart-libp2p-bitswap description: Use when fetching or serving content-addressed blocks between dart_libp2p peers with package dart_libp2p_bitswap (Bitswap 1.2.0): implementing a BlockStore, getBlock / wantBlocks / cancelWants, notifyNewBlock, BitswapOptions tuning, CIDs via dcid, and testing Bitswap code. triggers: dart_libp2p_bitswap, bitswap, Bitswap 1.2.0, /ipfs/bitswap/1.2.0, BlockStore, SizedBlockStore, BitswapOptions, getBlock, wantBlocks, notifyNewBlock, WANT_HAVE, DONT_HAVE, dcid --- # dart_libp2p_bitswap ## What it is A Dart implementation of the Bitswap 1.2.0 block exchange protocol (`/ipfs/bitswap/1.2.0`) on top of a `dart_libp2p` `Host`. It fetches blocks by CID from connected peers and serves blocks from a `BlockStore` you provide. Behaviour (want strategy, limits, defaults) follows boxo, the Go implementation used by kubo. It only moves raw blocks between peers that are already connected. It does not find providers (use the DHT skill, `dart-libp2p-kad-dht`, then connect), does not walk DAGs (your code decodes a block and asks for its links), and does not set up the host (see the `dart-libp2p-host` skill). For replicating Merkle-CRDTs, see `merkle-crdt-sync`. ## Install ```yaml dependencies: dart_libp2p_bitswap: ^2.0.0 dart_libp2p: ^4.12.0 dcid: ^1.0.0 ``` Dart SDK `^3.5.0`. Imports: ```dart import 'dart:typed_data'; import 'package:dart_libp2p/dart_libp2p.dart'; import 'package:dart_libp2p_bitswap/dart_libp2p_bitswap.dart'; import 'package:dcid/dcid.dart' as dart_cid; ``` The single library `dart_libp2p_bitswap.dart` exports `Bitswap`, `BitswapOptions`, `BlockStore`, `SizedBlockStore` and the generated protobuf types (`Message`, `Message_Wantlist`, `Message_Wantlist_Entry`, `Message_Block`, `Message_BlockPresence`, `Message_Wantlist_WantType`, `Message_BlockPresenceType`). Alias it if those names clash with yours. ## Core workflow ### 1. A BlockStore ```dart class InMemoryBlockStore implements BlockStore { final Map _blocks = {}; @override Future has(dart_cid.CID cid) async => _blocks.containsKey(cid.toString()); @override Future get(dart_cid.CID cid) async => _blocks[cid.toString()]; @override Future put(dart_cid.CID cid, Uint8List data) async => _blocks[cid.toString()] = data; } ``` Bitswap calls `has`/`get` to serve peers and `put` to store blocks it fetched (only blocks on its wantlist, after re-hashing them against their CID). ### 2. Start, fetch, stop (two hosts) ```dart // provider and fetcher are started Hosts (see dart-libp2p-host) const options = BitswapOptions(rebroadcastInterval: Duration(seconds: 1)); final providerStore = InMemoryBlockStore(); final providerBitswap = Bitswap(host: provider, blockStore: providerStore, options: options); final fetcherBitswap = Bitswap(host: fetcher, blockStore: InMemoryBlockStore(), options: options); await providerBitswap.start(); await fetcherBitswap.start(); final data = Uint8List.fromList('Hello from Bitswap!'.codeUnits); final cid = dart_cid.CID.fromData(dart_cid.CID.V1, 'raw', data); // sha2-256 await providerStore.put(cid, data); await fetcher.connect(AddrInfo(provider.id, provider.addrs)); final received = await fetcherBitswap.getBlock(cid, timeout: const Duration(seconds: 15)); if (received == null) { // timed out or cancelled } await fetcherBitswap.stop(); await providerBitswap.stop(); ``` This mirrors `example/dart_libp2p_bitswap_example.dart` in the repo (UDX, Noise, Yamux, `Libp2p.addrsFactory((addrs) => addrs)` so loopback addresses are kept): `dart run example/dart_libp2p_bitswap_example.dart`. ### Prefetch several blocks without waiting ```dart await bitswap.wantBlocks([cid1, cid2, cid3]); // stored via put() on arrival // later final b1 = await bitswap.getBlock(cid1); // local hit if already arrived bitswap.cancelWants([cid3]); // sends CANCEL; pending getBlock(cid3) -> null ``` ### Add a block from the application If your code writes into the store itself (not via Bitswap), tell Bitswap so peers waiting for it are served and local `getBlock` waiters complete: ```dart await store.put(cid, bytes); bitswap.notifyNewBlock(cid, bytes); // bytes are trusted to match cid ``` ### Parse a CID received from elsewhere ```dart final cid = dart_cid.CID.fromString( 'bafkreibm6jg3ux5qumhcn2b3flc3tyu6dmlb4xa7u5bf44yegnrjhc4yeq'); final bytesCid = dart_cid.CID.fromBytes(cid.toBytes()); ``` ### Size-aware store ```dart class MyStore implements SizedBlockStore { // has / get / put as above, plus: @override Future getSize(dart_cid.CID cid) async => /* size or null */ null; } ``` With a `SizedBlockStore`, Bitswap decides how to answer `WANT_HAVE` without reading the block, and fills `pendingBytes` in responses. ### Logging The library prints nothing by default. Loggers: `bitswap`, `bitswap.client`, `bitswap.client.wantlist`, `bitswap.server`, `bitswap.network`, `bitswap.packing`. ```dart import 'package:logging/logging.dart'; hierarchicalLoggingEnabled = true; Logger('bitswap').level = Level.FINE; Logger('bitswap').onRecord.listen( (r) => print('${r.level.name} ${r.loggerName}: ${r.message}')); ``` `WARNING` = problems (rejected blocks, dropped oversize blocks, no Bitswap peers), `FINE` = protocol activity, `FINEST` = per-chunk stream I/O. ## Key concepts / API map - `Bitswap({required Host host, required BlockStore blockStore, BitswapOptions options})`: facade over client (fetching) and server (serving). No side effects until `start()`. - `Future start()`: registers the `/ipfs/bitswap/1.2.0` stream handler and subscribes to `EvtPeerConnectednessChanged` on `host.eventBus`. Idempotent. - `Future stop()`: removes the handler, closes streams, fails pending `getBlock` with `StateError`. Idempotent; cannot be restarted. - `Future getBlock(CID cid, {Duration? timeout})`: local store first, else want it and wait. `null` on timeout or cancellation. - `Future wantBlocks(List cids)`: add to wantlist and broadcast. - `void cancelWants(List cids)`: drop wants, send CANCEL. - `void notifyNewBlock(CID cid, Uint8List data)`: announce an app-stored block. - `BlockStore`: `has`, `get`, `put` (all async). - `SizedBlockStore implements BlockStore`: adds `Future getSize(CID)`. - `BitswapOptions` (all optional, `const`): - `getBlockTimeout` = 30 s - `rebroadcastInterval` = 10 s - `maxWantsPerPeer` = 1024 (wants beyond this from one peer are ignored) - `serveBytesPerTurn` = 1 MiB (round-robin fairness between peers) - `wantHaveReplaceSize` = 1024 bytes (send the block instead of `HAVE`; 0 disables) - `handleIncomingMessage(PeerId, Message)` and `onPeerDisconnected(PeerId)`: `@visibleForTesting` hooks for injecting protocol messages in unit tests. Want strategy (1.2.0, as boxo): each CID is sent as `WANT_BLOCK` to one peer and `WANT_HAVE` to all other Bitswap peers, always with `sendDontHave`. On `DONT_HAVE` or disconnect, the `WANT_BLOCK` moves to a peer that said `HAVE`. After the first full wantlist, later updates to a peer are deltas. When a block arrives, every peer that was asked gets `CANCEL`. Peers that receive wants: peers that have sent us a Bitswap wantlist, plus connected peers whose peer-store protocols (from identify) include `/ipfs/bitswap/1.2.0`. ## Gotchas - **Call `start()` first.** `getBlock`, `wantBlocks` and `notifyNewBlock` throw `StateError('Bitswap not started; call start() first')` before `start()` and `StateError('Bitswap stopped')` after `stop()`. A stopped instance cannot be restarted: build a new `Bitswap`. - **No provider discovery.** Bitswap only asks peers you are already connected to. If no connected peer speaks Bitswap it logs `Found no Bitswap-supporting peers.` and `getBlock` waits until timeout. Find providers through the DHT and `host.connect(...)` to them first. - **Identify race on fresh connections.** Wants are only sent to a connected peer once identify has recorded that it speaks `/ipfs/bitswap/1.2.0`. A `getBlock` made right after `connect` may not go out until the next rebroadcast (default 10 s). In tests and latency-sensitive apps set a short `rebroadcastInterval` (the example uses 1 s), or make sure identify has completed before asking. - **Loopback hosts cannot dial each other by default.** dart_libp2p's default address factory drops loopback addresses; for local tests pass `Libp2p.addrsFactory((addrs) => addrs)` when building the host. - **`getBlock` returns `null` only on timeout or `cancelWants`.** Other errors (including `StateError` from `stop()`) propagate, so wrap calls that may race with shutdown. Concurrent `getBlock` for the same CID share one want; one caller timing out does not cancel it for the others, but when the last waiter gives up, CANCEL is sent. - **Unsolicited blocks are discarded.** Since 2.0.0 a block is stored only if it is on the local wantlist. Pushing blocks at a peer that did not ask does nothing; the receiver must `getBlock`/`wantBlocks` first. - **Call `notifyNewBlock` after writing to the store yourself.** Without it, peers that wanted the block before you had it are not served until they re-send their want, and local `getBlock` waiters keep waiting. `notifyNewBlock` does not check the hash: never pass data that does not match the CID. - **CIDs match by exact string.** The wantlist and peer ledgers key on `cid.toString()`. A CIDv0 (`Qm...`) and its CIDv1 (`bafy...`, dag-pb) are different wants, and the reference store above treats them as different blocks. The CID of a received block is rebuilt from the sender's prefix, so ask for the version the provider stores. `cid.toV1()` / `cid.toV0()` (`dcid`) convert; `toV0()` throws `FormatException` unless the CID is dag-pb + sha2-256. Key your store by multihash if you need both forms to hit. - **Supported hashes only.** Received blocks are re-hashed using the prefix: sha2-256/512, sha3-224/256/384/512, keccak-256/512, blake2b-256/512, blake3 (1 to 128-byte digests) and identity (up to 128 bytes). Anything else is rejected with a warning. CIDv0 must be dag-pb + sha2-256 + 32 bytes. Data that does not match produces a different CID and is dropped as unwanted. - **4 MiB message limit, both ways.** Inbound messages declaring more than 4 MiB reset the stream. Outbound responses are split into messages of at most 4 MiB, and a single block too big for one message is dropped with a warning. Keep blocks small (the spec recommends at most 2 MiB; IPFS chunks at 256 KiB to 1 MiB). There is no explicit block-size check on `put`. - **Only 1.2.0 is spoken.** The only protocol ID registered or dialled is `/ipfs/bitswap/1.2.0`; there is no fallback to 1.0.0 / 1.1.0, so peers that only speak older versions will not exchange blocks with this package. - **Go interop is untested.** Wire format, CIDv0 prefix layout (`0x00 0x70 0x12 0x20`), limits and defaults follow go-bitswap / boxo, and CIDv0 blocks from go-bitswap are accepted (2.0.0 fix), but the README states interop with Go nodes (kubo) has not been tested end to end. Verify before relying on it. - **Received `pendingBytes` is ignored** (as in boxo). It is only set on outgoing responses when the store is a `SizedBlockStore`. - **Store failures.** If `put` throws, the error is logged and the want stays open for another delivery. A failing `get` for one want does not stop other wants being served. Do not rely on exceptions from your store surfacing in `getBlock`. - **Upgrading from 1.x.** `IBlockStore` is now `BlockStore`; the `peerEventsStream` constructor argument is gone (Bitswap subscribes to `host.eventBus` itself); the CID package is `dcid`, not `dart_cid`. Older docs and snippets that use these names are out of date. ## Testing - **Real hosts in one process**: copy the pattern from `example/dart_libp2p_bitswap_example.dart`: two hosts on `/ip4/127.0.0.1/udp/0/udx`, `addrsFactory((addrs) => addrs)`, a short `rebroadcastInterval`, `connect`, then `getBlock` with an explicit timeout. Always `stop()` both Bitswaps and `close()` both hosts in `tearDown`. - **Unit tests with mocks**: the repo's `test/fake_swarm.dart` builds a mockito `MockHost` with a fake event bus, peer store (controls which peers "speak Bitswap") and `newStream` that records every sent `Message`. It is not exported; copy it if you need it. Drive the instance through `handleIncomingMessage(peer, message)` and `onPeerDisconnected(peer)`: ```dart final block = Message() ..payload.add(Message_Block() ..prefix = [0x01, 0x55, 0x12, 0x20] // CIDv1, raw, sha2-256, 32 bytes ..data = blockData); await bitswap.handleIncomingMessage(peerA, block); final dontHave = Message() ..blockPresences.add(Message_BlockPresence() ..cid = cid.toBytes() ..type = Message_BlockPresenceType.DontHave); ``` - **Timeouts and rebroadcasts**: use `package:fake_async` (`fakeAsync`, `time.elapse`, `time.flushMicrotasks`) as `test/timing_test.dart` does, rather than real sleeps. - Run the package's own suite with `dart test`. ## Red lines - Never store or serve a block without verifying its hash. Do not bypass the wantlist check or pass unverified bytes to `notifyNewBlock`. - Never raise the 4 MiB message limit or change the protocol ID; other Bitswap implementations reject larger messages and will not negotiate a different ID. - Never treat `getBlock` returning `null` as "block does not exist"; it only means it did not arrive in time. - Always `stop()` Bitswap before closing the host, or streams, timers and the event-bus subscription leak. - Do not set `maxWantsPerPeer` or `serveBytesPerTurn` unbounded on a public node; they are what stops one peer exhausting memory or starving others. - Do not rely on Go/kubo interop without testing it yourself. ## Reference - Docs: https://cryptopeer.net/libp2p/bitswap/ - Repo: https://github.com/stephanfeb/dart-libp2p-bitswap - Package: https://pub.dev/packages/dart_libp2p_bitswap - Spec: https://github.com/ipfs/specs/blob/main/src/bitswap-protocol.md - Related skills: `dart-libp2p-host`, `dart-libp2p-kad-dht`, `merkle-crdt-sync`