Skip to content
cryptopeercryptopeer.net

dart-libp2p-bitswap

Skill dart-libp2p-bitswap. Use when fetching or serving content-addressed blocks between dart_libp2p peers with package dart_libp2p_bitswap (Bitswap 1.2.0): implementing a BlockStore, getBlock / wantBlocks / cancelWants, notifyNewBlock, BitswapOptions tuning, CIDs via dcid, and testing Bitswap code.

SKILL.md (Claude Code, Codex, Cursor) · dart-libp2p-bitswap.md (Boubou) · install: curl -fsSL https://cryptopeer.net/skills/install.sh | sh -s -- dart-libp2p-bitswap

What it is

Section titled “What it is”

A Dart implementation of the Bitswap 1.2.0 block exchange protocol (/ipfs/bitswap/1.2.0) on top of a dart_libp2p Host. It fetches blocks by CID from connected peers and serves blocks from a BlockStore you provide. Behaviour (want strategy, limits, defaults) follows boxo, the Go implementation used by kubo.

It only moves raw blocks between peers that are already connected. It does not find providers (use the DHT skill, dart-libp2p-kad-dht, then connect), does not walk DAGs (your code decodes a block and asks for its links), and does not set up the host (see the dart-libp2p-host skill). For replicating Merkle-CRDTs, see merkle-crdt-sync.

Install

Section titled “Install”
dependencies:
  dart_libp2p_bitswap: ^2.0.0
  dart_libp2p: ^4.12.0
  dcid: ^1.0.0

Dart SDK ^3.5.0. Imports:

import 'dart:typed_data';


import 'package:dart_libp2p/dart_libp2p.dart';
import 'package:dart_libp2p_bitswap/dart_libp2p_bitswap.dart';
import 'package:dcid/dcid.dart' as dart_cid;

The single library dart_libp2p_bitswap.dart exports Bitswap, BitswapOptions, BlockStore, SizedBlockStore and the generated protobuf types (Message, Message_Wantlist, Message_Wantlist_Entry, Message_Block, Message_BlockPresence, Message_Wantlist_WantType, Message_BlockPresenceType). Alias it if those names clash with yours.

Core workflow

Section titled “Core workflow”

1. A BlockStore

Section titled “1. A BlockStore”
class InMemoryBlockStore implements BlockStore {
  final Map<String, Uint8List> _blocks = {};


  @override
  Future<bool> has(dart_cid.CID cid) async =>
      _blocks.containsKey(cid.toString());


  @override
  Future<Uint8List?> get(dart_cid.CID cid) async => _blocks[cid.toString()];


  @override
  Future<void> put(dart_cid.CID cid, Uint8List data) async =>
      _blocks[cid.toString()] = data;
}

Bitswap calls has/get to serve peers and put to store blocks it fetched (only blocks on its wantlist, after re-hashing them against their CID).

2. Start, fetch, stop (two hosts)

Section titled “2. Start, fetch, stop (two hosts)”
// provider and fetcher are started Hosts (see dart-libp2p-host)
const options = BitswapOptions(rebroadcastInterval: Duration(seconds: 1));


final providerStore = InMemoryBlockStore();
final providerBitswap =
    Bitswap(host: provider, blockStore: providerStore, options: options);
final fetcherBitswap =
    Bitswap(host: fetcher, blockStore: InMemoryBlockStore(), options: options);
await providerBitswap.start();
await fetcherBitswap.start();


final data = Uint8List.fromList('Hello from Bitswap!'.codeUnits);
final cid = dart_cid.CID.fromData(dart_cid.CID.V1, 'raw', data); // sha2-256
await providerStore.put(cid, data);


await fetcher.connect(AddrInfo(provider.id, provider.addrs));
final received =
    await fetcherBitswap.getBlock(cid, timeout: const Duration(seconds: 15));
if (received == null) {
  // timed out or cancelled
}


await fetcherBitswap.stop();
await providerBitswap.stop();

This mirrors example/dart_libp2p_bitswap_example.dart in the repo (UDX, Noise, Yamux, Libp2p.addrsFactory((addrs) => addrs) so loopback addresses are kept): dart run example/dart_libp2p_bitswap_example.dart.

Prefetch several blocks without waiting

Section titled “Prefetch several blocks without waiting”
await bitswap.wantBlocks([cid1, cid2, cid3]); // stored via put() on arrival
// later
final b1 = await bitswap.getBlock(cid1); // local hit if already arrived
bitswap.cancelWants([cid3]); // sends CANCEL; pending getBlock(cid3) -> null

Add a block from the application

Section titled “Add a block from the application”

If your code writes into the store itself (not via Bitswap), tell Bitswap so peers waiting for it are served and local getBlock waiters complete:

await store.put(cid, bytes);
bitswap.notifyNewBlock(cid, bytes); // bytes are trusted to match cid

Parse a CID received from elsewhere

Section titled “Parse a CID received from elsewhere”
final cid = dart_cid.CID.fromString(
    'bafkreibm6jg3ux5qumhcn2b3flc3tyu6dmlb4xa7u5bf44yegnrjhc4yeq');
final bytesCid = dart_cid.CID.fromBytes(cid.toBytes());

Size-aware store

Section titled “Size-aware store”
class MyStore implements SizedBlockStore {
  // has / get / put as above, plus:
  @override
  Future<int?> getSize(dart_cid.CID cid) async => /* size or null */ null;
}

With a SizedBlockStore, Bitswap decides how to answer WANT_HAVE without reading the block, and fills pendingBytes in responses.

Logging

Section titled “Logging”

The library prints nothing by default. Loggers: bitswap, bitswap.client, bitswap.client.wantlist, bitswap.server, bitswap.network, bitswap.packing.

import 'package:logging/logging.dart';


hierarchicalLoggingEnabled = true;
Logger('bitswap').level = Level.FINE;
Logger('bitswap').onRecord.listen(
    (r) => print('${r.level.name} ${r.loggerName}: ${r.message}'));

WARNING = problems (rejected blocks, dropped oversize blocks, no Bitswap peers), FINE = protocol activity, FINEST = per-chunk stream I/O.

Key concepts / API map

Section titled “Key concepts / API map”
  • Bitswap({required Host host, required BlockStore blockStore, BitswapOptions options}): facade over client (fetching) and server (serving). No side effects until start().
  • Future<void> start(): registers the /ipfs/bitswap/1.2.0 stream handler and subscribes to EvtPeerConnectednessChanged on host.eventBus. Idempotent.
  • Future<void> stop(): removes the handler, closes streams, fails pending getBlock with StateError. Idempotent; cannot be restarted.
  • Future<Uint8List?> getBlock(CID cid, {Duration? timeout}): local store first, else want it and wait. null on timeout or cancellation.
  • Future<void> wantBlocks(List<CID> cids): add to wantlist and broadcast.
  • void cancelWants(List<CID> cids): drop wants, send CANCEL.
  • void notifyNewBlock(CID cid, Uint8List data): announce an app-stored block.
  • BlockStore: has, get, put (all async).
  • SizedBlockStore implements BlockStore: adds Future<int?> getSize(CID).
  • BitswapOptions (all optional, const):
    • getBlockTimeout = 30 s
    • rebroadcastInterval = 10 s
    • maxWantsPerPeer = 1024 (wants beyond this from one peer are ignored)
    • serveBytesPerTurn = 1 MiB (round-robin fairness between peers)
    • wantHaveReplaceSize = 1024 bytes (send the block instead of HAVE; 0 disables)
  • handleIncomingMessage(PeerId, Message) and onPeerDisconnected(PeerId): @visibleForTesting hooks for injecting protocol messages in unit tests.

Want strategy (1.2.0, as boxo): each CID is sent as WANT_BLOCK to one peer and WANT_HAVE to all other Bitswap peers, always with sendDontHave. On DONT_HAVE or disconnect, the WANT_BLOCK moves to a peer that said HAVE. After the first full wantlist, later updates to a peer are deltas. When a block arrives, every peer that was asked gets CANCEL.

Peers that receive wants: peers that have sent us a Bitswap wantlist, plus connected peers whose peer-store protocols (from identify) include /ipfs/bitswap/1.2.0.

Gotchas

Section titled “Gotchas”
  • Call start() first. getBlock, wantBlocks and notifyNewBlock throw StateError('Bitswap not started; call start() first') before start() and StateError('Bitswap stopped') after stop(). A stopped instance cannot be restarted: build a new Bitswap.
  • No provider discovery. Bitswap only asks peers you are already connected to. If no connected peer speaks Bitswap it logs Found no Bitswap-supporting peers. and getBlock waits until timeout. Find providers through the DHT and host.connect(...) to them first.
  • Identify race on fresh connections. Wants are only sent to a connected peer once identify has recorded that it speaks /ipfs/bitswap/1.2.0. A getBlock made right after connect may not go out until the next rebroadcast (default 10 s). In tests and latency-sensitive apps set a short rebroadcastInterval (the example uses 1 s), or make sure identify has completed before asking.
  • Loopback hosts cannot dial each other by default. dart_libp2p’s default address factory drops loopback addresses; for local tests pass Libp2p.addrsFactory((addrs) => addrs) when building the host.
  • getBlock returns null only on timeout or cancelWants. Other errors (including StateError from stop()) propagate, so wrap calls that may race with shutdown. Concurrent getBlock for the same CID share one want; one caller timing out does not cancel it for the others, but when the last waiter gives up, CANCEL is sent.
  • Unsolicited blocks are discarded. Since 2.0.0 a block is stored only if it is on the local wantlist. Pushing blocks at a peer that did not ask does nothing; the receiver must getBlock/wantBlocks first.
  • Call notifyNewBlock after writing to the store yourself. Without it, peers that wanted the block before you had it are not served until they re-send their want, and local getBlock waiters keep waiting. notifyNewBlock does not check the hash: never pass data that does not match the CID.
  • CIDs match by exact string. The wantlist and peer ledgers key on cid.toString(). A CIDv0 (Qm...) and its CIDv1 (bafy..., dag-pb) are different wants, and the reference store above treats them as different blocks. The CID of a received block is rebuilt from the sender’s prefix, so ask for the version the provider stores. cid.toV1() / cid.toV0() (dcid) convert; toV0() throws FormatException unless the CID is dag-pb + sha2-256. Key your store by multihash if you need both forms to hit.
  • Supported hashes only. Received blocks are re-hashed using the prefix: sha2-256/512, sha3-224/256/384/512, keccak-256/512, blake2b-256/512, blake3 (1 to 128-byte digests) and identity (up to 128 bytes). Anything else is rejected with a warning. CIDv0 must be dag-pb + sha2-256 + 32 bytes. Data that does not match produces a different CID and is dropped as unwanted.
  • 4 MiB message limit, both ways. Inbound messages declaring more than 4 MiB reset the stream. Outbound responses are split into messages of at most 4 MiB, and a single block too big for one message is dropped with a warning. Keep blocks small (the spec recommends at most 2 MiB; IPFS chunks at 256 KiB to 1 MiB). There is no explicit block-size check on put.
  • Only 1.2.0 is spoken. The only protocol ID registered or dialled is /ipfs/bitswap/1.2.0; there is no fallback to 1.0.0 / 1.1.0, so peers that only speak older versions will not exchange blocks with this package.
  • Go interop is untested. Wire format, CIDv0 prefix layout (0x00 0x70 0x12 0x20), limits and defaults follow go-bitswap / boxo, and CIDv0 blocks from go-bitswap are accepted (2.0.0 fix), but the README states interop with Go nodes (kubo) has not been tested end to end. Verify before relying on it.
  • Received pendingBytes is ignored (as in boxo). It is only set on outgoing responses when the store is a SizedBlockStore.
  • Store failures. If put throws, the error is logged and the want stays open for another delivery. A failing get for one want does not stop other wants being served. Do not rely on exceptions from your store surfacing in getBlock.
  • Upgrading from 1.x. IBlockStore is now BlockStore; the peerEventsStream constructor argument is gone (Bitswap subscribes to host.eventBus itself); the CID package is dcid, not dart_cid. Older docs and snippets that use these names are out of date.

Testing

Section titled “Testing”
  • Real hosts in one process: copy the pattern from example/dart_libp2p_bitswap_example.dart: two hosts on /ip4/127.0.0.1/udp/0/udx, addrsFactory((addrs) => addrs), a short rebroadcastInterval, connect, then getBlock with an explicit timeout. Always stop() both Bitswaps and close() both hosts in tearDown.
  • Unit tests with mocks: the repo’s test/fake_swarm.dart builds a mockito MockHost with a fake event bus, peer store (controls which peers “speak Bitswap”) and newStream that records every sent Message. It is not exported; copy it if you need it. Drive the instance through handleIncomingMessage(peer, message) and onPeerDisconnected(peer):
final block = Message()
  ..payload.add(Message_Block()
    ..prefix = [0x01, 0x55, 0x12, 0x20] // CIDv1, raw, sha2-256, 32 bytes
    ..data = blockData);
await bitswap.handleIncomingMessage(peerA, block);


final dontHave = Message()
  ..blockPresences.add(Message_BlockPresence()
    ..cid = cid.toBytes()
    ..type = Message_BlockPresenceType.DontHave);
  • Timeouts and rebroadcasts: use package:fake_async (fakeAsync, time.elapse, time.flushMicrotasks) as test/timing_test.dart does, rather than real sleeps.
  • Run the package’s own suite with dart test.

Red lines

Section titled “Red lines”
  • Never store or serve a block without verifying its hash. Do not bypass the wantlist check or pass unverified bytes to notifyNewBlock.
  • Never raise the 4 MiB message limit or change the protocol ID; other Bitswap implementations reject larger messages and will not negotiate a different ID.
  • Never treat getBlock returning null as “block does not exist”; it only means it did not arrive in time.
  • Always stop() Bitswap before closing the host, or streams, timers and the event-bus subscription leak.
  • Do not set maxWantsPerPeer or serveBytesPerTurn unbounded on a public node; they are what stops one peer exhausting memory or starving others.
  • Do not rely on Go/kubo interop without testing it yourself.

Reference

Section titled “Reference”

Built by Stephan Februarygithub.com/stephanfebLinkedIn