Encryption & compression

This page explains the two payload transforms the client applies to messages: NaCl box encryption with keys derived from peer identities, and LZ4 compression. Both apply to messages only; documents, feeds and collections are stored exactly as you send them.

Encrypt a message

res, err := cl.SendMessage(ctx, recipientID, []byte("meet at noon"), client.WithEncryption())

The recipient does nothing special: RetrieveMessages sees the encrypted flag and decrypts the payload before returning it. A message that fails to decrypt fails the whole retrieve with an error naming the message ID.

How it works

• Keys. Each peer’s Ed25519 identity key is converted to X25519: the private key by the standard conversion (SHA-512 of the seed, clamped), the public key by the Edwards-to-Montgomery map. The recipient’s public key comes from its peer ID, so there is no key exchange and no key server. Both peers must use Ed25519 identities.
• Cipher. NaCl box: X25519 key agreement and XSalsa20-Poly1305 authenticated encryption, with a random 24-byte nonce per message.
• Binding. The recipient peer ID, the folder path (inbox when empty) and the message ID are sealed inside the box with the payload and checked on decryption. The server, or anyone who can write to the mailbox, cannot move a ciphertext to another folder, relabel it with another message ID or redirect it to another recipient without the decryption failing.
• Wire format. "RCE2", the 24-byte nonce, then the box of the binding header and payload. The client still reads the older unbound format (nonce then box) for messages sealed before the binding existed.

The Dart client implements the same format and both carry the same test vectors, so a message sealed by one opens in the other. See Dart client.

The functions behind WithEncryption are exported for use outside messaging: EncryptBoundPayload, DecryptBoundPayload, BindingFor, PeerIDToX25519PublicKey, Ed25519PrivateKeyToX25519 and Ed25519PublicKeyToX25519. You can use EncryptBoundPayload to seal a document for yourself or a reader before storing it; the binding fields are yours to choose.

What is encrypted and what is not

Part of the message	Encrypted
Payload	Yes
Sender and recipient peer IDs	No
Folder path	No
Created timestamp, expiry, priority	No
Protocol and IMAP flags, including the encrypted flag	No
Payload size	No (padding is up to you)

The encrypted flag is not authenticated. If someone strips it, the recipient’s client skips decryption and hands your app the ciphertext: a denial of service, not a disclosure.

What it protects

• Confidentiality and integrity of the payload against the server and the network.
• Placement, through the binding: a ciphertext cannot be presented as a different message.

What it does not protect

• No forward secrecy. The keys are the long-term identity keys. If either peer’s key leaks later, every message between them can be decrypted. An app that needs forward secrecy runs a ratcheting protocol such as Signal’s Double Ratchet over these messages and treats this layer as an outer envelope, or skips it.
• Verbatim replay. The same ciphertext stored again into the same folder under the same ID, for instance after the recipient deleted it, is not detected. Keep a record of message IDs you have seen if that matters.
• Non-repudiation. NaCl box authenticates the sender to the recipient only; the recipient cannot prove to anyone else who wrote a message.
• Metadata. The envelope above stays visible, and the server sees who connects when.

The same Ed25519 key also signs the Noise handshake and GossipSub messages. Guard it accordingly; see Identity keys.

Compression

res, err = cl.SendMessage(ctx, recipientID, bigPayload, client.WithCompression())     // 1 KiB threshold
res, err = cl.SendMessage(ctx, recipientID, bigPayload, client.WithCompression(4096)) // custom threshold

WithCompression applies LZ4 block compression to payloads of at least the threshold, which defaults to client.DefaultCompressionThreshold (1024 bytes). A payload below the threshold, or one that does not get smaller, is sent as is, without the compressed flag. The receiving client decompresses automatically and refuses anything that would expand beyond client.MaxDecompressedSize (16 MiB).

Order: compress, then encrypt

With both options, the client compresses first and encrypts the result; on retrieval it decrypts, then decompresses.

send:    payload -> LZ4 -> NaCl box -> frame
receive: frame -> NaCl box open -> LZ4 decompress -> payload

Ciphertext does not compress, so this is the only order in which compression helps. It also means the compressed size, and so something about the content, is visible to the server. If payload length is sensitive, pad the plaintext before sending and do not compress.

res, err = cl.SendMessage(ctx, recipientID, payload,
  client.WithCompression(),
  client.WithEncryption(),
)

The order of the options does not matter; the client always compresses before encrypting.