Skip to content

feat: improve IAP DFU example with card.binary #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rdlauer merged 2 commits into from
Sep 17, 2026
Merged

feat: improve IAP DFU example with card.binary #42

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a49a96f
update esp32 dfu sample to use card.binary
rdlauer Sep 17, 2026
6a3aa2d
fix per codex review
rdlauer Sep 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 33 additions & 13 deletions esp32-dfu/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,16 @@ the firmware image from Notehub, then this sketch reads it chunk-by-chunk via
`dfu.get`, writes it to the inactive OTA partition, validates MD5, sets the boot
partition, and reboots.

Chunks move through the Notecard's binary store rather than as base64 in the
response, which avoids the ~33% encoding overhead and is substantially faster —
the difference is most noticeable over I2C or a low-baud serial connection.

## Hardware

- An ESP32-based host (e.g. Adafruit HUZZAH32 Feather)
- A Blues Notecard on a Notecarrier F (or similar)
- A Blues Notecard on a Notecarrier F (or similar), running Notecard firmware
**v9.1.1 or later** (`dfu.get` gained its `binary` argument in that release —
check with `card.version`)
- A push button on `buttonPin` (default GPIO 21) — single press logs a simulated
sensor reading, double press forces a DFU poll and `hub.sync`

Expand All @@ -22,7 +28,8 @@ The sketch defaults to I2C for the Notecard. To use serial instead, uncomment
in Notehub and hardcode `#define PRODUCT_UID "..."` in the sketch.
2. Open `esp32-dfu/esp32-dfu.ino` in the Arduino IDE (the sketch filename must
match the directory name).
3. Install the **Blues Wireless Notecard** library via the Arduino Library Manager
3. Install the **Blues Wireless Notecard** library (**v1.5.0 or later**, which is
where the `NoteBinaryStore*` helpers landed) via the Arduino Library Manager
or `arduino-cli lib install "Blues Wireless Notecard"`.
4. Select an ESP32 board with an OTA-capable partition scheme.
5. Compile and upload.
Expand All @@ -33,19 +40,32 @@ The sketch defaults to I2C for the Notecard. To use serial instead, uncomment
- `loop()` calls `dfuPoll(false)` periodically (rate-limited to once per hour
unless forced via a double button press).
- When `dfu.status` reports `mode:"ready"` with a newer image, the sketch:
1. Sets `hub.set, mode:"dfu"` to put the Notecard in DFU mode.
2. Waits up to two minutes for DFU mode to actually engage (verified via
`dfu.get`).
3. Begins an `esp_ota_begin`/`esp_ota_write` sequence, reading 4 KB chunks
via `dfu.get` and verifying each chunk's MD5.
4. On success: reverts hub mode (`hub.set, mode:"-"`), validates the full-image
MD5, sets the boot partition, clears DFU state (`dfu.status, stop:true`),
and reboots.
5. On any failure: cleanly releases the OTA handle, reports the error to
Notehub via `dfu.status, stop:true, err:"..."`, and reverts hub mode.
1. Opens the inactive OTA partition with `esp_ota_begin`. This happens *first*,
because erasing a large partition is slow and would otherwise burn into the
Notecard's 15-minute DFU-mode timeout.
2. Issues a zero-length `dfu.get` to ask whether the Notecard can serve the
image right now. Notecards that hold the downloaded image in onboard flash
answer immediately and stay connected and syncing for the whole update.
3. Only if that fails with `not currently in the DFU operating mode`: sets
`hub.set, mode:"dfu"` and waits up to two minutes for DFU mode to engage.
4. Clears the binary store, then for each 8 KB chunk issues
`dfu.get, binary:true` — which parks the chunk in the binary store instead
of the response — and reads it back with `NoteBinaryStoreReceive()`, which
COBS-decodes it and verifies the Notecard's MD5. Each chunk goes straight
to `esp_ota_write`.
5. On success: releases the binary store, leaves DFU mode if it entered it
(`hub.set, mode:"dfu-completed"`), validates the full-image MD5, sets the
boot partition, clears DFU state (`dfu.status, stop:true`), and reboots.
6. On any failure: cleanly releases the OTA handle and the binary store,
reports the error to Notehub via `dfu.status, stop:true, err:"..."`, and
leaves DFU mode if it entered it.

The binary store is a single shared resource. If your own application uses it
for `web.post` uploads or `note.add` payloads, make sure those have finished
before a DFU starts.

## Files

- [esp32-dfu.ino](esp32-dfu.ino) — sketch entry point: setup, loop, button handling, version reporting.
- [dfu.cpp](dfu.cpp) — DFU state machine: partition discovery, chunked `dfu.get`, OTA write, MD5 validation.
- [dfu.cpp](dfu.cpp) — DFU state machine: partition discovery, chunked `dfu.get` through the binary store, OTA write, MD5 validation.
- [main.h](main.h) — shared declarations.
236 changes: 152 additions & 84 deletions esp32-dfu/dfu.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,46 @@
#include "esp_ota_ops.h"
#include "esp_flash_partitions.h"

// The largest chunk a single dfu.get request will return. Requests for more
// than this are rejected by the Notecard.
#define DFU_CHUNK_LEN 8192

// Whether this DFU had to put the Notecard into DFU mode. Notecards that hold
// the downloaded image in onboard flash serve it without DFU mode, so we only
// enter (and therefore only need to leave) it when the Notecard asks us to.
static bool dfuModeEntered = false;

// Leave DFU mode, if we entered it. Prefer "dfu-completed" over "-": it
// resumes whatever sync mode the Notecard was using beforehand, where "-"
// would reset it to the periodic default.
static void dfuExitDFUMode() {
if (!dfuModeEntered) {
return;
}
if (J *req = notecard.newRequest("hub.set")) {
JAddStringToObject(req, "mode", "dfu-completed");
notecard.sendRequest(req);
}
dfuModeEntered = false;
}

// Cleanly back out of DFU on any failure: release the ESP OTA handle (if open),
// tell the Notecard to clear staged DFU state (with an optional error string
// that surfaces on Notehub), and revert hub mode to whatever it was before DFU.
// release the Notecard's binary store, tell the Notecard to clear staged DFU
// state (with an optional error string that surfaces on Notehub), and leave DFU
// mode if we entered it.
static void dfuAbort(esp_ota_handle_t handle, const char *err) {
if (handle != 0) {
esp_ota_end(handle);
}
NoteBinaryStoreReset();
if (J *req = notecard.newRequest("dfu.status")) {
JAddBoolToObject(req, "stop", true);
if (err != NULL) {
JAddStringToObject(req, "err", err);
}
notecard.sendRequest(req);
}
if (J *req = notecard.newRequest("hub.set")) {
JAddStringToObject(req, "mode", "-");
notecard.sendRequest(req);
}
dfuExitDFUMode();
}

// Display DFU partition information
Expand Down Expand Up @@ -98,39 +120,10 @@ void dfuPoll(bool force) {
return;
}

// Enter DFU mode. Note that the Notecard will automatically switch us back out of
// DFU mode after 15m, so we don't leave the notecard in a bad state if we had a problem here.
if (J *req = notecard.newRequest("hub.set")) {
JAddStringToObject(req, "mode", "dfu");
notecard.sendRequest(req);
}

// Proceed with DFU
dfuCheckMs = millis();

// Wait until we have successfully entered the mode. The fact that this loop isn't
// just an infinite loop is simply defensive programming. If for some odd reason
// we don't enter DFU mode, we'll eventually come back here on the next DFU poll.
bool inDFUMode = false;
uint32_t beganDFUModeCheck = millis();
while (!inDFUMode && millis() < beganDFUModeCheck + (2 * ms1Min)) {
if (J *rsp = notecard.requestAndResponse(notecard.newRequest("dfu.get"))) {
if (!notecard.responseError(rsp))
inDFUMode = true;
notecard.deleteResponse(rsp);
}
if (!inDFUMode)
delay(2500);
}

// If we failed, leave DFU mode immediately
if (!inDFUMode) {
dfuAbort(0, "host failed to enter DFU mode");
return;
}

// The image is ready. If the version is the same as what's in memory, then of course don't
// bother to do the update.
// Prepare the partition that will receive the image BEFORE asking the
// Notecard for anything. Erasing a large flash region takes a while, and
// on a Notecard that needs DFU mode that erase would otherwise run against
// the Notecard's 15-minute DFU-mode timeout.
esp_err_t err;
// update handle : set by esp_ota_begin(), must be freed via esp_ota_end()
esp_ota_handle_t update_handle = 0 ;
Expand Down Expand Up @@ -160,109 +153,184 @@ void dfuPoll(bool force) {
return;
}

// Proceed with DFU
dfuCheckMs = millis();
dfuModeEntered = false;

// Ask whether the Notecard can serve the image right now. A zero-length
// dfu.get checks readiness without transferring anything. Notecards that
// hold the downloaded image in onboard flash answer immediately, and can
// stay connected and syncing for the whole update.
bool readyToRead = false;
bool needsDFUMode = false;
if (J *rsp = notecard.requestAndResponse(notecard.newRequest("dfu.get"))) {
readyToRead = !notecard.responseError(rsp);
if (!readyToRead) {
const char *rspErr = JGetString(rsp, "err");
APP_LOGF("dfu: not ready to read: %s\n", rspErr);
// Only one error means "you need to be in DFU mode"; everything
// else (a bus glitch, an image that is no longer staged) is not
// something DFU mode fixes. Note that this particular error
// carries no {error-token}, so the message is the only signal.
needsDFUMode = (strstr(rspErr, "DFU operating mode") != NULL);
}
notecard.deleteResponse(rsp);
} else {
APP_LOGF("dfu: no response to the readiness check\n");
}

// Don't disconnect a Notecard that was never going to need it. Leave the
// staged image alone and let the next poll try again, rather than reporting
// a failure to Notehub over what may be a transient error.
if (!readyToRead && !needsDFUMode) {
APP_LOGF("dfu: notecard can't serve the image right now; will retry\n");
esp_ota_end(update_handle);
return;
}

// Notecards without onboard flash read the image out of the cellular
// modem's file system, which they can only do once the network connection
// is closed. Entering DFU mode closes it, but the Notecard has to finish
// whatever it was doing first, so poll rather than assuming a fixed delay.
// Note that the Notecard leaves DFU mode on its own after 15m, so we don't
// strand it in a bad state if we fail partway through.
if (!readyToRead) {
APP_LOGF("dfu: entering DFU mode\n");
if (J *req = notecard.newRequest("hub.set")) {
JAddStringToObject(req, "mode", "dfu");
notecard.sendRequest(req);
}
dfuModeEntered = true;
uint32_t beganDFUModeCheck = millis();
while (!readyToRead && millis() < beganDFUModeCheck + (2 * ms1Min)) {
delay(2500);
if (J *rsp = notecard.requestAndResponse(notecard.newRequest("dfu.get"))) {
readyToRead = !notecard.responseError(rsp);
notecard.deleteResponse(rsp);
}
}
if (!readyToRead) {
dfuAbort(update_handle, "host failed to enter DFU mode");
return;
}
}

APP_LOGF("dfu: beginning firmware update\n");

// Each chunk arrives through the Notecard's binary store rather than as
// base64 in the response, so clear anything a previous operation left there.
NoteBinaryStoreReset();

// One buffer, reused for every chunk. NoteBinaryStoreReceive() reads the
// COBS-encoded bytes off the wire and decodes them in place, so the buffer
// has to be big enough for the *encoded* form of a full chunk, plus the
// terminating null it writes after the decoded data.
uint32_t chunkBufLen = NoteBinaryCodecMaxEncodedLength(DFU_CHUNK_LEN) + 1;
uint8_t *chunkBuf = (uint8_t *) malloc(chunkBufLen);
if (chunkBuf == NULL) {
APP_LOGF("dfu: can't allocate %lu-byte chunk buffer\n", (unsigned long)chunkBufLen);
dfuAbort(update_handle, "out of memory");
return;
}

// Loop over received chunks
int offset = 0;
int chunklen = 4096;
int left = imageLength;
NoteMD5Context md5Context;
NoteMD5Init(&md5Context);
while (left) {

// Read next chunk from card
int thislen = chunklen;
int thislen = DFU_CHUNK_LEN;
if (left < thislen)
thislen = left;

// If anywhere, this is the location of the highest probability of I/O error
// on the I2C or serial bus, simply because of the amount of data being transferred.
// As such, it's a conservative measure just to retry.
char *payload = NULL;
for (int retry=0; retry<5; retry++) {
bool chunkReceived = false;
for (int retry=0; retry<5 && !chunkReceived; retry++) {
APP_LOGF("dfu: reading chunk (offset:%d length:%d try:%d)\n", offset, thislen, retry+1);

// Request the next chunk from the notecard
// Ask the Notecard to move this chunk into its binary store. The
// response describes what landed there -- decoded "length", encoded
// "cobs", and an MD5 in "status" -- but carries no payload.
J *req = notecard.newRequest("dfu.get");
if (req == NULL) {
APP_LOGF("dfu: insufficient memory\n");
free(chunkBuf);
dfuAbort(update_handle, "out of memory");
return;
}
JAddNumberToObject(req, "offset", offset);
JAddNumberToObject(req, "length", thislen);
JAddBoolToObject(req, "binary", true);
J *rsp = notecard.requestAndResponse(req);
if (rsp == NULL) {
APP_LOGF("dfu: insufficient memory\n");
free(chunkBuf);
dfuAbort(update_handle, "out of memory");
return;
}
if (notecard.responseError(rsp)) {
APP_LOGF("dfu: error on read: %s\n", JGetString(rsp, "err"));
} else {
char *payloadB64 = JGetString(rsp, "payload");
if (payloadB64[0] == '\0') {
APP_LOGF("dfu: no payload\n");
notecard.deleteResponse(rsp);
dfuAbort(update_handle, "no payload");
return;
}
payload = (char *) malloc(JB64DecodeLen(payloadB64));
if (payload == NULL) {
APP_LOGF("dfu: can't allocate payload decode buffer\n");
notecard.deleteResponse(rsp);
dfuAbort(update_handle, "out of memory");
return;
}
int actuallen = JB64Decode(payload, payloadB64);
const char *expectedMD5 = JGetString(rsp, "status");
char chunkMD5[NOTE_MD5_HASH_STRING_SIZE] = {0};
NoteMD5HashString((uint8_t *)payload, actuallen, chunkMD5, sizeof(chunkMD5));
if (actuallen == thislen && strcmp(chunkMD5, expectedMD5) == 0) {
notecard.deleteResponse(rsp);
break;
}
notecard.deleteResponse(rsp);
continue;
}

free(payload);
payload = NULL;
// A Notecard that predates the binary argument ignores it and
// answers with a payload instead. Fail loudly rather than silently
// reading an empty binary store.
if (JGetObjectItem(rsp, "cobs") == NULL) {
APP_LOGF("dfu: this Notecard does not support dfu.get with binary:true (requires firmware v9.1.1 or later)\n");
notecard.deleteResponse(rsp);
free(chunkBuf);
dfuAbort(update_handle, "notecard firmware too old for binary DFU");
return;
}
notecard.deleteResponse(rsp);

if (thislen != actuallen)
APP_LOGF("dfu: decoded data not the correct length (%d != actual %d)\n", thislen, actuallen);
else
APP_LOGF("dfu: %d-byte decoded data MD5 mismatch (%s != actual %s)\n", actuallen, expectedMD5, chunkMD5);
// Pull the chunk out of the binary store. This issues
// card.binary.get, COBS-decodes in place, and verifies the chunk
// against the MD5 the Notecard reported -- the same integrity check
// the base64 path used to do by hand.
const char *binErr = NoteBinaryStoreReceive(chunkBuf, chunkBufLen, 0, thislen);
if (binErr != NULL) {
APP_LOGF("dfu: error reading binary store: %s\n", binErr);
continue;
}

notecard.deleteResponse(rsp);
chunkReceived = true;
}
if (payload == NULL) {
if (!chunkReceived) {
APP_LOGF("dfu: unrecoverable error on read\n");
free(chunkBuf);
dfuAbort(update_handle, "unrecoverable read error");
return;
}

// MD5 the chunk
NoteMD5Update(&md5Context, (uint8_t *)payload, thislen);
NoteMD5Update(&md5Context, chunkBuf, thislen);

// Write the chunk
err = esp_ota_write(update_handle, (const void *)payload, thislen);
err = esp_ota_write(update_handle, (const void *)chunkBuf, thislen);
if (err != ESP_OK) {
free(payload);
free(chunkBuf);
dfuAbort(update_handle, esp_err_to_name(err));
return;
}

// Move to next chunk
free(payload);
APP_LOGF("dfu: successfully transferred offset:%d len:%d\n", offset, thislen);
offset += thislen;
left -= thislen;
}

// Exit DFU mode. (Had we not done this, the Notecard exits DFU mode automatically after 15m.)
if (J *req = notecard.newRequest("hub.set")) {
JAddStringToObject(req, "mode", "-");
notecard.sendRequest(req);
}
// The whole image is off the Notecard now. Hand the binary store back to
// the rest of the application, and leave DFU mode if we entered it.
free(chunkBuf);
NoteBinaryStoreReset();
dfuExitDFUMode();

// Done
if (esp_ota_end(update_handle) != ESP_OK) {
Expand Down