Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 175 additions & 0 deletions .github/workflows/fuzz-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
name: Fuzz image

# Builds Dockerfile.fuzz -- the fast-check round-trip properties in tests/fuzz --
# and publishes it to GHCR for the ChainSafe fuzz platform.
#
# When an image is built:
# - manual run on main (Actions tab) -> build, test, publish `main`
# - manual run on any other branch -> build, test, publish `branch-<name>`
# - pull request touching code the targets exercise -> build and test only
# Publishing is deliberately manual: the fuzz platform follows the `main` tag
# and restarts the campaign on every publish, so run this once enough good
# changes have landed, not on every commit. Pull requests still build and test
# so a broken Dockerfile.fuzz is caught before the next publish.
#
# Two jobs: `build and test` runs for every event with read-only permissions;
# `publish` runs only on a manual run, holds the only write token, and pushes
# the exact image the build job tested.
#
# Tags: `main` is what the fuzz platform runs, and only a run on main moves it.
# A run on another branch publishes `branch-<name>` instead, for trying a branch
# on the platform before it merges. `sha-<short>` names each build's source
# commit, also recorded as the org.opencontainers.image.revision label.
# Old versions beyond the newest 20 are deleted.

on:
pull_request:
paths:
- "src/**"
- "tests/**"
- "build.zig"
- "build.zig.zon"
- "package.json"
- "pnpm-lock.yaml"
- "Dockerfile.fuzz"
- "Dockerfile.fuzz.dockerignore"
- ".github/workflows/fuzz-image.yml"
workflow_dispatch:

concurrency:
group: fuzz-image-${{ github.ref }}
cancel-in-progress: true

env:
IMAGE: ghcr.io/chainsafe/zapi-fuzz

jobs:
build:
name: build and test
runs-on: ubuntu-latest
# Read-only: this job runs on pull requests and never pushes.
permissions:
contents: read
steps:
- name: Checkout Repository
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1

- name: Build
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: .
file: Dockerfile.fuzz
platforms: linux/amd64
build-args: REVISION=${{ github.sha }}
tags: zapi-fuzz:ci
load: true
# Off so the image is a single manifest: one package version per
# publish, which the version cleanup below relies on.
provenance: false
sbom: false
cache-from: type=gha,scope=fuzz-image
cache-to: type=gha,scope=fuzz-image,mode=max

# The same restrictions the fuzz platform applies: a non-root uid, read-only
# root filesystem, no capabilities, no network, an init process.
- name: A campaign starts and reports status under the platform's restrictions
run: |
mkdir -p /tmp/fz/state /tmp/fz/artifacts /tmp/fz/failures /tmp/fz/metrics /tmp/fz/local
sudo chown -R 60001:60001 /tmp/fz
docker run -d --name smoke --user 60001:60001 --read-only --cap-drop all \
--security-opt no-new-privileges --network none --init --pids-limit 4096 \
-e FUZZ_PROJECT=zapi -e FUZZ_TARGET=round-trip \
-e FUZZ_STATE=/fuzz/state -e FUZZ_ARTIFACTS=/fuzz/artifacts \
-e FUZZ_FAILURES=/fuzz/failures -e FUZZ_METRICS=/fuzz/metrics \
-e TMPDIR=/local \
-v /tmp/fz/state:/fuzz/state -v /tmp/fz/artifacts:/fuzz/artifacts \
-v /tmp/fz/failures:/fuzz/failures -v /tmp/fz/metrics:/fuzz/metrics \
-v /tmp/fz/local:/local \
zapi-fuzz:ci
sleep 30
docker logs smoke
test "$(docker inspect -f '{{.State.Running}}' smoke)" = true
! docker logs smoke 2>&1 | grep -q '^FATAL'
cat /tmp/fz/metrics/status
awk '$1 == "execs_total" && $2 > 0 { ok = 1 } END { exit !ok }' /tmp/fz/metrics/status
# Every property passes on a clean build; a failure here is a real bug.
awk '$1 == "crashes_total" && $2 == 0 { ok = 1 } END { exit !ok }' /tmp/fz/metrics/status
docker stop -t 60 smoke
test "$(docker inspect -f '{{.State.ExitCode}}' smoke)" = 0

# Hand the tested image itself to the publish job, so what is published is
# exactly what passed above -- no second build.
- name: Save the tested image
if: github.event_name == 'workflow_dispatch' && startsWith(github.ref, 'refs/heads/')
run: docker save zapi-fuzz:ci -o /tmp/zapi-fuzz.tar

- name: Upload the tested image
if: github.event_name == 'workflow_dispatch' && startsWith(github.ref, 'refs/heads/')
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: zapi-fuzz-image
path: /tmp/zapi-fuzz.tar
retention-days: 1

publish:
name: publish
needs: build
# Manual runs on a branch only; see the header comment.
if: github.event_name == 'workflow_dispatch' && startsWith(github.ref, 'refs/heads/')
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Download the tested image
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: zapi-fuzz-image
path: /tmp

- name: Log in to GHCR
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Push
id: push
run: |
# main moves `main`; any other branch gets its own tag, sanitized to
# Docker's tag charset and length.
if [ "$GITHUB_REF_NAME" = main ]; then
tag=main
else
tag="branch-$(printf '%s' "$GITHUB_REF_NAME" | tr 'A-Z' 'a-z' | tr -c 'a-z0-9._-' '-' | cut -c1-100)"
fi
docker load -i /tmp/zapi-fuzz.tar
docker tag zapi-fuzz:ci "${IMAGE}:sha-${GITHUB_SHA::7}"
docker tag zapi-fuzz:ci "${IMAGE}:${tag}"
docker push "${IMAGE}:sha-${GITHUB_SHA::7}"
docker push "${IMAGE}:${tag}"
echo "tag=${tag}" >> "$GITHUB_OUTPUT"
echo "digest=$(docker image inspect -f '{{index .RepoDigests 0}}' "${IMAGE}:${tag}" | sed 's/.*@//')" >> "$GITHUB_OUTPUT"

- name: Summary
run: |
{
echo "### Fuzz image published"
echo
echo "\`${IMAGE}:${{ steps.push.outputs.tag }}\` now points at \`${{ steps.push.outputs.digest }}\`,"
echo "built from \`${GITHUB_SHA}\`. Fuzz platform targets following this tag"
echo "restart on the new image; their counters carry over."
} >> "$GITHUB_STEP_SUMMARY"

# Without provenance or SBOM attestations, each publish is exactly one
# package version, so this keeps the newest 20 builds.
- name: Delete old image versions
uses: actions/delete-package-versions@e5bc658cc4c965c472efe991f8beea3981499c55 # v5.0.0
with:
package-name: zapi-fuzz
package-type: container
min-versions-to-keep: 20
63 changes: 63 additions & 0 deletions Dockerfile.fuzz
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Fuzz image for the ChainSafe fuzz platform: the fast-check round-trip properties
# in tests/fuzz/, driven by tests/fuzz/platform/run-fuzz.ts. The platform provides
# the FUZZ_* environment and the /fuzz volume. See tests/fuzz/README.md.
#
# docker build -f Dockerfile.fuzz -t zapi-fuzz .

# --- build: Zig compiles the test_fuzz_numeric.node addon
FROM node:24-bookworm-slim AS build

ARG ZIG_VERSION=0.16.0
# From https://ziglang.org/download/index.json -- update together with ZIG_VERSION.
ARG ZIG_SHA256_X86_64=70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00
ARG ZIG_SHA256_AARCH64=ea4b09bfb22ec6f6c6ceac57ab63efb6b46e17ab08d21f69f3a48b38e1534f17
ARG TARGETARCH

RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl xz-utils \
&& rm -rf /var/lib/apt/lists/*

RUN case "${TARGETARCH}" in \
amd64) arch=x86_64; sum="${ZIG_SHA256_X86_64}" ;; \
arm64) arch=aarch64; sum="${ZIG_SHA256_AARCH64}" ;; \
*) echo "unsupported TARGETARCH '${TARGETARCH}'" >&2; exit 1 ;; \
esac \
&& curl -fsSL -o /tmp/zig.tar.xz \
"https://ziglang.org/download/${ZIG_VERSION}/zig-${arch}-linux-${ZIG_VERSION}.tar.xz" \
&& echo "${sum} /tmp/zig.tar.xz" | sha256sum -c - \
&& mkdir /opt/zig && tar -xJf /tmp/zig.tar.xz -C /opt/zig --strip-components=1 \
&& rm /tmp/zig.tar.xz
ENV PATH=/opt/zig:$PATH

WORKDIR /src
RUN corepack enable
COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile
COPY . .

# ReleaseSafe keeps Zig's safety checks (bounds, overflow), so those bugs panic
# instead of passing silently. -Dcpu=baseline: otherwise the addon targets the
# build runner's CPU and dies with SIGILL on a host without its extensions.
RUN --mount=type=cache,target=/root/.cache/zig \
cd tests && zig build --fork=.. --prefix ../zig-out -Doptimize=ReleaseSafe -Dcpu=baseline

# --- runtime: node, the addon, the tests and their node_modules only
FROM node:24-bookworm-slim

WORKDIR /opt/zapi
COPY --from=build /src/node_modules ./node_modules
COPY --from=build /src/zig-out/lib/test_fuzz_numeric.node ./zig-out/lib/
COPY --from=build /src/package.json ./
COPY --from=build /src/tests/fuzz ./tests/fuzz
COPY --from=build /src/tests/utils ./tests/utils
# The platform runs the container as an arbitrary non-root uid.
RUN chmod -R a+rX /opt/zapi

ARG REVISION=unknown
ENV FUZZ_REVISION=${REVISION}
LABEL org.opencontainers.image.source="https://github.com/ChainSafe/zapi" \
org.opencontainers.image.title="zapi-fuzz" \
org.opencontainers.image.description="fast-check round-trip fuzz targets for zapi" \
org.opencontainers.image.revision="${REVISION}"

ENTRYPOINT ["node", "/opt/zapi/tests/fuzz/platform/run-fuzz.ts"]
7 changes: 7 additions & 0 deletions Dockerfile.fuzz.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Build context for Dockerfile.fuzz (BuildKit reads <Dockerfile>.dockerignore).
.git
**/.zig-cache
**/zig-out
**/zig-pkg
**/node_modules
lib
22 changes: 22 additions & 0 deletions tests/fuzz/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,28 @@ Once fixed, **persist the counterexample** so it runs forever:
For numbers, use a JSON number directly. For `Uint8Array`, use a JSON array of byte values, e.g. `[0, 255, 1]`. NaN and ±Infinity aren't representable in JSON and are not currently supported as persisted seeds — those values are already in `edgeNumbers` and exercised on every run, so persisting them as regression cases adds nothing. If a future fuzz target needs persisted special-number seeds, extend `loadSeeds` and the relevant `revive*` helper at that time.
3. Commit with the fix.

## Continuous fuzzing (ChainSafe fuzz platform)

`Dockerfile.fuzz` packages these properties for the [fuzz platform](https://github.com/ChainSafe/lodestar-ansible-development/tree/main/playbooks/fuzz). `.github/workflows/fuzz-image.yml` builds and smoke-tests it on PRs. Publishing is manual: run the workflow from the Actions tab on `main` to move `ghcr.io/chainsafe/zapi-fuzz:main` (on another branch it publishes `branch-<name>`). The platform switches the campaign to each new image.

The entrypoint, `tests/fuzz/platform/run-fuzz.ts`, re-runs every `*.fuzz.test.ts` in a loop (`FUZZ_RUNS` cases per property, default 100 000, fresh random seed each time) until `SIGTERM`:

- a failing property is saved once per test + shrunk counterexample to `$FUZZ_FAILURES/failure-<hash>.json` (with seed and path for replay) and logged as `NEW_FAILURE`; fuzzing continues;
- `$FUZZ_METRICS/status` is rewritten after every iteration (`execs_total` counts property cases; no `coverage`/`corpus_entries`, since fast-check is not coverage-guided);
- counters persist in `$FUZZ_STATE/stats.json`;
- a suite that fails to load, or a run with no tests, exits non-zero (an infrastructure failure).

Run it locally as the platform does:

```bash
docker build -f Dockerfile.fuzz -t zapi-fuzz .
mkdir -p /tmp/fz && chmod 777 /tmp/fz
docker run --rm --init --read-only --network none --cap-drop all --user 60001:60001 \
-e TMPDIR=/fuzz/local -v /tmp/fz:/fuzz zapi-fuzz
```

To reproduce a saved failure, take its `seed` and `path` into the `it.only` shown above.

## Adding a new fuzz target

1. Add the round-trip export to `mod.zig`.
Expand Down
Loading
Loading