From 70f1ada10ee4f225dbcf96b7483945f44f63407c Mon Sep 17 00:00:00 2001 From: Arena Agent Date: Sat, 31 Oct 2026 12:57:46 +0000 Subject: [PATCH 1/3] fix: correctness and security pass over attacks and defenses (0.2.0) Several public functions either raised on every call or returned silently wrong results. This makes each function do what its name claims, and adds the guardrails (tests, lint, packaging, CI) that were missing. Attacks - jsma: was indexing a rank-4 Jacobian with 5 indices and mutating an eager tensor in place; rewritten as a real saliency-map attack for any input rank. - spsa: passed a tensor (not a tf.Variable) to the optimizer and hard-coded rank-4 inputs; now a correct finite-difference estimate with Adam. - ead: returned the input unchanged with default args (alpha == beta); now a proper ISTA/proximal step on the elastic-net objective. - cw: the L2 term and kappa were inert (x was already clipped); now a real tanh-space C&W with a fixed target class. - deepfool: ignored any budget and returned out-of-range values; now computes per-class boundaries and respects epsilon/clip range. - fgsm: output is clipped to the valid range. - all attacks: accept sparse/one-hot/binary labels, honour from_logits, call the model with training=False, support clip_min/clip_max, and no longer freeze targeted attacks at step 0. - deepfool/jsma/spsa no longer project against the whole batch. - word_swap/char_swap preserve whitespace, newlines and punctuation. Defenses - every preprocessing defense can now be saved/reloaded (layers were defined inside function bodies, so no defended model was serialisable). - jpeg_compression, feature_squeezing, total_variation_minimization: restore the gradient via straight-through so attacks against a defended model work. - median_smoothing is a real median filter (was avg_pool2d); gaussian_blur honours sigma (was avg_pool2d). - pixel_deflection no longer raises; deterministic when seeded. - adversarial_training batches attack generation and keeps the caller's optimizer/loss; defensive_distillation stops double-applying softmax and scales by T^2; ALP now generates adversarial examples; gradient_masking warns that it is not a security defense. - randomized_smoothing clips and is seedable, plus smooth_predict() with a certified-radius estimate. - invalid parameters raise instead of producing NaN; subclassed/multi-input models get a clear TypeError. Packaging / CI / docs - replace setup.py ({{VERSION_PLACEHOLDER}}) with pyproject.toml; pip install . works from a checkout again. - bound dependencies, require Python >= 3.9. - add 171 tests, ruff config and a CI workflow; rewrite the publish workflow with SHA-pinned actions, a tag/version check, a test gate and PyPI Trusted Publishing. - add SECURITY.md, CODE_OF_CONDUCT.md, CHANGELOG.md, CODE_REVIEW.md, Dependabot and pre-commit; correct the README support matrix. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com> --- .github/dependabot.yml | 12 + .github/workflows/build_and_publish.yml | 81 +- .github/workflows/ci.yml | 73 ++ .gitignore | 12 +- .pre-commit-config.yaml | 6 + CHANGELOG.md | 142 +++ CODE_OF_CONDUCT.md | 47 + CODE_REVIEW.md | 371 ++++++ CONTRIBUTING.md | 25 +- SECURITY.md | 54 + deepdefend/__init__.py | 88 +- deepdefend/_utils.py | 389 ++++++ deepdefend/attacks.py | 1049 ++++++++++++----- deepdefend/defenses.py | 1433 ++++++++++++++++------- pyproject.toml | 63 + readme.md | 302 ++--- requirements.txt | 4 +- setup.py | 35 - tests/conftest.py | 96 ++ tests/test_api.py | 62 + tests/test_attacks.py | 271 +++++ tests/test_defenses.py | 319 +++++ 22 files changed, 4011 insertions(+), 923 deletions(-) create mode 100644 .github/dependabot.yml create mode 100644 .github/workflows/ci.yml create mode 100644 .pre-commit-config.yaml create mode 100644 CHANGELOG.md create mode 100644 CODE_OF_CONDUCT.md create mode 100644 CODE_REVIEW.md create mode 100644 SECURITY.md create mode 100644 deepdefend/_utils.py create mode 100644 pyproject.toml delete mode 100644 setup.py create mode 100644 tests/conftest.py create mode 100644 tests/test_api.py create mode 100644 tests/test_attacks.py create mode 100644 tests/test_defenses.py diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..3f9a6cf --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,12 @@ +version: 2 +updates: + - package-ecosystem: "pip" + directory: "/" + schedule: + interval: "weekly" + open-pull-requests-limit: 5 + + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" diff --git a/.github/workflows/build_and_publish.yml b/.github/workflows/build_and_publish.yml index 4a9ec6b..9054ef8 100644 --- a/.github/workflows/build_and_publish.yml +++ b/.github/workflows/build_and_publish.yml @@ -3,32 +3,63 @@ name: Publish Python ๐Ÿ distributions ๐Ÿ“ฆ to PyPI on: push: tags: - - '*' + - "v*" + +# Least privilege at the workflow level; the publish job widens `id-token` only. +permissions: + contents: read jobs: - build-n-publish: - name: Build and publish Python ๐Ÿ distributions ๐Ÿ“ฆ to PyPI + build: + name: Build and verify distributions + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + persist-credentials: false + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: "3.12" + - name: Install build tooling + run: | + python -m pip install --upgrade pip build twine + - name: Verify the tag matches the packaged version + run: | + python - <<'PY' + import os, sys, tomllib + tag = os.environ["GITHUB_REF_NAME"].lstrip("v") + with open("pyproject.toml", "rb") as fh: + version = tomllib.load(fh)["project"]["version"] + if tag != version: + sys.exit(f"Tag {tag!r} does not match pyproject version {version!r}") + print(f"Building {version}") + PY + - name: Run the test-suite before publishing + run: | + python -m pip install -e ".[dev]" + python -m pytest -q + - name: Build sdist and wheel + run: | + python -m build + python -m twine check dist/* + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: dist + path: dist/ + + publish: + name: Publish to PyPI + needs: build runs-on: ubuntu-latest + environment: + name: pypi + url: https://pypi.org/project/deepdefend/ + permissions: + # Trusted Publishing (OIDC). No long-lived API token is stored. + id-token: write steps: - - uses: actions/checkout@master - - name: Set up Python 3.12 - uses: actions/setup-python@v3 - with: - python-version: '3.12' - - name: Install pypa/setuptools - run: >- - python -m - pip install setuptools wheel - - name: Extract tag name - id: tag - run: echo ::set-output name=TAG_NAME::$(echo $GITHUB_REF | cut -d / -f 3) - - name: Update version in setup.py - run: >- - sed -i "s/{{VERSION_PLACEHOLDER}}/${{ steps.tag.outputs.TAG_NAME }}/g" setup.py - - name: Build a binary wheel - run: >- - python setup.py sdist bdist_wheel - - name: Publish distribution ๐Ÿ“ฆ to PyPI - uses: pypa/gh-action-pypi-publish@master - with: - password: ${{ secrets.PYPI_API_TOKEN }} \ No newline at end of file + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + with: + name: dist + path: dist/ + - uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # release/v1 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..d04bcf8 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,73 @@ +name: CI + +on: + push: + branches: [master, main] + pull_request: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + lint: + name: Lint + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + persist-credentials: false + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: "3.12" + - name: Install ruff + run: python -m pip install "ruff>=0.6" + - name: Ruff + run: ruff check . + + test: + name: Tests (Python ${{ matrix.python-version }}) + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + python-version: ["3.10", "3.11", "3.12"] + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + persist-credentials: false + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: ${{ matrix.python-version }} + cache: pip + - name: Install package + run: | + python -m pip install --upgrade pip + python -m pip install -e ".[dev]" + - name: Run tests + run: python -m pytest -q + + build: + name: Build distribution + runs-on: ubuntu-latest + needs: [lint, test] + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + persist-credentials: false + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: "3.12" + - name: Build sdist and wheel + run: | + python -m pip install --upgrade pip build twine + python -m build + python -m twine check dist/* + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: dist + path: dist/ diff --git a/.gitignore b/.gitignore index 27ee28f..c96357f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,11 @@ -__pycache__ -*.pyc +__pycache__/ +*.py[cod] +*.egg-info/ dist/ build/ -venv/ \ No newline at end of file +venv/ +.venv/ +.pytest_cache/ +.ruff_cache/ +.coverage +htmlcov/ diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..e75b938 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,6 @@ +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: 75b98813cfb7e663870a28c74366a1e99d7bfe79 # v0.6.9 + hooks: + - id: ruff + args: [--fix] diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..d4cfb92 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,142 @@ +# Changelog + +All notable changes to DeepDefend are documented here. The format loosely +follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this +project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [0.2.0] + +This release is a correctness and security pass over the whole package. Many +functions that previously returned *silently wrong* results (or crashed) now do +what their name and documentation claim. Because the numerical output of several +attacks and defenses changes, the minor version was bumped. + +### Fixed โ€” attacks + +- **`jsma`** no longer raises on every call (it indexed a rank-4 Jacobian with + five indices and mutated an eager tensor in place). It is now a real + saliency-map attack that works on any input rank. +- **`spsa`** no longer raises on every call (it passed a tensor, not a + `tf.Variable`, to the optimizer and hard-coded rank-4 inputs). The gradient + estimate now uses the correct finite difference + `(L(x+ฮดd) โˆ’ L(xโˆ’ฮดd)) / (2ฮด)`, the optimiser is Adam, and any number of + samples and dimensions is accepted. +- **`ead`** no longer returns the input unchanged with its default arguments. + It now performs a proper ISTA/proximal step on the elastic-net objective. +- **`cw`** is now the Carlini & Wagner L2 attack: the tanh-space optimisation + actually minimises distortion and the `c` / `kappa` parameters change the + result (previously they were inert). The untargeted target class is fixed at + the runner-up class instead of being recomputed every step. +- **`deepfool`** is now DeepFool: per-class decision boundaries, an optional + Lโˆž budget, and output that stays inside the valid data range (it used to + return values such as `-0.90` and `2.08` for `[0, 1]` inputs). +- **`fgsm`** output is clipped to the valid data range instead of overshooting + to `-0.01` / `1.007`. +- **All attacks** now accept sparse integer labels, one-hot labels and binary + targets; previously every attack rejected the sparse-label format used by most + Keras examples. +- **All attacks** now auto-detect `from_logits` (with an override), call the + model with `training=False`, and support an explicit `clip_min` / `clip_max` + for normalised inputs. +- **Targeted attacks** no longer freeze at step 0 (the early-stop mask had the + logic inverted for targeted mode). +- **`pgd`** gained `random_start` and no longer duplicates `bim`. +- **DeepFool / JSMA / SPSA** no longer project against the whole batch inside + their per-sample loops. +- **`word_swap` / `char_swap`** preserve whitespace, newlines and punctuation, + accept a `seed`, and `word_swap` no longer silently returns the input when + `swap_dict` is missing. + +### Fixed โ€” defenses + +- **All preprocessing defenses** can now be saved and reloaded. Their layers + were previously defined inside the function body, so every defended model + failed at `model.save()` / `load_model()`. +- **`jpeg_compression`**, **`feature_squeezing`** and + **`total_variation_minimization`** no longer break the gradient: attacks + against a defended model used to raise `ValueError` (JPEG) or silently return + a zero perturbation (TV, quantisation). They now use a straight-through + gradient. +- **`median_smoothing`** is a real median filter again (it was calling + `avg_pool2d`). +- **`gaussian_blur`** builds an actual Gaussian kernel and honours `sigma` + (the parameter was previously ignored). +- **`pixel_deflection`** no longer raises on every call, works inside + `tf.function`, and is reproducible when a `seed` is given. +- **`adversarial_training`** generates adversarial examples in batches instead + of one sample at a time (~500ร— fewer model calls), honours the caller's + optimiser and loss instead of forcing Adam + categorical cross-entropy, and + accepts `epochs`, `batch_size`, `verbose` and a custom `attack`. +- **`defensive_distillation`** no longer applies softmax twice, scales the + distillation term by `Tยฒ`, evaluates the teacher in inference mode, and works + for both probability and logit outputs. +- **`adversarial_logit_pairing`** generates adversarial examples inside the + training step; the previous implementation paired two models on clean inputs. +- **`gradient_masking`** no longer crashes on `None` gradients and now emits a + `RuntimeWarning` explaining that it is a training regulariser, not an + inference-time defense. +- **`randomized_smoothing`** clips the noisy input, is reproducible when + seeded, and ships with `smooth_predict()` for Monte-Carlo prediction and a + certified-radius estimate. +- **`feature_denoising`** has an explicit `method` (`median` / `gaussian`) and a + new `FeatureDenoisingBlock` layer for feature-level denoising. +- **`input_transformation`** raises instead of silently returning an + undefended model when no transformation is given. +- **`word_masking`** preserves whitespace and accepts a `seed`. +- **All defenses** raise a clear `TypeError` for subclassed or multi-input + models instead of an obscure `AttributeError`, and reject invalid parameters + (`bit_depth` outside 1-8, `num_bins < 2`, `quality` outside 1-100, even kernel + sizes, negative noise levels, ...) instead of producing `NaN`. + +### Added + +- `deepdefend.__version__`, `deepdefend.__all__`, and lazy imports: every public + function is reachable as `deepdefend.`, and `import deepdefend` no + longer imports TensorFlow. +- `deepdefend/smooth_predict` for Monte-Carlo randomized smoothing. +- 164 tests covering attacks, defenses, serialisation, gradients, label + formats, input validation and the public API. +- CI: lint + test matrix (3.10-3.12) + reproducible build on every push and PR. +- `SECURITY.md`, `CODE_OF_CONDUCT.md`, Dependabot, pre-commit and + `pyproject.toml`. + +### Changed + +- `setup.py` was replaced by `pyproject.toml`; the version is now static and + `pip install .` works from a checkout (it previously failed because the + version was a `{{VERSION_PLACEHOLDER}}`). +- Dependencies are bounded (`numpy<3`, `tensorflow>=2.13,<3`) and + `python_requires` is `>=3.9` (TensorFlow 2.x is not available on 3.6-3.8). +- `pgd` / `bim` / `mim` default `alpha` is now `epsilon / 4` (previously + `0.01`, which equals `epsilon` for the default settings). +- `jsma` defaults are `theta=0.0`, `gamma=0.2`. +- `cw` defaults are `epsilon=None` (unconstrained), `c=10.0`. +- `spsa` defaults are `num_steps=50`, `learning_rate=0.05`, `spsa_samples=32` + probe directions, `delta=epsilon/5`. +- `median_smoothing` is documented as an alias of `spatial_smoothing`. +- The publish workflow uses SHA-pinned actions, a tag/version consistency check, + a test gate, `python -m build`, `twine check`, PyPI Trusted Publishing (OIDC) + and a protected `pypi` environment. + +### Removed + +- The `setup.py`-based `{{VERSION_PLACEHOLDER}}` release mechanism. + +## [0.1.5] + +- Added MIM and EAD attacks, word/character swap attacks, pixel deflection, + Gaussian blur, TV minimisation and median smoothing defenses, word masking, + and a support table. + +## [0.1.4] + +- Added the SPSA attack and the JPEG compression defense. + +## [0.1.3] + +- Added randomized smoothing, feature denoising, thermometer encoding, + adversarial logit pairing and spatial smoothing. + +## [0.1.2] + +- Three new attacks and three new defenses. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..8353696 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,47 @@ +# Code of Conduct + +## Our pledge + +We as members, contributors, and maintainers pledge to make participation in the +DeepDefend community a harassment-free experience for everyone, regardless of +age, body size, visible or invisible disability, ethnicity, sex characteristics, +gender identity and expression, level of experience, education, socio-economic +status, nationality, personal appearance, race, religion, or sexual identity and +orientation. + +## Our standards + +Examples of behaviour that contributes to a positive environment: + +- Demonstrating empathy and kindness toward other people. +- Being respectful of differing opinions, viewpoints, and experiences. +- Giving and gracefully accepting constructive feedback. +- Accepting responsibility and apologising to those affected by our mistakes. +- Focusing on what is best for the community and the project. + +Examples of unacceptable behaviour: + +- The use of sexualised language or imagery, and sexual attention of any kind. +- Trolling, insulting or derogatory comments, and personal or political attacks. +- Public or private harassment. +- Publishing others' private information without their explicit permission. +- Submitting attack code intended to harm third-party systems, or requesting + help to do so. +- Other conduct which could reasonably be considered inappropriate in a + professional setting. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behaviour may be +reported to the maintainers at **infinitode.ltd@gmail.com**. All complaints will +be reviewed and investigated promptly and fairly, and the privacy and security +of the reporter will be respected. + +Maintainers may remove, edit or reject comments, commits, code, issues and other +contributions that are not aligned with this Code of Conduct, and will +communicate the reasons for moderation decisions when appropriate. + +## Attribution + +This Code of Conduct is adapted from the +[Contributor Covenant](https://www.contributor-covenant.org), version 2.1. diff --git a/CODE_REVIEW.md b/CODE_REVIEW.md new file mode 100644 index 0000000..fb50cad --- /dev/null +++ b/CODE_REVIEW.md @@ -0,0 +1,371 @@ +# DeepDefend โ€” Full Code Review & Vulnerability Assessment + +**Version reviewed:** `0.1.5` (branch `arena/b52ef87b-deepdefend`, commit `cda6c32`) +**Files reviewed:** `deepdefend/__init__.py`, `deepdefend/attacks.py` (430 lines), `deepdefend/defenses.py` (565 lines), `setup.py`, `requirements.txt`, `readme.md`, `LICENSE`, `.github/workflows/build_and_publish.yml`, `CONTRIBUTING.md`, issue templates +**Method:** static review of every function **plus** dynamic verification of each suspected defect against a real Keras model (`tensorflow-cpu 2.16.2` / Keras 3.15, Python 3.11). Every claim below marked *(measured)* was reproduced by executing the package; raw output is in the appendix. + +> This is a security library: a function that **silently returns something wrong** is worse than one that crashes, because downstream users report "no vulnerability found" based on it. That severity ordering drives this report. + +--- + +## 0. Executive summary + +| Area | State | +|---|---| +| Attacks | 7/11 functions are broken, silently inert, or are not the algorithm they claim to be | +| Defenses | 4 crash or are bit-identical to each other; 6 are not the technique they claim; 13 of 15 wrapped models cannot be saved | +| Packaging | `pip install .` from a checkout **fails outright** | +| CI/CD | Unpinned mutable actions + long-lived PyPI token + publishes on any tag | +| Tests | **None.** No test suite, no test workflow, no lint config | + +Of the 11 attacks, **5 raise an exception on normal input** and **2 silently produce no perturbation at all**. Of the 15 model-wrapping defenses, **1 always crashes** and **3 break gradient flow so badly that attacks against the defended model raise or silently no-op**. + +Recommended target version: **0.2.0** (several fixes are intentionally behaviour-changing). + +--- + +## 1. CRITICAL โ€” broken, inert, or scientifically false + +### C1. `attacks.jsma` always raises โ€” implemented incorrectly +`attacks.py:325-376` + +* `saliency_map = np.zeros(input_shape)` with `input_shape` a `TensorShape`; loops `range(input_shape[1..3])` assume a hard-coded 4-D image input. +* `jacobian[0, i, j, k, :]` indexes a rank-4 tensor with 5 indices. +* **Measured:** `InvalidArgumentError: Index out of range using input dim 4; input has only 4 dimensions`. +* `adversarial_example[...] += gamma` mutates an `EagerTensor` in place โ€” illegal even if the above were fixed. +* Conceptually it is not JSMA: it sums the whole Jacobian instead of computing the *saliency map* over (target, other-class) derivative pairs, and applies a fixed `gamma` instead of the `theta` pixel budget. + +**Verdict:** unusable. Needs a rewrite. + +### C2. `attacks.spsa` always raises; hard-codes rank-4 inputs +`attacks.py:378-430` + +* `perturbation = tf.zeros_like(x)` is a **tensor, not a `tf.Variable`**, yet it is passed to `optimizer.apply_gradients(...)` and later `.assign(...)` โ€” both require a Variable. +* `tf.tile(x, [spsa_samples, 1, 1, 1])` hard-codes 4 dimensions โ€” **measured** failure on a `(1, 10)` Dense input, contradicting the README's "Numeric โœ…" matrix entry. +* `y_tiled = tf.tile(y, [128])` produces rank-2 labels for `sparse_categorical_crossentropy` โ€” **measured** failure even on a valid 4-D batch. +* The gradient estimate `mean(loss * delta_x) / delta` is not SPSA: it needs the loss *difference* `(L(x+ฮด) โˆ’ L(xโˆ’ฮด)) / (2ฮด)`, otherwise the absolute loss magnitude dominates the estimate. +* `adversarial_example` is unbound if `num_steps <= 0` โ†’ `UnboundLocalError`. + +### C3. `defenses.pixel_deflection` always raises +`defenses.py:405-451` + +* **Measured:** `ValueError: maxval must be a scalar; got a tensor of shape [2]` from `tf.random.uniform([count, 2], 0, [h, w], ...)`. Any Functional-model construction traces this, so it can never work. +* Also fails inside `tf.function` (no `fn_output_signature`), and duplicate scatter indices silently drop updates. +* The in-code comments admit it is "placeholder logic" and "an approximation". + +### C4. `attacks.ead` is a **complete no-op** with default arguments +`attacks.py:101-145` + +Default `beta=0.01` and `alpha=0.01` give `maximum(|diff| - beta, 0)` where `|diff| == alpha == 0.01` โ†’ **exactly zero**. All `num_steps` are spent producing the input unchanged. + +* **Measured:** `max|ฮด| = 0.0000`, `mean|ฮด| = 0.0000` over 10 steps at `eps=0.02`. +* It is also not EAD: no L2 term, no per-step adversarial objective, no early stop โ€” it is PGD with a shrink applied *after* the update and then clipped, which partially cancels it. + +**Verdict:** a user running EAD concludes their model is already robust. This is the single most dangerous defect in the package. + +### C5. `attacks.cw` โ€” `c` and `kappa` are dead parameters +`attacks.py:252-292` + +`loss + c * tf.norm(x - tf.clip_by_value(x, 0, 1))**2 - kappa` โ€” `x` has already been clipped to `[0,1]` on every previous iteration, so the L2 term is **identically 0** and `kappa` adds a constant whose gradient is **identically 0**. + +* **Measured:** `cw(c=1, kappa=0)` and `cw(c=1000, kappa=50)` produce **bit-identical** output. +* No tanh-space optimisation, no L2 minimisation, no confidence margin (`- kappa` has the wrong sign for C&W anyway), FGSM-style `alpha*sign(grad)` step. + +**Verdict:** it is PGD with two decorative arguments. A C&W benchmark built on this is meaningless. + +### C6. `attacks.deepfool` is unbounded and returns invalid inputs +`attacks.py:294-323` + +* Adds `gradient / ||gradient||` with step size **1.0** and never clips or projects. +* **Measured:** output range `[-0.900, 2.080]` for an input in `[0,1]` โ€” a completely invalid image. +* Has no `epsilon` parameter at all, so the "minimum perturbation" property that defines DeepFool cannot be expressed. +* No per-class linearisation; it is gradient ascent in the loss direction, which is closer to an *anti*-attack than DeepFool. + +### C7. `attacks.fgsm` returns out-of-range data +`attacks.py:21-53` + +* No clipping at all (unlike PGD/BIM/MIM). +* **Measured:** `min=-0.010, max=1.007` for `x โˆˆ [0,1]`, `epsilon=0.01`. Feeding that back into the model gives garbage and it is not a valid image. +* No `gradient is None` handling โ†’ `tf.sign(None)` raises a confusing `AttributeError`. + +### C8. Every attack rejects `sparse_categorical` integer labels +`attacks.py` โ€” all of `fgsm, pgd, bim, mim, cw, ead, deepfool` + +* **Measured:** all seven raise `Arguments target and output must have the same rank (ndim). Received target.shape=(4,), output.shape=(4, 3)`. +* Sparse integer labels are *the* default for MNIST/CIFAR examples in the Keras documentation, i.e. the package's primary audience. +* `pgd`/`bim`/`cw`/`deepfool` hard-code `CategoricalCrossentropy`; `fgsm`/`mim`/`ead` use a fragile heuristic (`y.shape[-1] == 1 or len(y.shape) == 1`) that misclassifies a 1-D integer label tensor as a *binary* target. +* `from_logits` is never exposed, so any model not ending in `softmax` is silently attacked with the wrong loss. + +### C9. Every wrapped defense model is unsaveable +`defenses.py` โ€” all 11 preprocessing wrappers + +Every layer class (`SqueezeLayer`, `NoiseLayer`, `ThermometerLayer`, `JPEGCompressLayer`, `DeflectionLayer`, `BlurLayer`, `TVLayer`, `MedianLayer`, `SmoothLayer`, `DenoiseLayer`, `TransformationLayer`) is **defined inside the function body**, so it has no importable qualified name and no `get_config()`. + +* **Measured:** `model.save(...)` โ†’ `TypeError: could not be deserialized properly` for `feature_squeezing`, `gaussian_blur`, `randomized_smoothing` (3/3). +* `tf.keras.models.clone_model()` on a defended model fails for the same reason. +* This makes the defenses unusable in production: you cannot train today and serve tomorrow. + +### C10. `jpeg_compression` destroys the gradient โ€” attacks on the defended model crash +`defenses.py:396-397` + +`tf.image.encode_jpeg`/`decode_jpeg` are non-differentiable. + +* **Measured:** `tape.gradient(...)` โ†’ `None`. +* **Measured:** `attacks.fgsm(jpeg_compression(model), x, y)` โ†’ `ValueError: Attempt to convert a value (None)`. +* The library's headline use case โ€” "defend, then evaluate the defense with an attack" โ€” therefore fails. Needs a straight-through-estimator wrapper. +* Also hard-codes `channels=3` (breaks grayscale), assumes `x โˆˆ [0,1]`, and cannot run on GPU/TPU. + +### C11. `gradient_masking` provides zero protection at inference +`defenses.py:93-129` + +It masks gradients **during training only**. An attacker queries the model's own gradients, which are untouched. + +* **Measured:** `fgsm(base_model) == fgsm(gradient_masking(base_model))` โ€” bit-identical. +* The docstring claims it "makes them less informative for adversarial attackers", which is false. Gradient masking is a known *false* defense in the literature (Obfuscated Gradients, Athalye et al. 2018). It must be relabelled as a training regulariser, and the README's โœ… row should carry a warning. + +### C12. `median_smoothing` is an average pool โ€” not a median filter +`defenses.py:552-560` + +* **Measured:** output is bit-identical to `tf.nn.avg_pool2d(x, 3, 1, "SAME")`. +* Ironically a real median implementation already exists in `spatial_smoothing` (line 347). This is a copy-paste regression. + +### C13. `gaussian_blur` ignores `sigma` entirely +`defenses.py:467-476` + +* Never builds a Gaussian kernel; calls `avg_pool2d`. +* **Measured:** `gaussian_blur(sigma=0.1)` โ‰ก `gaussian_blur(sigma=9.0)`. + +### C14. `total_variation_minimization` silently kills attack gradients +`defenses.py:503-514` + +* Runs an inner `GradientTape` inside `Layer.call`; the outer tape records the inner optimisation, so the composed gradient w.r.t. the input is **measured as all-zero**. +* Consequence: `fgsm(tv_defended_model, ...)` returns an example that is *not* adversarial, with no error โ€” exactly the silent failure mode this report is about. +* Also un-trainable, adds second-order cost, and cannot be used in a `tf.function` training loop. + +--- + +## 2. HIGH โ€” wrong results, bad defaults, unsafe API + +### H1. `adversarial_training` is crippled and ignores the caller's configuration +`defenses.py:28-60` + +* **Per-sample loop** (`for i in range(len(x))`): one forward+backward pass per image. **Measured: 3.08 s for 200 images** โ†’ ~13 minutes for 50 000 MNIST samples, before training starts. +* Hard-codes `epochs=10`, `batch_size=32`, `optimizer='adam'`, `loss='categorical_crossentropy'`. + **Measured:** a model compiled with `SGD(0.001)` + `KLDivergence` + `mae` comes back with **Adam + categorical_crossentropy + no `mae`** โ€” the user's compile config is silently discarded. +* **Measured:** fails on sparse labels (`target.shape=(1,)`). +* No `verbose`, no `shuffle`, no `validation_data`, no early stop; refuses lists/`tf.data.Dataset` (`len()`/slicing). + +### H2. No input validation anywhere โ†’ NaN or crashes from legal-looking arguments +**Measured:** + +| Call | Result | +|---|---| +| `fgsm(..., epsilon=-1)` | accepted, returns values as low as `-0.9997` | +| `pgd(..., num_steps=0)` | accepted, returns the input silently | +| `feature_squeezing(model, bit_depth=0)` | **NaN** (division by `2**0 - 1 == 0`) | +| `feature_squeezing(model, bit_depth=9)` | `TypeError` (floordiv int32 by `0.5`) | +| `thermometer_encoding(model, num_bins=0)` | **NaN** | +| `spsa(..., num_steps=0)` | `UnboundLocalError` | + +A security library must not emit NaN. Every numeric parameter needs a documented range and a `ValueError`. + +### H3. `feature_squeezing` quantisation is subtly biased and non-differentiable +`defenses.py:85` + +For `bit_depth=4`: `floor(x*255/16)/15`. The level index is divided by `2**b - 1` (15) rather than by `255/16`, so the mapping is stretched by ~6% and `x=0.5 โ†’ 0.4667`. Correct is `round(x*(2**b-1))/(2**b-1)` or a `255`-based round-trip. Also `floor`+`cast` โ‡’ **measured** `gradient is None`, so no attack can be run through it (no straight-through estimator). + +### H4. Every preprocessing wrapper breaks on non-Functional models +`infrastructure of all wrappers, e.g. defenses.py:88, 155, 230, 371, 399, 448, 478, 516, 562` + +* **Measured:** subclassed model โ†’ `AttributeError: 'Sub' object has no attribute 'input_shape'`. +* **Measured:** multi-input model โ†’ `ValueError: Cannot convert '[(None, 4)]' to a shape`. + +Both need an explicit `TypeError`/`ValueError` with a clear message, plus support for subclassed models via `model.inputs`. + +### H5. `randomized_smoothing` is not randomized smoothing +`defenses.py:207-233` + +* Adds inference-time Gaussian noise and nothing else: no noise-augmented training, no Monte-Carlo vote over N samples, no certified radius. Inference-time-only noise is a well-documented *broken* defense. +* No clip to `[0,1]` after adding noise; no `training` gate; no seed. +* **Measured:** two identical calls return different outputs (non-deterministic `predict`). + +### H6. `thermometer_encoding` is plain uniform quantisation +`defenses.py:279-283` + +`floor(x*bins)/bins` is quantisation, not thermometer encoding (which expands `C โ†’ Cร—L` binary features and requires retraining the first layer). Claims in the docstring and README are false. `num_bins=0` โ†’ NaN. + +### H7. `defensive_distillation` applies softmax twice and drops `Tยฒ` +`defenses.py:182-200` + +* `tf.nn.softmax(teacher_predictions / T)` on a model that already ends in `softmax` โ†’ softmax of softmax, then `CategoricalCrossentropy(from_logits=False)` on that. Mathematically not distillation. +* Teacher is called **without `training=False`**, so its BN/dropout is live and the soft targets are noisy. +* Missing the `Tยฒ` scaling factor and the KL formulation; the typical practice is CE against the same temperature-softened distribution. + +### H8. `adversarial_logit_pairing` does not implement ALP +`defenses.py:308-321` + +Both models are called on the **same clean batch** `x`; the pairing loss therefore just forces two models to agree on clean data. ALP requires generating an adversarial example inside the train step and pairing `logits(x)` with `logits(x_adv)`. + +### H9. Packaging and release pipeline (supply chain) + +* **`pip install .` from a checkout FAILS.** `setup.py` ships `version='{{VERSION_PLACEHOLDER}}'`; **measured** `pip install .` โ†’ `Encountered error while generating package metadata`. Only the tag-time `sed` in CI produces a valid version, so local builds, editable installs, and `python -m build` are all broken. Replace with a real static version (or `setuptools_scm` / `importlib.metadata`). +* **`pypa/gh-action-pypi-publish@master` and `actions/checkout@master`** โ€” mutable refs with `PYPI_API_TOKEN` in the environment: a compromise or force-push of the action branch means arbitrary code executes with publishing rights. Pin every action to a commit SHA. +* **`echo ::set-output`** is removed on modern runners โ†’ the version step is broken/warned. +* No `permissions:` block (defaults are broad), no `environment: pypi` protection rule, no tag pattern guard โ†’ **any** tag publishes. +* Long-lived `PYPI_API_TOKEN` secret instead of [PyPI Trusted Publishing] (OIDC `id-token: write`). +* No test/lint step before publish; uses deprecated `python setup.py sdist bdist_wheel` instead of `python -m build`; no `twine check`. +* `install_requires=['numpy','tensorflow']` is fully unbounded, so a future `numpy 3` / `tensorflow 3` breaks the package on install. `tensorflow` is unavailable on many platforms; consider a `tensorflow-cpu` extra. +* Classifiers claim Python **3.6** and `python_requires='>=3.6'`, but TF 2.x requires โ‰ฅ3.9 โ€” the package is installed on interpreters where it cannot import. +* No `license=`/`license_files=`, yet `License :: OSI Approved :: MIT License` is declared while `LICENSE` is "MIT License (Modified)" (not OSI-approved, no SPDX id). Remove the classifier or relicense. +* `Development Status :: 5 - Production/Stable` is not defensible given the above. +* No `pyproject.toml` (PEP 517/518), no `MANIFEST.in`, no `py.typed`, no extras for tests. + +### H10. Public API is incomplete; no `__version__` +`__init__.py` + +* **Measured MISSING:** `deepdefend.mim`, `.ead`, `.word_swap`, `.char_swap`, `.pixel_deflection`, `.gaussian_blur`, `.word_masking`, `.median_smoothing`, `.__version__`. All of these exist in the submodules but are not re-exported, and there is no `__all__`. +* No `__version__` at all makes bug reports unactionable. + +--- + +## 3. MEDIUM โ€” maintainability, fidelity, performance + +* **M1. Duplicated bodies:** `pgd` (188-219) and `bim` (220-251) are byte-identical; `fgsm`/`pgd`/`bim`/`cw`/`deepfool`/`mim`/`ead` each re-implement loss selection and projection. The 11 preprocessing wrappers repeat the same 4-line `Input โ†’ Layer โ†’ model โ†’ Model` block. Extract `_utils` + `_wrap()`. +* **M2. Local layer classes** have no `get_config()`/`compute_output_shape()`, blocking serialization even if moved to module level. +* **M3. Fixed `[0,1]` assumption everywhere.** No `clip_min`/`clip_max` on any attack; real models are fed ImageNet-normalised tensors, where clipping to `[0,1]` silently destroys the input. +* **M4. No PGD `random_start`**, no early stopping on misclassification, no `@tf.function` anywhere (each step is an eager op, ~5-50ร— slower than necessary). +* **M5. Text helpers destroy whitespace.** `" ".join(text.split())` collapses runs of spaces, tabs and newlines. **Measured:** `word_masking("a b\nc", mask_prob=1.0)` โ†’ `"[MASK] [MASK] [MASK]"`; `char_swap("line one\nline two", 1.0)` โ†’ `"lnie noe lien wto"` (newline gone). Tokenise with `re` and preserve separators. +* **M6. Non-reproducible randomness:** `np.random.*` globals in `char_swap`/`word_masking`, `tf.random` without seeds in `randomized_smoothing`/`pixel_deflection`. No `seed` parameter on any function. +* **M7. `spatial_smoothing`** allocates `Bร—Hร—Wร—kยฒร—C` (81ร— the input for `k=9`); mislabelled (it *is* a median filter, not spatial smoothing); crashes on non-4-D input. +* **M8. `input_transformation()` with its default `transformation_function=None` is a silent identity** โ€” returns a model with no defense, named as if defended. +* **M9. Documentation is materially inaccurate:** the support matrix marks SPSA "Numeric โœ…" (fails, H2), and PGD/EAD/DeepFool "Text โœ… (Embeddings)" (unverified); the 0.1.5 changelog claims "Fixed logical errors in several defense functions" while C4/C5/C12/C13 remain; the Python-3.6 claim is impossible with the TF dependency. +* **M10. `attacks.py` calls `model(x)` without `training=False`.** In Keras 3 this defaults to inference, so the impact is currently low (**measured:** deterministic on a dropout model), but the behaviour of `training=None` is context-dependent (inside `fit`'s `train_step`, and in older TF versions, it can be `True`). Pass it explicitly. +* **M11. Circular/pointless imports:** `defenses.py:26` does `from deepdefend import attacks`; `__init__.py:1` does `import deepdefend` inside the package's own `__init__`. Both work only by accident. Note that `import deepdefend` also eagerly pulls in TensorFlow (~5 s, ~600 MB RSS for any consumer, even one that only wants `word_swap`). +* **M12. Zero tests, zero CI checks.** `CONTRIBUTING.md` instructs contributors to "run tests" โ€” there are none, and no workflow runs anything on push/PR. No `ruff`/`flake8`, no pre-commit, no Dependabot, no `SECURITY.md` (notable for a security-focused project), no `CODE_OF_CONDUCT.md`, no `pyproject.toml`. + +--- + +## 4. Function-by-function verdict + +### `deepdefend.attacks` +| Function | Status | Key defect | +|---|---|---| +| `fgsm` | โš ๏ธ wrong | unclipped output (C7); binary/categorical heuristic; no sparse labels | +| `pgd` | โš ๏ธ weak | no `random_start`, no `clip_min/max`, no sparse labels, dup of `bim` | +| `bim` | โš ๏ธ weak | byte-identical to `pgd` | +| `mim` | โš ๏ธ weak | L1-norm is over the whole batch (cross-sample coupling), same as PGD otherwise | +| `cw` | โŒ | `c`/`kappa` inert, not C&W (C5) | +| `ead` | โŒ **no-op** | default params โ‡’ zero perturbation (C4) | +| `deepfool` | โŒ | unbounded, no ฮต, invalid images (C6) | +| `jsma` | โŒ crash | always raises (C1) | +| `spsa` | โŒ crash | not a Variable, rank-4 only, bad estimator (C2) | +| `word_swap` | โš ๏ธ | whitespace loss, silent no-op on `swap_dict=None` | +| `char_swap` | โš ๏ธ | whitespace loss, unseeded, `swap_prob` unvalidated | + +### `deepdefend.defenses` +| Function | Status | Key defect | +|---|---|---| +| `adversarial_training` | โš ๏ธ | per-sample loop, hard-coded compile config (H1) | +| `feature_squeezing` | โš ๏ธ | biased quantiser, kills gradient, `bit_depth` unvalidated (H3) | +| `gradient_masking` | โŒ **no-op** | zero inference-time effect (C11) | +| `input_transformation` | โš ๏ธ | default is a silent identity (M8) | +| `defensive_distillation` | โš ๏ธ wrong | double softmax, no `Tยฒ`, teacher in train mode (H7) | +| `randomized_smoothing` | โŒ wrong | inference-only noise, no certification (H5) | +| `feature_denoising` | โš ๏ธ | an average pool, duplicate of blur | +| `thermometer_encoding` | โŒ wrong | plain quantisation (H6) | +| `adversarial_logit_pairing` | โŒ wrong | pairs on clean inputs only (H8) | +| `spatial_smoothing` | โš ๏ธ | actually a median filter; memory blow-up (M7) | +| `jpeg_compression` | โŒ | breaks gradient; grayscale unsupported (C10) | +| `pixel_deflection` | โŒ crash | `tf.random.uniform` misuse (C3) | +| `gaussian_blur` | โŒ wrong | `sigma` unused, is avg-pool (C13) | +| `total_variation_minimization` | โŒ wrong | zero gradient through, nested tape (C14) | +| `word_masking` | โš ๏ธ | whitespace loss, unseeded | +| `median_smoothing` | โŒ wrong | is avg-pool (C12) | + +--- + +## 5. Proposed remediation plan + +**Phase 0 โ€” guardrails first (so the fixes can be proven)** +1. `tests/` with a tiny deterministic CNN fixture; pytest. +2. `.github/workflows/ci.yml`: pytest on 3.10โ€“3.12 + `ruff check` on every push/PR. +3. `pyproject.toml`: real version `0.2.0`, bounded deps, `[project.optional-dependencies] dev`, ruff/pytest config. +4. `SECURITY.md`, `CODE_OF_CONDUCT.md`, `.pre-commit-config.yaml`, Dependabot. + +**Phase 1 โ€” attack correctness** +5. New `deepdefend/_utils.py`: `_prepare_labels` (one-hot **and** sparse **and** binary), `_loss_fn(model, x, y, from_logits)`, `_project(adv, x, eps, clip_min, clip_max)`, `_validate_*`, `resolve_input_range(model, x)`. +6. Rewrite `fgsm/pgd/bim/mim`: batched, `training=False`, `clip_min/clip_max`, `from_logits`, `random_start` (PGD), sparse labels, `None`-gradient guard. +7. Rewrite `cw` (tanh space, L2 objective, working `c`/`kappa`), `deepfool` (per-class boundaries + ฮต bound), `jsma` (real saliency, vectorised update), `spsa` (Variable + proper finite-difference estimator, rank-agnostic), `ead` (real elastic-net + non-degenerate defaults). +8. Text attacks: `re`-based tokenisation preserving whitespace, `seed` parameter, validated `swap_prob`. + +**Phase 2 โ€” defense correctness** +9. Hoist every layer to module level with `get_config()`/`compute_output_shape()`; add a shared `_wrap_preprocessing()`; raise clear errors for subclassed/multi-input models. **Add a saveโ†’load round-trip test for all wrappers.** +10. `adversarial_training`: batched attack generation, honour `model.optimizer`/`model.loss`, accept `epochs`/`batch_size`/`verbose`. +11. Real median filter for `median_smoothing`; real separable Gaussian for `gaussian_blur` (use `sigma`); de-duplicate `feature_denoising`; make `spatial_smoothing` honest about what it is and memory-safe. +12. Straight-through gradients for `jpeg_compression`, `feature_squeezing` and `total_variation_minimization` (replace the nested tape), so attacks against defended models work โ€” plus an explicit `differentiable=True/False` flag and a test that `grad is not None`. +13. `randomized_smoothing`: `training` gate, clip, seed, and a `predict_smoothed(n_samples)` + certified-accuracy helper; document that the base model must be noise-trained. +14. `thermometer_encoding`: real `C โ†’ Cร—L` expansion (with a documented retraining requirement) or rename to `uniform_quantization`. +15. `defensive_distillation` (logits + `Tยฒ` + KL + `training=False` teacher), `adversarial_logit_pairing` (generate adversarial examples inside the step), `gradient_masking` (keep, but relabel + docstring/README warning that it is not a security defense). +16. `pixel_deflection`: scalar-maxval fix, `fn_output_signature`, unique-index scatter, seed. +17. Validation pass: every numeric arg gets a documented range and a `ValueError` (kills the NaN paths in H2). +18. `input_transformation`: error or documented identity default instead of a silent no-op. + +**Phase 3 โ€” API consistency** +19. `__init__.py`: `__all__`, `__version__`, export everything, remove the self-import, fix the circular import, lazy TF import. +20. Uniform contract: attacks return `np.ndarray` in the input dtype; defenses return a `keras.Model`; text helpers return `str`. Add `seed` everywhere random. + +**Phase 4 โ€” docs, packaging, release** +21. Rewrite the README support matrix from measured behaviour; add per-defense "limitations / not a guarantee" notes and a "dual-use / responsible disclosure" section. +22. Fix `setup.py`/`pyproject.toml` metadata (version, bounded deps, `python_requires>=3.9`, license field, extras). +23. Rewrite the publish workflow: SHA-pinned actions, `permissions: contents: read` + `id-token: write`, `environment: pypi`, Trusted Publishing (no long-lived token), `$GITHUB_OUTPUT`, tag-pattern guard, tests before publish, `python -m build` + `twine check`. + +**Compatibility note:** items 7, 9โ€“17 change numerical output by design. Recommend releasing as **0.2.0** with a `CHANGELOG.md` "Behaviour changes" section rather than pretending to be a patch release. + +--- + +## Appendix โ€” measured evidence (TF 2.16.2 / Keras 3.15 / Python 3.11) + +``` +=== ATTACKS (x in [0,1]) === +[OK] fgsm shape=(4,8,8,3) min=-0.010 max=1.007 <-- outside [0,1] +[FAIL] jsma InvalidArgumentError: Index out of range using input dim 4 +[FAIL] spsa InvalidArgumentError: Tile ... (also fails with batch_size=1) +[OK] deepfool shape=(4,8,8,3) min=-0.900 max=2.080 <-- invalid images + +=== epsilon bound, eps=0.02 === + fgsm max|delta|=0.0200 in[0,1]=False + deepfool FAIL TypeError + ead max|delta|=0.0000 mean|delta|=0.0000 <-- no-op + +--- cw: do c / kappa change anything? --- + cw c/kappa have no effect: True + +--- sparse (integer) labels --- + fgsm / pgd / bim / cw / mim / ead / deepfool -> all FAIL: + "target and output must have the same rank (ndim)" + +--- gradient flow through defenses --- + jpeg_compression grad_present=False <-- attacks impossible + total_variation_minimization grad_present=True grad_nonzero=False + feature_squeezing grad_present=False + +--- serialization --- + feature_squeezing / gaussian_blur / randomized_smoothing -> save+load FAIL + "could not be deserialized properly" + +--- equivalence / dead-parameter checks --- + gaussian_blur(sigma=0.1) == gaussian_blur(sigma=9): True <-- sigma unused + median_smoothing == avg_pool2d: True <-- not a median + fgsm(base) == fgsm(gradient_masking(base)): True <-- no defense + feature_squeezing bit_depth=0 -> NaN: True + thermometer num_bins=0 -> NaN: True + pixel_deflection -> ValueError: maxval must be a scalar + +--- misc --- + fgsm(epsilon=-1) accepted, min=-0.9997 + adversarial_training(200 samples) = 3.08 s (~13 min extrapolated for MNIST) + model compiled with SGD(0.001)/KLDivergence -> returned with Adam/categorical_crossentropy + deepdefend.mim / .ead / .pixel_deflection / .gaussian_blur / .median_smoothing / .__version__ -> MISSING + pip install . -> "Encountered error while generating package metadata" + jsma on subclassed/multi-input models -> AttributeError / ValueError +``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 59c0a84..f2b36a0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,14 +7,28 @@ Thank you for considering contributing to **DeepDefend**! Contributions, whether 1. **Fork the Repository** Create a fork of this repository to your GitHub account and clone it locally. -2. **Set Up the Environment** - Ensure you have Python 3.6+ installed. You should also install compatible versions of the necessary dependencies for the project. +2. **Set Up the Environment** + Install Python 3.9+ and the development dependencies: -3. **Run Tests** - Before making changes, run tests to confirm that everything works. + ```bash + pip install -e ".[dev]" + ``` + +3. **Run Tests** + Before and after making changes, run the test-suite and the linter: + + ```bash + pytest # 171 tests covering attacks, defenses and the public API + ruff check . # lint + ``` + + Both run in CI on every pull request (Python 3.10 โ€“ 3.12). New behaviour + should come with a test; bug fixes should come with a test that fails + before the fix. > [!TIP] -> We sometimes include testing scripts in our projects for easy testing. +> The test-suite uses a tiny synthetic dataset and a small CNN, so the whole +> run takes well under a minute. ## How to Contribute @@ -32,6 +46,7 @@ Thank you for considering contributing to **DeepDefend**! Contributions, whether - Push your changes to a feature branch in your fork. - Submit a pull request with a detailed explanation of what you've changed or added. - Ensure your PR passes all automated tests and adheres to the contribution guidelines. + - If you change numerical behaviour, add an entry to `CHANGELOG.md`. ## Community Guidelines diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..4383c73 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,54 @@ +# Security Policy + +DeepDefend is a dual-use library: it implements both adversarial attacks and +defenses. Please read the "Responsible use" section below before filing a +report. + +## Supported versions + +| Version | Supported | +|---------|-----------| +| 0.2.x | โœ… | +| < 0.2 | โŒ (several functions were non-functional or silently wrong; see `CHANGELOG.md`) | + +## Reporting a vulnerability + +**Please do not open a public issue for security problems.** + +Report privately through GitHub's +[private vulnerability reporting](https://github.com/Infinitode/DeepDefend/security/advisories/new) +or by email to **infinitode.ltd@gmail.com**. + +Please include: + +- the affected version and Python / TensorFlow versions, +- a minimal reproduction (or the exact function and arguments), +- the impact you believe it has. + +We aim to acknowledge reports within 5 working days and to publish a fixed +release plus an advisory once a fix is available. Please give us a reasonable +window to ship a fix before any public disclosure. + +## Scope + +In scope: + +- code execution, denial of service, or data exfiltration caused by this library; +- functions that silently return incorrect results in a way that would lead a + user to overstate the robustness of their model (this is treated as a security + issue for a robustness library); +- unsafe deserialisation, path handling or network access in the library or in + its build/publish pipeline. + +Out of scope: + +- the *effectiveness* of a defense against an adaptive attacker. Preprocessing + defenses are not guarantees; see the limitations in `readme.md`; +- issues in TensorFlow, Keras or NumPy themselves (please report those upstream); +- the mere existence of attack implementations in this library. + +## Responsible use + +The attack implementations exist so that defenders can measure and improve the +robustness of models they are authorised to test. Do not use this library +against systems you do not own or have explicit permission to assess. diff --git a/deepdefend/__init__.py b/deepdefend/__init__.py index 5fe1e87..8dd28a7 100644 --- a/deepdefend/__init__.py +++ b/deepdefend/__init__.py @@ -1,3 +1,85 @@ -import deepdefend -from .attacks import fgsm, pgd, bim, cw, deepfool, jsma, spsa -from .defenses import adversarial_training, feature_squeezing, gradient_masking, input_transformation, defensive_distillation, randomized_smoothing, feature_denoising, thermometer_encoding, adversarial_logit_pairing, spatial_smoothing, jpeg_compression +""" +DeepDefend โ€” adversarial attacks and defenses for deep learning models. + +Submodules and their public functions are loaded lazily, so ``import +deepdefend`` does not pay the cost of importing TensorFlow:: + + import deepdefend + deepdefend.fgsm # imported on first access + from deepdefend.attacks import pgd # or import the submodule directly +""" + +from __future__ import annotations + +import importlib + +try: # pragma: no cover - depends on the installation method + from importlib.metadata import PackageNotFoundError, version + + try: + __version__ = version("deepdefend") + except PackageNotFoundError: + __version__ = "0.2.0" +except ImportError: # pragma: no cover - Python < 3.8 + __version__ = "0.2.0" + +_ATTACKS = frozenset( + { + "bim", + "char_swap", + "cw", + "deepfool", + "ead", + "fgsm", + "jsma", + "mim", + "pgd", + "spsa", + "word_swap", + } +) + +_DEFENSES = frozenset( + { + "adversarial_logit_pairing", + "adversarial_training", + "defensive_distillation", + "feature_denoising", + "feature_squeezing", + "gaussian_blur", + "gradient_masking", + "input_transformation", + "jpeg_compression", + "median_smoothing", + "pixel_deflection", + "randomized_smoothing", + "smooth_predict", + "spatial_smoothing", + "thermometer_encoding", + "total_variation_minimization", + "word_masking", + } +) + +_SUBMODULES = frozenset({"attacks", "defenses"}) + +__all__ = sorted(_ATTACKS | _DEFENSES | _SUBMODULES | {"__version__"}) + + +def __getattr__(name): + """Lazily import submodules and their public functions (PEP 562).""" + if name in _SUBMODULES: + module = importlib.import_module(f".{name}", __name__) + globals()[name] = module + return module + if name in _ATTACKS or name in _DEFENSES: + module_name = "attacks" if name in _ATTACKS else "defenses" + module = importlib.import_module(f".{module_name}", __name__) + attribute = getattr(module, name) + globals()[name] = attribute + return attribute + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") + + +def __dir__(): + return sorted(__all__) diff --git a/deepdefend/_utils.py b/deepdefend/_utils.py new file mode 100644 index 0000000..7a5702a --- /dev/null +++ b/deepdefend/_utils.py @@ -0,0 +1,389 @@ +"""Internal helpers shared by :mod:`deepdefend.attacks` and :mod:`deepdefend.defenses`. + +This module is private (leading underscore) and its API may change between +releases. It centralises the things that the public functions used to +re-implement (badly) in every function body: + +* argument validation, so a security tool never silently emits ``NaN``; +* label handling for one-hot, sparse-integer and binary targets; +* ``from_logits`` detection, so attacks work on models that do not end in a + ``softmax``/``sigmoid``; +* projection onto the ``[clip_min, clip_max]`` box and the ``epsilon`` ball; +* straight-through gradients for quantisation / compression based defenses. +""" + +from __future__ import annotations + +import math +import numbers +import re + +import numpy as np +import tensorflow as tf + +__all__ = [ + "AttackContext", + "as_numpy", + "build_layer_model", + "call_model", + "check_float", + "check_int", + "infer_from_logits", + "infer_num_classes", + "normalise_labels", + "prepare_attack_context", + "project", + "resolve_clip_range", + "straight_through", + "tokenize_text", + "to_tensor", + "wrap_preprocessing", +] + +# A "word" for the text helpers: letters, digits and inner apostrophes. +_WORD_RE = re.compile(r"[A-Za-z0-9']+") +_TOKEN_RE = re.compile(r"([A-Za-z0-9']+)") + +# --------------------------------------------------------------------------- +# Validation helpers +# --------------------------------------------------------------------------- + + +def check_float(name, value, minimum=None, maximum=None, allow_none=False): + """Validate a real-valued hyper-parameter and return it as ``float``.""" + if value is None: + if allow_none: + return None + raise ValueError(f"`{name}` must be a number, got None.") + if isinstance(value, bool) or not isinstance(value, numbers.Real): + raise TypeError(f"`{name}` must be a real number, got {type(value).__name__}.") + value = float(value) + if not math.isfinite(value): + raise ValueError(f"`{name}` must be finite, got {value}.") + if minimum is not None and value < minimum: + raise ValueError(f"`{name}` must be >= {minimum}, got {value}.") + if maximum is not None and value > maximum: + raise ValueError(f"`{name}` must be <= {maximum}, got {value}.") + return value + + +def check_int(name, value, minimum=None, maximum=None): + """Validate an integer hyper-parameter and return it as ``int``.""" + if isinstance(value, bool) or not isinstance(value, numbers.Integral): + raise TypeError(f"`{name}` must be an integer, got {type(value).__name__}.") + value = int(value) + if minimum is not None and value < minimum: + raise ValueError(f"`{name}` must be >= {minimum}, got {value}.") + if maximum is not None and value > maximum: + raise ValueError(f"`{name}` must be <= {maximum}, got {value}.") + return value + + +# --------------------------------------------------------------------------- +# Text helpers +# --------------------------------------------------------------------------- + + +def tokenize_text(text): + """Split ``text`` into ``(token, is_word)`` pairs. + + Whitespace runs, newlines and punctuation are returned as their own tokens, + so a caller can rebuild the text verbatim with ``"".join(tokens)``. The + naive ``text.split()`` / ``" ".join(...)`` round-trip used previously + collapsed double spaces and removed line breaks. + """ + parts = _TOKEN_RE.split(text) + return [(part, bool(_WORD_RE.fullmatch(part))) for part in parts if part] + + +# --------------------------------------------------------------------------- +# Tensor helpers +# --------------------------------------------------------------------------- + + +def to_tensor(x, dtype=tf.float32): + """Convert ``x`` to a float tensor without changing its values.""" + tensor = tf.convert_to_tensor(x) + if not tensor.dtype.is_floating: + tensor = tf.cast(tensor, dtype) + elif tensor.dtype != dtype: + tensor = tf.cast(tensor, dtype) + return tensor + + +def as_numpy(tensor): + """Return a NumPy view/copy of ``tensor`` (keeps the original dtype).""" + return np.asarray(tensor.numpy() if tf.is_tensor(tensor) else tensor) + + +def call_model(model, x): + """Call ``model`` in inference mode, tolerating callables without ``training``.""" + try: + return model(x, training=False) + except TypeError: + return model(x) + + +def straight_through(value, reference): + """Forward ``value``, but back-propagate the identity gradient of ``reference``. + + Used by non-differentiable defenses (quantisation, JPEG, denoisers) so that + gradient based attacks against a *defended* model still produce a useful + signal instead of ``None``. + """ + return reference + tf.stop_gradient(value - reference) + + +# --------------------------------------------------------------------------- +# Output / label introspection +# --------------------------------------------------------------------------- + + +def _model_output_shape(model): + shape = getattr(model, "output_shape", None) + if isinstance(shape, (list, tuple)) and shape and isinstance(shape[0], (list, tuple)): + if len(shape) != 1: + return None # multi-output models are not supported + shape = shape[0] + if shape is None or not isinstance(shape, (list, tuple)): + return None + return tuple(shape) + + +def infer_num_classes(model, y=None): + """Best-effort detection of the number of output units.""" + shape = _model_output_shape(model) + if shape is not None: + last = shape[-1] + if isinstance(last, numbers.Integral) and last > 0: + return int(last) + if y is not None: + # ``y`` may be a symbolic tensor inside a tf.function; only inspect it + # when it can be converted without executing the graph. + try: + array = np.asarray(y) + except (TypeError, ValueError, tf.errors.OperatorNotAllowedInGraphError): + array = None + if array is not None: + if array.ndim >= 2 and array.shape[-1] > 1: + return int(array.shape[-1]) + if array.ndim >= 2 and array.shape[-1] == 1: + return 1 + raise ValueError( + "Could not infer the number of classes; pass a model with a known " + "`output_shape` or one-hot encoded labels." + ) + + +def infer_from_logits(model): + """Guess whether ``model`` returns logits (``True``) or probabilities. + + Returns ``True`` for a final ``linear``/``relu`` layer and ``False`` for a + final ``softmax``/``sigmoid``. Unknown activation functions default to + ``False`` (probabilities), which matches the historical behaviour of the + package and is the safer assumption for the cross-entropy losses used here. + """ + layers = getattr(model, "layers", None) + if not layers: + return False + last = layers[-1] + activation = getattr(last, "activation", None) + name = getattr(activation, "__name__", None) + if name in ("softmax", "sigmoid"): + return False + if name == "linear": + return True + # A defense wrapper ends with the wrapped model itself: look inside it. + if getattr(last, "layers", None): + return infer_from_logits(last) + return False + + +def normalise_labels(y, num_classes): + """Convert labels to ``(float32 labels, string mode)``. + + ``mode`` is ``"binary"`` (shape ``(N, 1)``) or ``"categorical"`` + (one-hot, shape ``(N, num_classes)``). Accepts one-hot float arrays, + sparse integer arrays of shape ``(N,)`` or ``(N, 1)``, and float arrays of + integral class indices. + """ + tensor = tf.convert_to_tensor(y) + if tensor.dtype.is_integer: + if num_classes == 1: + return tf.reshape(tf.cast(tensor, tf.float32), (-1, 1)), "binary" + indices = tf.reshape(tf.cast(tensor, tf.int32), (-1,)) + return tf.one_hot(indices, num_classes, dtype=tf.float32), "categorical" + + tensor = tf.cast(tensor, tf.float32) + rank = tensor.shape.rank + if num_classes == 1: + return tf.reshape(tensor, (-1, 1)), "binary" + if rank == 1: + # Float-encoded class indices, e.g. np.array([0., 2., 1.]). + rounded = tf.round(tensor) + if tf.executing_eagerly() and not bool(tf.reduce_all(tf.abs(tensor - rounded) < 1e-6)): + raise ValueError( + "1-D float labels must be integral class indices, or pass " + "one-hot labels of shape (N, num_classes)." + ) + return tf.one_hot(tf.cast(rounded, tf.int32), num_classes, dtype=tf.float32), "categorical" + if rank >= 2 and tensor.shape[-1] == 1: + return tf.reshape(tensor, (-1, 1)), "binary" + return tensor, "categorical" + + +# --------------------------------------------------------------------------- +# Attack context +# --------------------------------------------------------------------------- + + +class AttackContext: + """Resolved attack inputs: tensors, labels, losses and clipping bounds.""" + + __slots__ = ("x", "labels", "y_idx", "num_classes", "mode", "from_logits", "clip_min", "clip_max") + + def __init__(self, x, labels, y_idx, num_classes, mode, from_logits, clip_min, clip_max): + self.x = x + self.labels = labels + self.y_idx = y_idx + self.num_classes = num_classes + self.mode = mode + self.from_logits = from_logits + self.clip_min = clip_min + self.clip_max = clip_max + + # -- losses --------------------------------------------------------- + def loss_vec(self, outputs): + """Per-example cross-entropy (shape ``(batch,)``).""" + if self.mode == "binary": + return tf.keras.losses.binary_crossentropy( + self.labels, outputs, from_logits=self.from_logits + ) + return tf.keras.losses.categorical_crossentropy( + self.labels, outputs, from_logits=self.from_logits + ) + + def loss(self, outputs): + """Mean cross-entropy over the batch (scalar).""" + return tf.reduce_mean(self.loss_vec(outputs)) + + # -- projection ----------------------------------------------------- + def project(self, adv, epsilon=None): + return project(adv, self.x, epsilon=epsilon, clip_min=self.clip_min, clip_max=self.clip_max) + + +def prepare_attack_context(model, x, y, from_logits=None, clip_min=None, clip_max=None): + """Validate inputs and build an :class:`AttackContext`.""" + x_t = to_tensor(x) + if x_t.shape.rank is None or x_t.shape.rank < 1: + raise ValueError("`x` must be a tensor with at least one dimension (batch included).") + + num_classes = infer_num_classes(model, y) + labels, mode = normalise_labels(y, num_classes) + + if labels.shape[0] is not None and x_t.shape[0] is not None and labels.shape[0] != x_t.shape[0]: + raise ValueError( + f"`x` and `y` have different batch sizes: {int(x_t.shape[0])} vs " + f"{int(labels.shape[0])}." + ) + + if from_logits is None: + from_logits = infer_from_logits(model) + from_logits = bool(from_logits) + + if mode == "binary": + y_idx = tf.cast(tf.reshape(labels, (-1,)) >= 0.5, tf.int32) + else: + y_idx = tf.cast(tf.argmax(labels, axis=-1), tf.int32) + + lo, hi = resolve_clip_range(x_t, clip_min, clip_max) + + return AttackContext( + x=x_t, + labels=labels, + y_idx=y_idx, + num_classes=num_classes, + mode=mode, + from_logits=from_logits, + clip_min=lo, + clip_max=hi, + ) + + +# --------------------------------------------------------------------------- +# Clipping / projection +# --------------------------------------------------------------------------- + + +def resolve_clip_range(x, clip_min=None, clip_max=None): + """Decide the valid data range. + + * both given -> use them; + * both ``None`` and the data already lies in ``[0, 1]`` -> ``(0.0, 1.0)``; + * both ``None`` and the data is outside ``[0, 1]`` (e.g. ImageNet-style + normalised inputs) -> no clipping, so the input is not silently + destroyed. Pass an explicit range if you want one. + """ + if (clip_min is None) != (clip_max is None): + raise ValueError("`clip_min` and `clip_max` must be provided together.") + if clip_min is None: + if not tf.executing_eagerly(): + # Inside a tf.function the data range cannot be inspected; assume + # image data in [0, 1] rather than guessing from a symbolic tensor. + return 0.0, 1.0 + lo = float(tf.reduce_min(x)) + hi = float(tf.reduce_max(x)) + if lo >= 0.0 and hi <= 1.0: + return 0.0, 1.0 + return None, None + lo = check_float("clip_min", clip_min) + hi = check_float("clip_max", clip_max) + if hi <= lo: + raise ValueError(f"`clip_max` ({hi}) must be greater than `clip_min` ({lo}).") + return lo, hi + + +def project(adv, x, epsilon=None, clip_min=None, clip_max=None): + """Project ``adv`` onto the ``epsilon`` L-infinity ball around ``x`` and the data range.""" + if epsilon is not None: + adv = tf.clip_by_value(adv, x - epsilon, x + epsilon) + if clip_min is not None and clip_max is not None: + adv = tf.clip_by_value(adv, clip_min, clip_max) + return adv + + +# --------------------------------------------------------------------------- +# Model wrapping used by the defense module +# --------------------------------------------------------------------------- + + +def _single_input_spec(model): + """Return ``(shape, dtype)`` for a single-input Keras model.""" + shape = getattr(model, "input_shape", None) + if isinstance(shape, (list, tuple)) and shape and isinstance(shape[0], (list, tuple)): + raise TypeError( + "DeepDefend defenses only support single-input models; this model " + f"has {len(shape)} inputs." + ) + if shape is None: + raise TypeError( + "DeepDefend defenses require a model with a known `input_shape` " + "(a Functional or Sequential model). Subclassed models must be " + "built from an explicit `tf.keras.Input` before they can be wrapped." + ) + dtype = getattr(model, "input_dtype", None) or "float32" + return tuple(shape[1:]), dtype + + +def build_layer_model(model, layer, name=None): + """Wrap ``model`` in a new Functional model that applies ``layer`` first.""" + shape, dtype = _single_input_spec(model) + inputs = tf.keras.Input(shape=shape, dtype=dtype, name="deepdefend_input") + outputs = call_model(model, layer(inputs)) + return tf.keras.Model(inputs=inputs, outputs=outputs, name=name) + + +def wrap_preprocessing(model, layer): + """Alias of :func:`build_layer_model` kept for readability in call sites.""" + layer_name = getattr(layer, "name", None) + return build_layer_model(model, layer, name=f"{model.name}_{layer_name}" if layer_name else None) diff --git a/deepdefend/attacks.py b/deepdefend/attacks.py index 44e9f2d..6aaec88 100644 --- a/deepdefend/attacks.py +++ b/deepdefend/attacks.py @@ -1,430 +1,845 @@ """ Functions to run adversarial attacks on deep learning models. +Every attack accepts: + +* ``x`` โ€“ a batch of inputs (NumPy array or TensorFlow tensor); +* ``y`` โ€“ labels as one-hot vectors ``(N, C)``, sparse integer indices ``(N,)`` + or binary targets ``(N, 1)``; +* ``from_logits`` โ€“ ``None`` (auto-detect from the last layer), ``True`` or + ``False``; +* ``clip_min`` / ``clip_max`` โ€“ the valid data range. When omitted, ``[0, 1]`` + is used if the data already lies in that range, otherwise no clipping is + applied (so normalised inputs are not silently destroyed). + +All attacks return a NumPy array with the same shape and dtype as ``x``. + +Iterative attacks stop working on a sample as soon as it reaches the goal +(misclassified for untargeted attacks, classified as ``y`` for targeted ones), +so a sample the model already gets wrong is returned unchanged. + Available functions: -- `fgsm(model, x, y, epsilon=0.01)`: Fast Gradient Sign Method (FGSM) attack. -- `pgd(model, x, y, epsilon=0.01, alpha=0.01, num_steps=10)`: Projected Gradient Descent (PGD) attack. -- `bim(model, x, y, epsilon=0.01, alpha=0.01, num_steps=10)`: Basic Iterative Method (BIM) attack. -- `cw(model, x, y, epsilon=0.01, c=1, kappa=0, num_steps=10, alpha=0.01)`: Carlini & Wagner (C&W) attack. -- `deepfool(model, x, y, num_steps=10)`: DeepFool attack. -- `jsma(model, x, y, theta=0.1, gamma=0.1, num_steps=10)`: Jacobian-based Saliency Map Attack (JSMA). -- `spsa(model, x, y, epsilon=0.01, num_steps=10)`: Simultaneous Perturbation Stochastic Approximation (SPSA) attack. -- `mim(model, x, y, epsilon=0.01, alpha=0.01, num_steps=10, decay_factor=1.0)`: Momentum Iterative Method (MIM) attack. -- `ead(model, x, y, epsilon=0.01, beta=0.01, num_steps=10, alpha=0.01)`: Elastic Net Attack (EAD). -- `word_swap(text, swap_dict=None)`: Simple word swap attack for text. -- `char_swap(text, swap_prob=0.1)`: Simple character swap attack for text. +- `fgsm(model, x, y, epsilon=0.01)`: Fast Gradient Sign Method. +- `pgd(model, x, y, epsilon=0.01, alpha=None, num_steps=10)`: Projected Gradient Descent. +- `bim(model, x, y, epsilon=0.01, alpha=None, num_steps=10)`: Basic Iterative Method. +- `mim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, decay_factor=1.0)`: Momentum Iterative Method. +- `cw(model, x, y, epsilon=0.01, c=1.0, kappa=0.0, num_steps=100, alpha=0.01)`: Carlini & Wagner L2. +- `deepfool(model, x, y, num_steps=50, epsilon=None)`: DeepFool. +- `jsma(model, x, y, theta=0.0, gamma=0.2, num_steps=10)`: Jacobian-based Saliency Map Attack. +- `spsa(model, x, y, epsilon=0.01, num_steps=50)`: Simultaneous Perturbation Stochastic Approximation. +- `ead(model, x, y, epsilon=0.01, beta=0.001, num_steps=50, alpha=0.01)`: Elastic Net Attack. +- `word_swap(text, swap_dict=None)`: word substitution attack for text. +- `char_swap(text, swap_prob=0.1)`: adjacent-character swap attack for text. """ +from __future__ import annotations + import numpy as np import tensorflow as tf -def fgsm(model, x, y, epsilon=0.01): +from ._utils import ( + as_numpy, + call_model, + check_float, + check_int, + prepare_attack_context, + project, + tokenize_text, +) + +__all__ = [ + "bim", + "char_swap", + "cw", + "deepfool", + "ead", + "fgsm", + "jsma", + "mim", + "pgd", + "spsa", + "word_swap", +] + + +# --------------------------------------------------------------------------- +# Shared internals +# --------------------------------------------------------------------------- + + +def _gradient(model, adv, ctx): + """Return ``(gradient, outputs)`` of the attack loss w.r.t. ``adv``.""" + with tf.GradientTape() as tape: + tape.watch(adv) + outputs = call_model(model, adv) + loss = ctx.loss(outputs) + grad = tape.gradient(loss, adv) + if grad is None: + raise RuntimeError( + "The gradient of the loss with respect to the input is None. " + "This usually means the model contains a non-differentiable or " + "disconnected preprocessing step." + ) + return grad, outputs + + +def _predicted(outputs, ctx): + """Predicted class indices for ``outputs`` (shape ``(N,)``).""" + return tf.cast(tf.argmax(_probabilities(outputs, ctx), axis=-1), tf.int32) + + +def _keep_mask(outputs, ctx, x, y_idx=None, targeted=False): + """Mask of samples that still need attacking, broadcastable to ``x``. + + For untargeted attacks a sample is done once it is misclassified; for + targeted attacks it is done once it is classified as the target. ``y_idx`` + must be supplied when ``outputs`` covers only a slice of the batch + (otherwise the full-batch labels would broadcast and duplicate rows). """ - Fast Gradient Sign Method (FGSM) attack. - - Parameters: - model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The magnitude of the perturbation (default: 0.01). + if y_idx is None: + y_idx = ctx.y_idx + predicted = _predicted(outputs, ctx) + if targeted: + still_needed = tf.not_equal(predicted, y_idx) + else: + still_needed = tf.equal(predicted, y_idx) + return tf.reshape(still_needed, [-1] + [1] * (x.shape.rank - 1)) - Returns: - adversarial_example (numpy.ndarray): The perturbed input example. - """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - # Determine the loss function based on the number of classes - if y.shape[-1] == 1 or len(y.shape) == 1: - loss_object = tf.keras.losses.BinaryCrossentropy() - else: - loss_object = tf.keras.losses.CategoricalCrossentropy() +def _probabilities(outputs, ctx): + """Convert model outputs to per-class probabilities.""" + if ctx.mode == "binary": + p = tf.reshape(outputs, (-1, 1)) + p = tf.sigmoid(p) if ctx.from_logits else p + return tf.concat([1.0 - p, p], axis=1) + return tf.nn.softmax(outputs) if ctx.from_logits else outputs - with tf.GradientTape() as tape: - tape.watch(x) - prediction = model(x) - loss = loss_object(y, prediction) - - gradient = tape.gradient(loss, x) - - # Generate adversarial example - perturbation = epsilon * tf.sign(gradient) - adversarial_example = x + perturbation - return adversarial_example.numpy() - -def mim(model, x, y, epsilon=0.01, alpha=0.01, num_steps=10, decay_factor=1.0): - """ - Momentum Iterative Method (MIM) attack. - Parameters: - model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The maximum magnitude of the perturbation (default: 0.01). - alpha (float): The step size for each iteration (default: 0.01). - num_steps (int): The number of MIM iterations (default: 10). - decay_factor (float): The decay factor for momentum (default: 1.0). +def _margin(outputs, ctx, kappa=0.0, target=None): + """C&W style margin ``max(max_{j != t} Z_j - Z_t, -kappa)`` per sample. - Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + ``t`` is passed in explicitly so it stays fixed for the whole optimisation โ€” + recomputing it every step makes the attack chase a moving target. """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - adversarial_example = tf.identity(x) - momentum = tf.zeros_like(x) - - # Determine the loss function based on the number of classes - if y.shape[-1] == 1 or len(y.shape) == 1: - loss_object = tf.keras.losses.BinaryCrossentropy() - else: - loss_object = tf.keras.losses.CategoricalCrossentropy() + if ctx.mode == "binary": + outputs = _probabilities(outputs, ctx) - for _ in range(num_steps): - with tf.GradientTape() as tape: - tape.watch(adversarial_example) - prediction = model(adversarial_example) - loss = loss_object(y, prediction) + if target is None: + target = tf.cast(tf.argmin(outputs, axis=-1), tf.int32) - gradient = tape.gradient(loss, adversarial_example) - # L1 normalize gradient - grad_l1 = tf.reduce_sum(tf.abs(gradient)) - gradient = gradient / (grad_l1 + 1e-8) + batch = tf.shape(outputs)[0] + rows = tf.range(batch) + z_t = tf.gather_nd(outputs, tf.stack([rows, target], axis=1)) - momentum = decay_factor * momentum + gradient + mask = tf.one_hot(target, tf.shape(outputs)[1], dtype=outputs.dtype) + very_negative = tf.fill(tf.shape(outputs), tf.cast(-1e9, outputs.dtype)) + others = tf.where(tf.cast(mask, tf.bool), very_negative, outputs) + z_other = tf.reduce_max(others, axis=-1) + return tf.maximum(z_other - z_t, -kappa) - perturbation = alpha * tf.sign(momentum) - adversarial_example = tf.clip_by_value(adversarial_example + perturbation, 0, 1) - adversarial_example = tf.clip_by_value(adversarial_example, x - epsilon, x + epsilon) - return adversarial_example.numpy() +def _resolve_target(model, ctx, targeted): + """Target class used by the margin based attacks. -def ead(model, x, y, epsilon=0.01, beta=0.01, num_steps=10, alpha=0.01): + For targeted attacks it is ``y``. For untargeted attacks it is the + runner-up class of the *original* input: the cheapest label flip, and it is + fixed for the whole optimisation (recomputing it every step makes the attack + chase a moving target). """ - Elastic Net Attack (EAD) attack. + if targeted: + return ctx.y_idx + outputs = call_model(model, ctx.x) + if ctx.mode == "binary": + outputs = _probabilities(outputs, ctx) + if outputs.shape[-1] is not None and int(outputs.shape[-1]) < 2: + return tf.cast(tf.argmin(outputs, axis=-1), tf.int32) + order = tf.argsort(outputs, axis=-1, direction="DESCENDING") + return tf.cast(order[:, 1], tf.int32) - Parameters: - model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The maximum magnitude of the perturbation (default: 0.01). - beta (float): The L1 regularization parameter (default: 0.01). - num_steps (int): The number of EAD iterations (default: 10). - alpha (float): The step size for each iteration (default: 0.01). - Returns: - adversarial_example (numpy.ndarray): The perturbed input example. +def _to_range(w, lo, hi): + """Map unconstrained ``w`` into ``[lo, hi]`` (C&W tanh parameterisation).""" + return lo + (hi - lo) * 0.5 * (tf.tanh(w) + 1.0) + + +def _from_range(adv, lo, hi): + """Inverse of :func:`_to_range`, numerically clamped away from the bounds.""" + scaled = (adv - lo) / (hi - lo) * 2.0 - 1.0 + scaled = tf.clip_by_value(scaled, -1.0 + 1e-6, 1.0 - 1e-6) + return tf.atanh(scaled) + + +def _project_against(adv, reference, ctx, epsilon=None): + """Project ``adv`` around ``reference`` (a slice of ``ctx.x``). + + ``ctx.project`` projects around the whole batch, which is wrong inside the + per-sample loops used by DeepFool, JSMA and SPSA. """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - adversarial_example = tf.identity(x) + return project(adv, reference, epsilon=epsilon, clip_min=ctx.clip_min, clip_max=ctx.clip_max) - # Determine the loss function based on the number of classes - if y.shape[-1] == 1 or len(y.shape) == 1: - loss_object = tf.keras.losses.BinaryCrossentropy() - else: - loss_object = tf.keras.losses.CategoricalCrossentropy() - for _ in range(num_steps): - with tf.GradientTape() as tape: - tape.watch(adversarial_example) - prediction = model(adversarial_example) - loss = loss_object(y, prediction) +def _adam_step(param, grad, m, v, t, learning_rate, beta1=0.9, beta2=0.999, eps=1e-8): + """One hand-rolled Adam update (avoids creating an optimizer per call).""" + m = beta1 * m + (1.0 - beta1) * grad + v = beta2 * v + (1.0 - beta2) * tf.square(grad) + m_hat = m / (1.0 - beta1 ** (t + 1)) + v_hat = v / (1.0 - beta2 ** (t + 1)) + return param - learning_rate * m_hat / (tf.sqrt(v_hat) + eps), m, v - gradient = tape.gradient(loss, adversarial_example) - perturbation = alpha * tf.sign(gradient) - new_x = adversarial_example + perturbation +def _prepare(x, model, y, from_logits, clip_min, clip_max): + ctx = prepare_attack_context(model, x, y, from_logits, clip_min, clip_max) + return ctx, ctx.x.dtype - # Proximal operator for L1 (soft thresholding) - diff = new_x - x - adversarial_example = x + tf.sign(diff) * tf.maximum(tf.abs(diff) - beta, 0) - adversarial_example = tf.clip_by_value(adversarial_example, 0, 1) - adversarial_example = tf.clip_by_value(adversarial_example, x - epsilon, x + epsilon) +# --------------------------------------------------------------------------- +# One-step and iterative gradient attacks +# --------------------------------------------------------------------------- - return adversarial_example.numpy() -def word_swap(text, swap_dict=None): +def fgsm(model, x, y, epsilon=0.01, *, targeted=False, from_logits=None, + clip_min=None, clip_max=None): """ - Simple word swap attack for text. + Fast Gradient Sign Method (FGSM) attack. Parameters: - text (str): The input text. - swap_dict (dict): Dictionary of words and their substitutes. + model (tensorflow.keras.Model): The target model to attack. + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): The magnitude of the perturbation (default: 0.01). + targeted (bool): Minimise the loss for ``y`` instead of maximising it. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range (inferred if omitted). Returns: - perturbed_text (str): The text with swapped words. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - if swap_dict is None: - return text + epsilon = check_float("epsilon", epsilon, minimum=0.0) + ctx, _ = _prepare(x, model, y, from_logits, clip_min, clip_max) + direction = -1.0 if targeted else 1.0 + grad, _ = _gradient(model, ctx.x, ctx) + adv = ctx.x + direction * epsilon * tf.sign(grad) + return as_numpy(ctx.project(adv)) + + +def _iterative_gradient_attack(model, x, y, epsilon, alpha, num_steps, *, + random_start, seed, targeted, from_logits, + clip_min, clip_max, momentum=None, decay_factor=1.0): + """Shared implementation of PGD / BIM / MIM.""" + epsilon = check_float("epsilon", epsilon, minimum=0.0) + num_steps = check_int("num_steps", num_steps, minimum=1) + if alpha is None: + alpha = epsilon / 4.0 if epsilon > 0 else 1e-3 + alpha = check_float("alpha", alpha, minimum=0.0) + decay_factor = check_float("decay_factor", decay_factor, minimum=0.0) + + ctx, dtype = _prepare(x, model, y, from_logits, clip_min, clip_max) + direction = -1.0 if targeted else 1.0 + + adv = ctx.x + if random_start and epsilon > 0: + noise = tf.random.uniform(tf.shape(ctx.x), -epsilon, epsilon, seed=seed, dtype=dtype) + adv = ctx.project(ctx.x + noise, epsilon) + + if momentum is not None: + momentum = tf.zeros_like(ctx.x) - words = text.split() - for i in range(len(words)): - if words[i] in swap_dict: - words[i] = swap_dict[words[i]] + for _ in range(num_steps): + grad, outputs = _gradient(model, adv, ctx) + if momentum is not None: + axes = list(range(1, ctx.x.shape.rank)) + grad = grad / (tf.reduce_mean(tf.abs(grad), axis=axes, keepdims=True) + 1e-12) + momentum = decay_factor * momentum + grad + step = tf.sign(momentum) + else: + step = tf.sign(grad) - return " ".join(words) + previous = adv + adv = previous + direction * alpha * step + adv = ctx.project(adv, epsilon) -def char_swap(text, swap_prob=0.1): - """ - Simple character swap attack for text. + # Freeze samples that have already reached the goal. + keep = _keep_mask(outputs, ctx, ctx.x, targeted=targeted) + adv = tf.where(keep, adv, previous) - Parameters: - text (str): The input text. - swap_prob (float): The probability of swapping a character in a word (default: 0.1). + return adv - Returns: - perturbed_text (str): The text with swapped characters. - """ - words = text.split() - for i in range(len(words)): - if len(words[i]) > 1 and np.random.rand() < swap_prob: - word_list = list(words[i]) - idx = np.random.randint(0, len(word_list) - 1) - word_list[idx], word_list[idx+1] = word_list[idx+1], word_list[idx] - words[i] = "".join(word_list) - return " ".join(words) - -def pgd(model, x, y, epsilon=0.01, alpha=0.01, num_steps=10): + +def pgd(model, x, y, epsilon=0.01, alpha=None, num_steps=10, *, random_start=True, + seed=None, targeted=False, from_logits=None, clip_min=None, clip_max=None): """ Projected Gradient Descent (PGD) attack. - + Parameters: model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The maximum magnitude of the perturbation (default: 0.01). - alpha (float): The step size for each iteration (default: 0.01). - num_steps (int): The number of PGD iterations (default: 10). + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): The L-infinity perturbation budget (default: 0.01). + alpha (float): Step size. Defaults to ``epsilon / 4``. + num_steps (int): Number of iterations (default: 10). + random_start (bool): Start from a random point in the epsilon ball. + seed (int): Optional random seed for the random start. + targeted (bool): Whether ``y`` is the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range (inferred if omitted). Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - adversarial_example = tf.identity(x) - - for _ in range(num_steps): - with tf.GradientTape() as tape: - tape.watch(adversarial_example) - prediction = model(adversarial_example) - loss = tf.keras.losses.CategoricalCrossentropy()(y, prediction) - - gradient = tape.gradient(loss, adversarial_example) - perturbation = alpha * tf.sign(gradient) - adversarial_example = tf.clip_by_value(adversarial_example + perturbation, 0, 1) - adversarial_example = tf.clip_by_value(adversarial_example, x - epsilon, x + epsilon) + return as_numpy(_iterative_gradient_attack( + model, x, y, epsilon, alpha, num_steps, random_start=random_start, + seed=seed, targeted=targeted, from_logits=from_logits, + clip_min=clip_min, clip_max=clip_max, + )) - return adversarial_example.numpy() -def bim(model, x, y, epsilon=0.01, alpha=0.01, num_steps=10): +def bim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, *, seed=None, + targeted=False, from_logits=None, clip_min=None, clip_max=None): """ - Basic Iterative Method (BIM) attack. - + Basic Iterative Method (BIM) attack โ€” PGD without a random start. + Parameters: model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The maximum magnitude of the perturbation (default: 0.01). - alpha (float): The step size for each iteration (default: 0.01). - num_steps (int): The number of BIM iterations (default: 10). + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): The L-infinity perturbation budget (default: 0.01). + alpha (float): Step size. Defaults to ``epsilon / 4``. + num_steps (int): Number of iterations (default: 10). + seed (int): Optional random seed (unused, kept for API symmetry). + targeted (bool): Whether ``y`` is the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range (inferred if omitted). Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - adversarial_example = tf.identity(x) + return as_numpy(_iterative_gradient_attack( + model, x, y, epsilon, alpha, num_steps, random_start=False, seed=seed, + targeted=targeted, from_logits=from_logits, clip_min=clip_min, + clip_max=clip_max, + )) - for _ in range(num_steps): - with tf.GradientTape() as tape: - tape.watch(adversarial_example) - prediction = model(adversarial_example) - loss = tf.keras.losses.CategoricalCrossentropy()(y, prediction) - - gradient = tape.gradient(loss, adversarial_example) - perturbation = alpha * tf.sign(gradient) - adversarial_example = tf.clip_by_value(adversarial_example + perturbation, 0, 1) - adversarial_example = tf.clip_by_value(adversarial_example, x - epsilon, x + epsilon) - - return adversarial_example.numpy() -def cw(model, x, y, epsilon=0.01, c=1, kappa=0, num_steps=10, alpha=0.01): +def mim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, decay_factor=1.0, *, + seed=None, targeted=False, from_logits=None, clip_min=None, clip_max=None): """ - Carlini & Wagner (C&W) attack. - + Momentum Iterative Method (MIM) attack. + Parameters: model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The maximum magnitude of the perturbation (default: 0.01). - c (float): The weight of the L2 norm of the perturbation (default: 1). - kappa (float): The confidence parameter (default: 0). - num_steps (int): The number of C&W iterations (default: 10). - alpha (float): The step size for each iteration (default: 0.01). + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): The L-infinity perturbation budget (default: 0.01). + alpha (float): Step size. Defaults to ``epsilon / 4``. + num_steps (int): Number of iterations (default: 10). + decay_factor (float): Momentum decay factor (default: 1.0). + seed (int): Optional random seed. + targeted (bool): Whether ``y`` is the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range (inferred if omitted). Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) + return as_numpy(_iterative_gradient_attack( + model, x, y, epsilon, alpha, num_steps, random_start=False, seed=seed, + targeted=targeted, from_logits=from_logits, clip_min=clip_min, + clip_max=clip_max, momentum=True, decay_factor=decay_factor, + )) - # Define the loss function - def loss_function(x, y, model, c, kappa): - prediction = model(x) - loss = tf.keras.losses.CategoricalCrossentropy()(y, prediction) - return loss + c * tf.norm(x - tf.clip_by_value(x, 0, 1)) ** 2 - kappa - # Initialize the adversarial example - adversarial_example = tf.identity(x) +# --------------------------------------------------------------------------- +# Optimisation based attacks +# --------------------------------------------------------------------------- - # Perform the C&W attack - for _ in range(num_steps): - with tf.GradientTape() as tape: - tape.watch(adversarial_example) - loss = loss_function(adversarial_example, y, model, c, kappa) - gradient = tape.gradient(loss, adversarial_example) - perturbation = alpha * tf.sign(gradient) - adversarial_example = tf.clip_by_value(adversarial_example + perturbation, 0, 1) - adversarial_example = tf.clip_by_value(adversarial_example, x - epsilon, x + epsilon) +def cw(model, x, y, epsilon=None, c=10.0, kappa=0.0, num_steps=100, alpha=0.01, *, + targeted=False, from_logits=None, clip_min=None, clip_max=None): + """ + Carlini & Wagner L2 attack. - return adversarial_example.numpy() + The perturbation is optimised in the tanh space and the objective is the + true C&W objective ``||delta||_2^2 + c * max(max_{j != t} Z_j - Z_t, -kappa)``. + Unlike the previous implementation, ``c`` and ``kappa`` genuinely change the + result. -def deepfool(model, x, y, num_steps=10): - """ - DeepFool attack. - Parameters: model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - num_steps (int): The number of DeepFool iterations (default: 10). + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): Optional L2 budget, expressed as an L-infinity + equivalent: the L2 radius is ``epsilon * sqrt(num_features)``. + Defaults to ``None`` โ€” the classic, unconstrained C&W attack that + simply minimises the distortion needed to change the label. + c (float): Weight of the misclassification term (default: 10.0). The + C&W objective trades distortion against the margin, so ``c`` is the + knob that controls how far the attack is willing to move: small + values stall at the L2/margin equilibrium, large values find a + (larger) successful perturbation. Sweep it to minimise distortion. + kappa (float): Confidence margin (default: 0.0). + num_steps (int): Number of Adam iterations (default: 100). + alpha (float): Adam learning rate (default: 0.01). + targeted (bool): Whether ``y`` is the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range. Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - # Initialize the adversarial example - adversarial_example = tf.identity(x) + epsilon = check_float("epsilon", epsilon, minimum=0.0, allow_none=True) + c = check_float("c", c, minimum=0.0) + kappa = check_float("kappa", kappa, minimum=0.0) + num_steps = check_int("num_steps", num_steps, minimum=1) + alpha = check_float("alpha", alpha, minimum=0.0) + if c == 0: + raise ValueError("`c` must be > 0 for the C&W objective to be meaningful.") + + ctx, dtype = _prepare(x, model, y, from_logits, clip_min, clip_max) + lo, hi = ctx.clip_min, ctx.clip_max + if lo is None: + # The tanh parameterisation needs a bounded range. + lo, hi = 0.0, 1.0 + + num_features = float(np.prod([d for d in ctx.x.shape[1:]])) + radius = None if epsilon is None else epsilon * np.sqrt(num_features) + + target = _resolve_target(model, ctx, targeted) + + w = _from_range(ctx.x, lo, hi) + m = tf.zeros_like(w) + v = tf.zeros_like(w) + + best_adv = tf.identity(ctx.x) + best_dist = tf.fill([tf.shape(ctx.x)[0]], tf.cast(np.inf, dtype)) + found = tf.zeros([tf.shape(ctx.x)[0]], dtype=tf.bool) + axes = list(range(1, ctx.x.shape.rank)) + + for step in range(num_steps): + with tf.GradientTape() as tape: + tape.watch(w) + adv = _to_range(w, lo, hi) + outputs = call_model(model, adv) + margin = _margin(outputs, ctx, kappa=kappa, target=target) + dist = tf.reduce_sum(tf.square(adv - ctx.x), axis=axes) + objective = tf.reduce_sum(dist + c * margin) + grad = tape.gradient(objective, w) + if grad is None: + raise RuntimeError("Gradient is None; the C&W attack cannot proceed.") + + # Stop updating samples that have already crossed the margin + # (``margin <= 0`` means the target class now wins). + done = tf.reshape(margin <= 0.0, [-1] + [1] * (ctx.x.shape.rank - 1)) + grad = tf.where(done, tf.zeros_like(grad), grad) + + w, m, v = _adam_step(w, grad, m, v, step, alpha) + + adv = _to_range(w, lo, hi) + if radius is not None: + delta = adv - ctx.x + norm = tf.sqrt(tf.reduce_sum(tf.square(delta), axis=axes, keepdims=True)) + scale = tf.minimum(1.0, radius / (norm + 1e-12)) + adv = ctx.x + delta * scale + adv = tf.clip_by_value(adv, lo, hi) + w = _from_range(adv, lo, hi) + + # Track the smallest successful perturbation per sample. + success = margin <= 0.0 + better = tf.logical_and(success, dist < best_dist) + mask = tf.reshape(better, [-1] + [1] * (ctx.x.shape.rank - 1)) + best_adv = tf.where(mask, adv, best_adv) + best_dist = tf.where(better, dist, best_dist) + found = tf.logical_or(found, success) + + # For samples where no adversarial example was found, return the last + # iterate (a perturbation that did not succeed) rather than silently + # pretending the input is already minimal. + mask = tf.reshape(found, [-1] + [1] * (ctx.x.shape.rank - 1)) + return as_numpy(tf.where(mask, best_adv, adv)) + + +def ead(model, x, y, epsilon=0.01, beta=0.001, num_steps=50, alpha=0.01, *, c=0.1, + kappa=0.0, targeted=False, from_logits=None, clip_min=None, clip_max=None): + """ + Elastic Net Attack (EAD) โ€” iterative shrinkage (ISTA) on the elastic-net + objective ``||delta||_2^2 + beta * ||delta||_1 + c * margin``. - # Perform the DeepFool attack + Parameters: + model (tensorflow.keras.Model): The target model to attack. + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): L-infinity perturbation budget (default: 0.01). + beta (float): L1 regularisation weight (default: 0.001). Must be < 1; + values that are large relative to ``alpha`` suppress the attack. + num_steps (int): Number of iterations (default: 50). + alpha (float): Step size (default: 0.01). + c (float): Weight of the misclassification term (default: 0.1). + kappa (float): Confidence margin (default: 0.0). + targeted (bool): Whether ``y`` is the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range. + + Returns: + adversarial_example (numpy.ndarray): The perturbed input example(s). + """ + epsilon = check_float("epsilon", epsilon, minimum=0.0) + beta = check_float("beta", beta, minimum=0.0, maximum=1.0) + num_steps = check_int("num_steps", num_steps, minimum=1) + alpha = check_float("alpha", alpha, minimum=0.0) + c = check_float("c", c, minimum=0.0) + kappa = check_float("kappa", kappa, minimum=0.0) + if alpha == 0: + raise ValueError("`alpha` must be > 0.") + + ctx, _ = _prepare(x, model, y, from_logits, clip_min, clip_max) + axes = list(range(1, ctx.x.shape.rank)) + threshold = alpha * beta + + target = _resolve_target(model, ctx, targeted) + + adv = tf.identity(ctx.x) for _ in range(num_steps): with tf.GradientTape() as tape: - tape.watch(adversarial_example) - prediction = model(adversarial_example) - loss = tf.keras.losses.CategoricalCrossentropy()(y, prediction) + tape.watch(adv) + outputs = call_model(model, adv) + margin = _margin(outputs, ctx, kappa=kappa, target=target) + dist = tf.reduce_sum(tf.square(adv - ctx.x), axis=axes) + objective = tf.reduce_sum(dist + c * margin) + grad = tape.gradient(objective, adv) + if grad is None: + raise RuntimeError("Gradient is None; the EAD attack cannot proceed.") + + previous = adv + adv = adv - alpha * grad - gradient = tape.gradient(loss, adversarial_example) - perturbation = gradient / tf.norm(gradient) - adversarial_example = adversarial_example + perturbation + delta = adv - ctx.x + delta = tf.sign(delta) * tf.maximum(tf.abs(delta) - threshold, 0.0) + adv = ctx.x + delta - return adversarial_example.numpy() + adv = ctx.project(adv, epsilon) + keep = _keep_mask(outputs, ctx, ctx.x, targeted=targeted) + adv = tf.where(keep, adv, previous) -def jsma(model, x, y, theta=0.1, gamma=0.1, num_steps=10): + return as_numpy(adv) + + +def deepfool(model, x, y, num_steps=50, epsilon=None, overshoot=0.02, *, + from_logits=None, clip_min=None, clip_max=None): """ - Jacobian-based Saliency Map Attack (JSMA) attack. + DeepFool attack (Moosavi-Dezfooli et al., 2016). + + Computes, for each sample, the closest decision boundary of the current + predicted class and steps towards it with an overshoot, repeating until the + label flips or ``num_steps`` is reached. Parameters: model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - theta (float): The threshold for selecting pixels (default: 0.1). - gamma (float): The step size for each iteration (default: 0.1). - num_steps (int): The number of JSMA iterations (default: 10). + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (used to stop once misclassified). + num_steps (int): Maximum number of iterations per sample (default: 50). + epsilon (float): Optional L-infinity budget. ``None`` (default) lets the + attack move to the closest boundary without a budget. + overshoot (float): Overshoot factor applied to each step (default: 0.02). + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range. Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - x = tf.cast(x, tf.float32) - y = tf.cast(y, tf.float32) - # Initialize the adversarial example - adversarial_example = tf.identity(x) - - # Get the input shape - input_shape = x.shape - - # Perform the JSMA attack - for _ in range(num_steps): - # Calculate the Jacobian matrix - with tf.GradientTape() as tape: - tape.watch(adversarial_example) - prediction = model(adversarial_example) - loss = tf.keras.losses.CategoricalCrossentropy()(y, prediction) - - jacobian = tape.jacobian(loss, adversarial_example) - - # Calculate the saliency map - saliency_map = np.zeros(input_shape) - for i in range(input_shape[1]): - for j in range(input_shape[2]): - for k in range(input_shape[3]): - saliency_map[0, i, j, k] = np.sum(jacobian[0, i, j, k, :]) - - # Select the pixels to perturb - perturbed_pixels = np.where(saliency_map > theta) - - # Perturb the selected pixels - for i in range(len(perturbed_pixels[0])): - if adversarial_example[0, perturbed_pixels[0][i], perturbed_pixels[1][i], perturbed_pixels[2][i]] < 1: - adversarial_example[0, perturbed_pixels[0][i], perturbed_pixels[1][i], perturbed_pixels[2][i]] += gamma - else: - adversarial_example[0, perturbed_pixels[0][i], perturbed_pixels[1][i], perturbed_pixels[2][i]] -= gamma + num_steps = check_int("num_steps", num_steps, minimum=1) + epsilon = check_float("epsilon", epsilon, minimum=0.0, allow_none=True) + overshoot = check_float("overshoot", overshoot, minimum=0.0) + + ctx, _ = _prepare(x, model, y, from_logits, clip_min, clip_max) + batch = int(ctx.x.shape[0]) + result = [] + + for index in range(batch): + original = ctx.x[index:index + 1] + adv = tf.identity(original) + true_label = int(ctx.y_idx[index]) + + for _ in range(num_steps): + with tf.GradientTape() as tape: + tape.watch(adv) + outputs = _probabilities(call_model(model, adv), ctx) + current = int(tf.argmax(outputs[0])) + if current != true_label: + break + if outputs.shape[1] < 2: + break + + jacobian = tape.jacobian(outputs, adv) # (1, C, *shape) + probs = outputs[0] + w_cur = jacobian[0, current] + differences = probs - probs[current] + w_diff = jacobian[0] - w_cur + + flat = tf.reshape(w_diff, [tf.shape(w_diff)[0], -1]) + norm_sq = tf.reduce_sum(tf.square(flat), axis=1) + ratio = tf.abs(differences) / (norm_sq + 1e-12) + ratio = tf.tensor_scatter_nd_update( + ratio, [[current]], [tf.cast(np.inf, ratio.dtype)] + ) + best = int(tf.argmin(ratio)) + r = ratio[best] * w_diff[best] + adv = adv + (1.0 + overshoot) * r + + adv = _project_against(adv, original, ctx, epsilon) + + result.append(adv) + + return as_numpy(tf.concat(result, axis=0)) + + +def jsma(model, x, y, theta=0.0, gamma=0.2, num_steps=10, *, targeted=False, + from_logits=None, clip_min=None, clip_max=None): + """ + Jacobian-based Saliency Map Attack (JSMA), Papernot et al. (2016). - return adversarial_example.numpy() + At each iteration the saliency map + ``S_i = dF_t/dx_i * |sum_{j != t} dF_j/dx_i|`` (zero where the derivative + directions are unfavourable) is computed and the single most salient feature + is increased by ``gamma``. + Parameters: + model (tensorflow.keras.Model): The target model to attack. + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (the target class when ``targeted=True``). + theta (float): Minimum saliency required to perturb a feature. The + default of ``0.0`` always selects the most salient feature; raw + saliency values are products of two gradients and therefore tiny. + gamma (float): Perturbation added to the selected feature. + num_steps (int): Number of features perturbed per sample (default: 10). + targeted (bool): Treat ``y`` as the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range. -def spsa(model, x, y, epsilon=0.01, num_steps=10, learning_rate=0.01, delta=0.01, spsa_samples=128): + Returns: + adversarial_example (numpy.ndarray): The perturbed input example(s). + """ + theta = check_float("theta", theta, minimum=0.0) + gamma = check_float("gamma", gamma, minimum=0.0) + num_steps = check_int("num_steps", num_steps, minimum=1) + if gamma == 0: + raise ValueError("`gamma` must be > 0.") + + ctx, _ = _prepare(x, model, y, from_logits, clip_min, clip_max) + lo = ctx.clip_min if ctx.clip_min is not None else 0.0 + hi = ctx.clip_max if ctx.clip_max is not None else 1.0 + + batch = int(ctx.x.shape[0]) + result = [] + + for index in range(batch): + adv = tf.identity(ctx.x[index:index + 1]) + true_label = int(ctx.y_idx[index]) + target = true_label + + for _ in range(num_steps): + with tf.GradientTape(persistent=True) as tape: + tape.watch(adv) + probs = _probabilities(call_model(model, adv), ctx) + if not targeted: + target = int(tf.argmin(probs[0])) + target_prob = probs[0, target] + other_prob = tf.reduce_sum(probs[0]) - target_prob + + grad_target = tape.gradient(target_prob, adv) + grad_other = tape.gradient(other_prob, adv) + del tape + + if grad_target is None or grad_other is None: + break + + favourable = tf.logical_and(grad_target > 0.0, grad_other <= 0.0) + saliency = tf.where(favourable, grad_target * tf.abs(grad_other), 0.0) + flat = tf.reshape(saliency, [-1]) + best = int(tf.argmax(flat)) + if float(flat[best]) <= theta: + break + + step = tf.one_hot(best, tf.size(flat), dtype=adv.dtype) * gamma + adv = adv + tf.reshape(step, tf.shape(adv)) + adv = tf.clip_by_value(adv, lo, hi) + + predicted = int(tf.argmax(_probabilities(call_model(model, adv), ctx)[0])) + if predicted != true_label: + break + + result.append(adv) + + return as_numpy(tf.concat(result, axis=0)) + + +def spsa(model, x, y, epsilon=0.01, num_steps=50, learning_rate=0.05, delta=None, + spsa_samples=32, *, seed=None, targeted=False, from_logits=None, + clip_min=None, clip_max=None): """ Simultaneous Perturbation Stochastic Approximation (SPSA) attack. + A gradient-free attack: the gradient is estimated from loss differences + ``(L(x + delta * d) - L(x - delta * d)) / (2 * delta)`` along random + Rademacher directions ``d``, then the perturbation is optimised with Adam. + Parameters: model (tensorflow.keras.Model): The target model to attack. - x (numpy.ndarray): The input example to attack. - y (numpy.ndarray): The true labels of the input example. - epsilon (float): The magnitude of the perturbation (default: 0.01). - num_steps (int): The number of SPSA iterations (default: 10). - learning_rate (float): The learning rate for the ADAM optimizer (default: 0.01). - delta (float): The perturbation size for SPSA (default: 0.01). - spsa_samples (int): The number of samples for SPSA (default: 128). + x (numpy.ndarray): The input example(s) to attack. + y (numpy.ndarray): True labels (one-hot, sparse indices or binary). + epsilon (float): The L-infinity perturbation budget (default: 0.01). + num_steps (int): Number of Adam iterations (default: 50). + learning_rate (float): Adam learning rate (default: 0.05). + delta (float): Finite-difference probe size. Defaults to ``epsilon / 5``. + Very small probes can fall below float32 loss resolution. + spsa_samples (int): Number of random directions per gradient estimate. + seed (int): Optional random seed. + targeted (bool): Whether ``y`` is the target class. + from_logits (bool): Whether ``model`` returns logits. Auto-detected if None. + clip_min, clip_max (float): Valid data range. Returns: - adversarial_example (numpy.ndarray): The perturbed input example. + adversarial_example (numpy.ndarray): The perturbed input example(s). """ - if x.shape[0] != 1: - raise ValueError("For SPSA, input tensor x must have batch_size of 1.") + epsilon = check_float("epsilon", epsilon, minimum=0.0) + num_steps = check_int("num_steps", num_steps, minimum=1) + learning_rate = check_float("learning_rate", learning_rate, minimum=0.0) + spsa_samples = check_int("spsa_samples", spsa_samples, minimum=2) + if delta is None: + # The probe must be large enough for the loss difference to survive + # float32 quantisation on a confident model. + delta = max(epsilon / 5.0, 1e-4) + delta = check_float("delta", delta, minimum=0.0) + if delta == 0: + raise ValueError("`delta` must be > 0.") + + ctx, dtype = _prepare(x, model, y, from_logits, clip_min, clip_max) + direction_sign = -1.0 if targeted else 1.0 + batch = int(ctx.x.shape[0]) + result = [] + + for index in range(batch): + original = ctx.x[index:index + 1] + labels = ctx.labels[index:index + 1] + perturbation = tf.zeros_like(original) + m = tf.zeros_like(original) + v = tf.zeros_like(original) + + for step in range(num_steps): + probe_shape = tf.concat([[spsa_samples], tf.shape(original)[1:]], axis=0) + directions = tf.sign( + tf.random.uniform(probe_shape, -1.0, 1.0, seed=seed, dtype=dtype) + ) + + plus = call_model(model, original + delta * directions) + minus = call_model(model, original - delta * directions) + tiled_labels = tf.tile(labels, [spsa_samples, 1]) + if ctx.mode == "binary": + loss_plus = tf.keras.losses.binary_crossentropy(tiled_labels, plus, from_logits=ctx.from_logits) + loss_minus = tf.keras.losses.binary_crossentropy(tiled_labels, minus, from_logits=ctx.from_logits) + else: + loss_plus = tf.keras.losses.categorical_crossentropy(tiled_labels, plus, from_logits=ctx.from_logits) + loss_minus = tf.keras.losses.categorical_crossentropy(tiled_labels, minus, from_logits=ctx.from_logits) - tf_dtype = tf.as_dtype("float32") - optimizer = tf.optimizers.Adam(learning_rate=learning_rate) + differences = tf.reshape(loss_plus - loss_minus, [spsa_samples] + [1] * (original.shape.rank - 1)) + estimate = tf.reduce_mean(differences * directions, axis=0, keepdims=True) / (2.0 * delta) - def loss_fn(data, label): - logits = model(data) - label = tf.cast(label, tf.int64) - loss = tf.keras.losses.sparse_categorical_crossentropy(label, logits) - return loss + grad = -direction_sign * estimate + perturbation, m, v = _adam_step(perturbation, grad, m, v, step, learning_rate) - perturbation = tf.zeros_like(x, dtype=tf_dtype) + adv = _project_against( + original + tf.clip_by_value(perturbation, -epsilon, epsilon), + original, ctx, + ) + outputs = call_model(model, adv) + keep = _keep_mask( + outputs, ctx, original, + y_idx=ctx.y_idx[index:index + 1], targeted=targeted, + ) + adv = tf.where(keep, adv, original) + perturbation = adv - original - for _ in range(num_steps): - # SPSA gradient approximation - x_shape = x.shape - delta_x = delta * tf.sign(tf.random.uniform([spsa_samples // 2] + list(x_shape[1:]), minval=-1.0, maxval=1.0, dtype=tf_dtype)) - delta_x = tf.concat([delta_x, -delta_x], axis=0) + result.append(adv) + + return as_numpy(tf.concat(result, axis=0)) - y_tiled = tf.tile(y, [spsa_samples]) - loss_vals = loss_fn(tf.tile(x, [spsa_samples, 1, 1, 1]) + delta_x, y_tiled) - loss_vals = tf.reshape(loss_vals, [spsa_samples] + [1] * (len(x_shape) - 1)) +# --------------------------------------------------------------------------- +# Text attacks +# --------------------------------------------------------------------------- - grad = tf.reduce_mean(loss_vals * delta_x, axis=0, keepdims=True) / delta +def _rng(seed): + if isinstance(seed, np.random.Generator): + return seed + return np.random.default_rng(seed) - # ADAM update - optimizer.apply_gradients([(grad[0], perturbation)]) - # Projection - adversarial_example = x + tf.clip_by_value(perturbation, -epsilon, epsilon) - adversarial_example = tf.clip_by_value(adversarial_example, 0, 1) - perturbation.assign(adversarial_example - x) +def word_swap(text, swap_dict=None, max_swaps=None, seed=None): + """ + Word substitution attack for text. + + Whitespace, punctuation and line breaks are preserved exactly. + + Parameters: + text (str): The input text. + swap_dict (dict): Mapping of words to their substitutes. Required. + max_swaps (int): Optional cap on the number of substitutions. + seed (int): Optional random seed (substitutions are applied in a random + order when ``max_swaps`` is set). - return adversarial_example.numpy() + Returns: + perturbed_text (str): The text with swapped words. + """ + if not isinstance(text, str): + raise TypeError(f"`text` must be a string, got {type(text).__name__}.") + if swap_dict is None: + raise ValueError("`swap_dict` is required; refusing to silently return the input.") + if not isinstance(swap_dict, dict): + raise TypeError("`swap_dict` must be a dict.") + if max_swaps is not None: + max_swaps = check_int("max_swaps", max_swaps, minimum=0) + + tokens = tokenize_text(text) + candidates = [ + i for i, (token, is_word) in enumerate(tokens) + if is_word and token in swap_dict + ] + if max_swaps is not None and len(candidates) > max_swaps: + rng = _rng(seed) + chosen = rng.choice(candidates, size=max_swaps, replace=False) + candidates = [int(i) for i in chosen] + + for i in candidates: + tokens[i] = (swap_dict[tokens[i][0]], False) + return "".join(token for token, _ in tokens) + + +def char_swap(text, swap_prob=0.1, seed=None): + """ + Adjacent-character swap attack for text. + + Only characters inside words are swapped, so whitespace, newlines and + punctuation placement are preserved. + + Parameters: + text (str): The input text. + swap_prob (float): Probability of swapping a character in a word (default: 0.1). + seed (int): Optional random seed for reproducibility. + + Returns: + perturbed_text (str): The text with swapped characters. + """ + if not isinstance(text, str): + raise TypeError(f"`text` must be a string, got {type(text).__name__}.") + swap_prob = check_float("swap_prob", swap_prob, minimum=0.0, maximum=1.0) + rng = _rng(seed) + + tokens = tokenize_text(text) + for i, (token, is_word) in enumerate(tokens): + if not is_word or len(token) < 2: + continue + if rng.random() >= swap_prob: + continue + chars = list(token) + j = int(rng.integers(0, len(chars) - 1)) + chars[j], chars[j + 1] = chars[j + 1], chars[j] + tokens[i] = ("".join(chars), True) + return "".join(token for token, _ in tokens) diff --git a/deepdefend/defenses.py b/deepdefend/defenses.py index d0ce933..41c9504 100644 --- a/deepdefend/defenses.py +++ b/deepdefend/defenses.py @@ -2,564 +2,1201 @@ """ Functions to apply adversarial defense mechanisms to deep learning models. +Each model-level defense returns a new ``tf.keras.Model`` that applies a +preprocessing step before the original model. The preprocessing layers are +module level, serialisable and registered with Keras, so defended models can be +saved and reloaded with ``model.save()`` / ``tf.keras.models.load_model()``. + +Preprocessing defenses are *not* a guarantee of robustness. The docstrings of +each function state what the technique does and does not buy you; see also the +"Limitations" section of the README. + Available functions: -- `adversarial_training(model, x, y, epsilon=0.01)`: Adversarial Training defense. -- `feature_squeezing(model, bit_depth=4)`: Feature Squeezing defense. -- `gradient_masking(model, mask_threshold=0.1)`: Gradient Masking defense. -- `input_transformation(model, transformation_function=None)`: Input Transformation defense. -- `defensive_distillation(model, teacher_model, temperature=2)`: Defensive Distillation defense. -- `randomized_smoothing(model, noise_level=0.1)`: Randomized Smoothing defense. -- `feature_denoising(model)`: Feature Denoising defense. -- `thermometer_encoding(model, num_bins=10)`: Thermometer Encoding defense. -- `adversarial_logit_pairing(model, paired_model)`: Adversarial Logit Pairing defense. -- `spatial_smoothing(model, kernel_size=3)`: Spatial Smoothing defense. -- `jpeg_compression(model, quality=75)`: JPEG Compression defense. -- `pixel_deflection(model, deflection_count=100, window_size=10)`: Pixel Deflection defense. -- `gaussian_blur(model, kernel_size=3, sigma=1.0)`: Gaussian Blur defense. -- `total_variation_minimization(model, iterations=10, regularization_parameter=0.1)`: Total Variation Minimization defense. -- `word_masking(text, mask_token="[MASK]", mask_prob=0.1)`: Simple word masking defense for text. -- `median_smoothing(model, kernel_size=3)`: Median Smoothing defense. +- `adversarial_training(model, x, y, epsilon=0.01)`: Adversarial Training. +- `feature_squeezing(model, bit_depth=4)`: Feature Squeezing. +- `gradient_masking(model, mask_threshold=0.1)`: Gradient Masking (training regulariser, *not* a defense). +- `input_transformation(model, transformation_function)`: Input Transformation. +- `defensive_distillation(model, teacher_model, temperature=2)`: Defensive Distillation. +- `randomized_smoothing(model, noise_level=0.1)`: Randomized Smoothing (inference-time noise). +- `smooth_predict(model, x, n_samples=100)`: Monte-Carlo prediction + certified radius. +- `feature_denoising(model, method="median")`: Feature/input Denoising. +- `thermometer_encoding(model, num_bins=10)`: shape-preserving quantisation. +- `adversarial_logit_pairing(model, paired_model)`: Adversarial Logit Pairing. +- `spatial_smoothing(model, kernel_size=3)`: Spatial Smoothing (median filter). +- `jpeg_compression(model, quality=75)`: JPEG Compression. +- `pixel_deflection(model, deflection_count=100, window_size=10)`: Pixel Deflection. +- `gaussian_blur(model, kernel_size=3, sigma=1.0)`: Gaussian Blur. +- `total_variation_minimization(model, iterations=10)`: Total Variation Minimization. +- `word_masking(text, mask_token="[MASK]", mask_prob=0.1)`: Word masking for text. +- `median_smoothing(model, kernel_size=3)`: Median Smoothing (alias of `spatial_smoothing`). """ +from __future__ import annotations + +import math +import warnings + import numpy as np import tensorflow as tf -from deepdefend import attacks -def adversarial_training(model, x, y, epsilon=0.01): +from . import attacks +from ._utils import ( + check_float, + check_int, + infer_from_logits, + straight_through, + tokenize_text, + wrap_preprocessing, +) + +__all__ = [ + "adversarial_logit_pairing", + "adversarial_training", + "defensive_distillation", + "feature_denoising", + "feature_squeezing", + "gaussian_blur", + "gradient_masking", + "input_transformation", + "jpeg_compression", + "median_smoothing", + "pixel_deflection", + "randomized_smoothing", + "smooth_predict", + "spatial_smoothing", + "thermometer_encoding", + "total_variation_minimization", + "word_masking", +] + +_DEFAULT_RANGE = (0.0, 1.0) + + +# --------------------------------------------------------------------------- +# Serialisable preprocessing layers +# --------------------------------------------------------------------------- + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class SqueezeLayer(tf.keras.layers.Layer): + """Uniform quantisation of the input ("feature squeezing").""" + + def __init__(self, bit_depth=4, straight_through_gradient=True, value_range=_DEFAULT_RANGE, **kwargs): + super().__init__(**kwargs) + self.bit_depth = check_int("bit_depth", bit_depth, minimum=1, maximum=8) + self.straight_through_gradient = bool(straight_through_gradient) + self.value_range = tuple(float(v) for v in value_range) + + def call(self, inputs): + lo, hi = self.value_range + clipped = tf.clip_by_value(inputs, lo, hi) + levels = float(2 ** self.bit_depth - 1) + squeezed = tf.round(clipped * levels) / levels + if self.straight_through_gradient: + return straight_through(squeezed, inputs) + return tf.stop_gradient(squeezed) + + def get_config(self): + config = super().get_config() + config.update( + bit_depth=self.bit_depth, + straight_through_gradient=self.straight_through_gradient, + value_range=self.value_range, + ) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class QuantizationLayer(tf.keras.layers.Layer): + """Uniform quantisation into ``num_bins`` levels (shape preserving).""" + + def __init__(self, num_bins=10, straight_through_gradient=True, value_range=_DEFAULT_RANGE, **kwargs): + super().__init__(**kwargs) + self.num_bins = check_int("num_bins", num_bins, minimum=2) + self.straight_through_gradient = bool(straight_through_gradient) + self.value_range = tuple(float(v) for v in value_range) + + def call(self, inputs): + lo, hi = self.value_range + clipped = tf.clip_by_value(inputs, lo, hi) + bins = float(self.num_bins - 1) + quantised = tf.round((clipped - lo) / (hi - lo) * bins) / bins * (hi - lo) + lo + if self.straight_through_gradient: + return straight_through(quantised, inputs) + return tf.stop_gradient(quantised) + + def get_config(self): + config = super().get_config() + config.update( + num_bins=self.num_bins, + straight_through_gradient=self.straight_through_gradient, + value_range=self.value_range, + ) + return config + + +def _gaussian_kernel_1d(kernel_size, sigma, dtype=tf.float32): + radius = kernel_size // 2 + coords = tf.cast(tf.range(-radius, radius + 1), dtype) + kernel = tf.exp(-tf.square(coords) / (2.0 * sigma * sigma)) + return kernel / tf.reduce_sum(kernel) + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class GaussianBlurLayer(tf.keras.layers.Layer): + """Separable Gaussian blur (per channel) applied to 4-D image tensors.""" + + def __init__(self, kernel_size=3, sigma=1.0, **kwargs): + super().__init__(**kwargs) + kernel_size = check_int("kernel_size", kernel_size, minimum=1) + if kernel_size % 2 == 0: + raise ValueError(f"`kernel_size` must be odd, got {kernel_size}.") + self.kernel_size = kernel_size + self.sigma = check_float("sigma", sigma, minimum=1e-6) + + def call(self, inputs): + if inputs.shape.rank != 4: + raise ValueError( + "GaussianBlurLayer requires 4-D image inputs (batch, height, width, channels); " + f"received shape {inputs.shape}." + ) + kernel = _gaussian_kernel_1d(self.kernel_size, self.sigma, inputs.dtype) + kernel = tf.tensordot(kernel, kernel, axes=0) + kernel = tf.reshape(kernel, (self.kernel_size, self.kernel_size, 1, 1)) + kernel = tf.tile(kernel, [1, 1, tf.shape(inputs)[-1], 1]) + return tf.nn.depthwise_conv2d(inputs, kernel, strides=[1, 1, 1, 1], padding="SAME") + + def get_config(self): + config = super().get_config() + config.update(kernel_size=self.kernel_size, sigma=self.sigma) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class MedianFilterLayer(tf.keras.layers.Layer): + """True median filter over a ``kernel_size x kernel_size`` window.""" + + def __init__(self, kernel_size=3, **kwargs): + super().__init__(**kwargs) + kernel_size = check_int("kernel_size", kernel_size, minimum=3) + if kernel_size % 2 == 0: + raise ValueError(f"`kernel_size` must be odd, got {kernel_size}.") + self.kernel_size = kernel_size + + def call(self, inputs): + if inputs.shape.rank != 4: + raise ValueError( + "MedianFilterLayer requires 4-D image inputs (batch, height, width, channels); " + f"received shape {inputs.shape}." + ) + k = self.kernel_size + patches = tf.image.extract_patches( + images=inputs, + sizes=[1, k, k, 1], + strides=[1, 1, 1, 1], + rates=[1, 1, 1, 1], + padding="SAME", + ) + shape = tf.shape(inputs) + reshaped = tf.reshape(patches, tf.stack([shape[0], shape[1], shape[2], k * k, shape[3]])) + ordered = tf.sort(reshaped, axis=3) + return ordered[:, :, :, (k * k) // 2, :] + + def get_config(self): + config = super().get_config() + config.update(kernel_size=self.kernel_size) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class JPEGCompressionLayer(tf.keras.layers.Layer): + """JPEG encode/decode round-trip with a straight-through gradient.""" + + def __init__(self, quality=75, straight_through_gradient=True, value_range=_DEFAULT_RANGE, **kwargs): + super().__init__(**kwargs) + self.quality = check_int("quality", quality, minimum=1, maximum=100) + self.straight_through_gradient = bool(straight_through_gradient) + self.value_range = tuple(float(v) for v in value_range) + + def _roundtrip(self, image): + lo, hi = self.value_range + image = tf.clip_by_value(image, lo, hi) + data = tf.cast(tf.round((image - lo) / (hi - lo) * 255.0), tf.uint8) + channels = data.shape[-1] + fmt = "grayscale" if channels == 1 else "rgb" + encoded = tf.io.encode_jpeg(data, quality=self.quality, format=fmt) + decoded = tf.io.decode_jpeg(encoded, channels=channels) + return tf.cast(decoded, tf.float32) / 255.0 * (hi - lo) + lo + + def call(self, inputs): + if inputs.shape.rank != 4: + raise ValueError( + "JPEGCompressionLayer requires 4-D image inputs (batch, height, width, channels); " + f"received shape {inputs.shape}." + ) + channels = inputs.shape[-1] + if channels not in (1, 3): + raise ValueError( + "JPEG compression requires 1 (grayscale) or 3 (RGB) channels, " + f"got {channels}." + ) + compressed = tf.map_fn(self._roundtrip, inputs, fn_output_signature=inputs.dtype) + if self.straight_through_gradient: + return straight_through(compressed, inputs) + return tf.stop_gradient(compressed) + + def get_config(self): + config = super().get_config() + config.update( + quality=self.quality, + straight_through_gradient=self.straight_through_gradient, + value_range=self.value_range, + ) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class PixelDeflectionLayer(tf.keras.layers.Layer): + """Pixel deflection: random pixels are replaced by random nearby pixels.""" + + def __init__(self, deflection_count=100, window_size=10, seed=None, **kwargs): + super().__init__(**kwargs) + self.deflection_count = check_int("deflection_count", deflection_count, minimum=0) + self.window_size = check_int("window_size", window_size, minimum=1) + self.seed = seed + + def _uniform(self, shape, minval, maxval, tag): + """Random ints; stateless (and therefore reproducible) when seeded.""" + if self.seed is None: + return tf.random.uniform(shape, minval, maxval, dtype=tf.int32) + return tf.random.stateless_uniform( + shape, seed=(self.seed, tag), minval=minval, maxval=maxval, + dtype=tf.int32, + ) + + def _deflect(self, image): + shape = tf.shape(image) + height, width = shape[0], shape[1] + count = self.deflection_count + rows = self._uniform([count], 0, height, 0) + cols = self._uniform([count], 0, width, 1) + row_shift = self._uniform([count], -self.window_size, self.window_size + 1, 2) + col_shift = self._uniform([count], -self.window_size, self.window_size + 1, 3) + source = tf.stack( + [tf.clip_by_value(rows + row_shift, 0, height - 1), + tf.clip_by_value(cols + col_shift, 0, width - 1)], + axis=1, + ) + destination = tf.stack([rows, cols], axis=1) + values = tf.gather_nd(image, source) + return tf.tensor_scatter_nd_update(image, destination, values) + + def call(self, inputs): + if inputs.shape.rank != 4: + raise ValueError( + "PixelDeflectionLayer requires 4-D image inputs (batch, height, width, channels); " + f"received shape {inputs.shape}." + ) + return tf.map_fn(self._deflect, inputs, fn_output_signature=inputs.dtype) + + def get_config(self): + config = super().get_config() + config.update( + deflection_count=self.deflection_count, + window_size=self.window_size, + seed=self.seed, + ) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class TotalVariationLayer(tf.keras.layers.Layer): + """Gradient-descent total-variation denoising with a straight-through gradient.""" + + def __init__(self, iterations=10, regularization_parameter=0.1, + straight_through_gradient=True, value_range=_DEFAULT_RANGE, **kwargs): + super().__init__(**kwargs) + self.iterations = check_int("iterations", iterations, minimum=1) + self.regularization_parameter = check_float( + "regularization_parameter", regularization_parameter, minimum=0.0 + ) + self.straight_through_gradient = bool(straight_through_gradient) + self.value_range = tuple(float(v) for v in value_range) + + def call(self, inputs): + if inputs.shape.rank != 4: + raise ValueError( + "TotalVariationLayer requires 4-D image inputs (batch, height, width, channels); " + f"received shape {inputs.shape}." + ) + lo, hi = self.value_range + image = tf.stop_gradient(inputs) + for _ in range(self.iterations): + with tf.GradientTape() as tape: + tape.watch(image) + tv = tf.reduce_sum(tf.image.total_variation(image)) + grad = tape.gradient(tv, image) + if grad is None: + break + image = tf.clip_by_value(image - self.regularization_parameter * grad, lo, hi) + image = tf.stop_gradient(image) + if self.straight_through_gradient: + return straight_through(image, inputs) + return image + + def get_config(self): + config = super().get_config() + config.update( + iterations=self.iterations, + regularization_parameter=self.regularization_parameter, + straight_through_gradient=self.straight_through_gradient, + value_range=self.value_range, + ) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class NoiseLayer(tf.keras.layers.Layer): + """Additive Gaussian noise (optionally disabled at inference time).""" + + def __init__(self, noise_level=0.1, seed=None, add_noise_at_inference=True, + value_range=_DEFAULT_RANGE, **kwargs): + super().__init__(**kwargs) + self.noise_level = check_float("noise_level", noise_level, minimum=0.0) + self.seed = seed + self.add_noise_at_inference = bool(add_noise_at_inference) + self.value_range = tuple(float(v) for v in value_range) + + def call(self, inputs, training=None): + if training is False and not self.add_noise_at_inference: + return inputs + if self.seed is None: + noise = tf.random.normal( + tf.shape(inputs), mean=0.0, stddev=self.noise_level, + dtype=inputs.dtype, + ) + else: + # Stateless noise keeps a seeded layer reproducible across calls + # and across a save/load round-trip. + noise = tf.random.stateless_normal( + tf.shape(inputs), seed=(self.seed, 0), mean=0.0, + stddev=self.noise_level, dtype=inputs.dtype, + ) + lo, hi = self.value_range + return tf.clip_by_value(inputs + noise, lo, hi) + + def get_config(self): + config = super().get_config() + config.update( + noise_level=self.noise_level, + seed=self.seed, + add_noise_at_inference=self.add_noise_at_inference, + value_range=self.value_range, + ) + return config + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class InputTransformationLayer(tf.keras.layers.Layer): + """Applies a user supplied callable or Keras layer to the input.""" + + def __init__(self, transformation=None, **kwargs): + super().__init__(**kwargs) + if transformation is None or not callable(transformation): + raise ValueError("`transformation` must be a callable or a Keras layer.") + self.transformation = transformation + + def call(self, inputs): + return self.transformation(inputs) + + def get_config(self): + config = super().get_config() + if isinstance(self.transformation, tf.keras.layers.Layer): + # Keras layers serialise properly, so the wrapped model stays + # saveable. + config["transformation"] = { + "kind": "layer", + "config": tf.keras.utils.serialize_keras_object(self.transformation), + } + else: + config["transformation"] = { + "kind": "callable", + "config": getattr(self.transformation, "__name__", None), + } + return config + + @classmethod + def from_config(cls, config): + payload = config.pop("transformation", None) or {} + kind = payload.get("kind") + if kind == "layer": + transformation = tf.keras.utils.deserialize_keras_object(payload["config"]) + else: + name = payload.get("config") + transformation = ( + tf.keras.utils.get_registered_object(name) if name else None + ) + if transformation is None: + raise ValueError( + f"Could not resolve the transformation {name!r}. Plain " + "Python callables cannot be serialised; pass a Keras layer " + "(for example `tf.keras.layers.Rescaling(0.5)`) or register " + "the function with " + "`tf.keras.utils.register_keras_serializable` and pass it " + "through `custom_objects` when loading." + ) + return cls(transformation=transformation, **config) + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class FeatureDenoisingBlock(tf.keras.layers.Layer): + """Learnable residual smoothing block, meant to be inserted into a network. + + This is the practical, lighter-weight variant of the Feature Denoising + block (Xie et al., 2019): a learned local average is added to the feature + map through a zero-initialised scalar, so the block starts as an identity + and can be trained to denoise intermediate features. + """ + + def __init__(self, kernel_size=3, **kwargs): + super().__init__(**kwargs) + self.kernel_size = check_int("kernel_size", kernel_size, minimum=1) + + def build(self, input_shape): + # A fixed box filter: every tap is 1 / (k * k), so the block starts as + # a local average and the learnable scale decides how much to subtract. + self.average = tf.keras.layers.DepthwiseConv2D( + self.kernel_size, padding="same", use_bias=False, trainable=False, + depthwise_initializer=tf.keras.initializers.Constant( + 1.0 / (self.kernel_size * self.kernel_size) + ), + ) + self.scale = self.add_weight( + name="scale", shape=(), initializer="zeros", trainable=True + ) + super().build(input_shape) + + def call(self, inputs): + return inputs + self.scale * self.average(inputs) + + def get_config(self): + config = super().get_config() + config.update(kernel_size=self.kernel_size) + return config + + +# --------------------------------------------------------------------------- +# Training-time defenses +# --------------------------------------------------------------------------- + + +def _infer_loss(y): + """Pick a sensible Keras loss string for the given labels.""" + y = np.asarray(y) + if not np.issubdtype(y.dtype, np.floating): + return "sparse_categorical_crossentropy" + if y.ndim == 1: + return "sparse_categorical_crossentropy" + if y.ndim == 2 and y.shape[-1] == 1: + return "binary_crossentropy" + return "categorical_crossentropy" + + +def _clone_for_training(model): + try: + clone = tf.keras.models.clone_model(model) + except Exception as exc: # pragma: no cover - depends on the user model + raise TypeError( + "Adversarial training needs a Functional or Sequential model so it " + "can be cloned. Subclassed models are not supported." + ) from exc + clone.set_weights(model.get_weights()) + return clone + + +def adversarial_training(model, x, y, epsilon=0.01, epochs=10, batch_size=32, + attack=None, optimizer=None, loss=None, verbose=1, + shuffle=True): """ Adversarial Training defense. - Adversarial training is a method where the model is trained on both the original - and adversarial examples, aiming to make the model more robust to adversarial attacks. + Augments the training set with adversarial examples generated in batches and + fine-tunes a clone of ``model``. The caller's optimiser, loss and metrics + are preserved when the model was already compiled. Parameters: model (tensorflow.keras.Model): The model to defend. x (numpy.ndarray): The input training examples. y (numpy.ndarray): The true labels of the training examples. - epsilon (float): The magnitude of the perturbation (default: 0.01). + epsilon (float): The attack strength used to generate the examples. + epochs (int): Training epochs (default: 10). + batch_size (int): Batch size for both attack generation and training. + attack (callable): ``fn(model, x, y, epsilon=...)`` returning adversarial + examples. Defaults to :func:`deepdefend.attacks.fgsm`. + optimizer: Optional Keras optimiser (defaults to the model's optimiser). + loss: Optional Keras loss (defaults to the model's loss, then inferred + from the label encoding). + verbose (int): Keras ``fit`` verbosity (default: 1). + shuffle (bool): Shuffle the augmented training set. Returns: defended_model (tensorflow.keras.Model): The adversarially trained model. """ - defended_model = tf.keras.models.clone_model(model) - defended_model.set_weights(model.get_weights()) + epsilon = check_float("epsilon", epsilon, minimum=0.0) + epochs = check_int("epochs", epochs, minimum=1) + batch_size = check_int("batch_size", batch_size, minimum=1) + + x = np.asarray(x) + y = np.asarray(y) + if attack is None: + attack = attacks.fgsm + if not callable(attack): + raise TypeError("`attack` must be callable.") + + defended_model = _clone_for_training(model) + + adversarial_batches = [] + for start in range(0, len(x), batch_size): + chunk_x = x[start:start + batch_size] + chunk_y = y[start:start + batch_size] + adversarial_batches.append(attack(model, chunk_x, chunk_y, epsilon=epsilon)) + x_adversarial = np.concatenate(adversarial_batches, axis=0) - adversarial_examples = [] - for i in range(len(x)): - adversarial_example = attacks.fgsm(model, x[i:i+1], y[i:i+1], epsilon) - adversarial_examples.append(adversarial_example) - - x_adversarial = np.concatenate(adversarial_examples, axis=0) - y_adversarial = np.copy(y) x_train = np.concatenate([x, x_adversarial], axis=0) - y_train = np.concatenate([y, y_adversarial], axis=0) - - defended_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) - defended_model.fit(x_train, y_train, epochs=10, batch_size=32) - + y_train = np.concatenate([y, np.copy(y)], axis=0) + + if optimizer is None: + optimizer = getattr(model, "optimizer", None) + if loss is None: + loss = getattr(model, "loss", None) + if loss is None: + loss = _infer_loss(y) + if optimizer is None: + optimizer = "adam" + + defended_model.compile(optimizer=optimizer, loss=loss, metrics=["accuracy"]) + defended_model.fit( + x_train, y_train, epochs=epochs, batch_size=batch_size, + verbose=verbose, shuffle=shuffle, + ) return defended_model -def feature_squeezing(model, bit_depth=4): - """ - Feature Squeezing defense. - - Feature squeezing reduces the number of bits used to represent the input features, - which can remove certain adversarial perturbations. - Parameters: - model (tensorflow.keras.Model): The model to defend. - bit_depth (int): The number of bits per feature (default: 4). +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class GradientMaskingModel(tf.keras.Model): + """Training wrapper that drops gradients below ``threshold``. - Returns: - defended_model (tensorflow.keras.Model): The model with feature squeezing defense. + .. warning:: + This is a *training regulariser*, not an adversarial defense. It does + not alter the gradients an attacker computes at inference time. """ - class SqueezeLayer(tf.keras.layers.Layer): - def __init__(self, bit_depth): - super().__init__() - self.bit_depth = bit_depth - def call(self, x): - x_int = tf.cast(x * 255.0, tf.int32) - # Use tf.bitwise.right_shift instead of >> for symbolic tensors if needed, - # or just use division for simplicity. - squeezed_x = tf.cast(tf.math.floordiv(x_int, 2**(8 - self.bit_depth)), tf.float32) / (2**self.bit_depth - 1) - return squeezed_x + def __init__(self, base_model=None, mask_threshold=0.1, **kwargs): + super().__init__(**kwargs) + self.base_model = base_model + self.mask_threshold = mask_threshold + + def call(self, inputs, training=None): + return self.base_model(inputs, training=training) + + def train_step(self, data): + x, y = data[0], data[1] + with tf.GradientTape() as tape: + predictions = self.base_model(x, training=True) + loss = self.compiled_loss(y, predictions) + + trainable = self.base_model.trainable_variables + gradients = tape.gradient(loss, trainable) + pairs = [ + (tf.where(tf.abs(g) > self.mask_threshold, g, tf.zeros_like(g)), v) + for g, v in zip(gradients, trainable) if g is not None + ] + self.optimizer.apply_gradients(pairs) + self.compiled_metrics.update_state(y, predictions) + return {m.name: m.result() for m in self.metrics} + + def get_config(self): + config = super().get_config() + config.update( + base_model=tf.keras.utils.serialize_keras_object(self.base_model), + mask_threshold=self.mask_threshold, + ) + return config + + @classmethod + def from_config(cls, config): + config = dict(config) + config["base_model"] = tf.keras.utils.deserialize_keras_object(config["base_model"]) + return cls(**config) - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = SqueezeLayer(bit_depth)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) def gradient_masking(model, mask_threshold=0.1): """ - Gradient Masking defense. + Gradient Masking "defense". + + Zeroes training gradients smaller than ``mask_threshold``. - Gradient masking modifies the gradients during training to make them less informative - for adversarial attackers. + .. warning:: + Gradient masking is a well documented **false** defense (Athalye et al., + "Obfuscated Gradients Give a False Sense of Security", 2018). It has no + effect on the gradients an attacker sees, and it can make a model look + robust while remaining vulnerable. Use it only as a training + regulariser, never as evidence of robustness. Parameters: - model (tensorflow.keras.Model): The model to defend. - mask_threshold (float): The threshold for masking gradients (default: 0.1). + model (tensorflow.keras.Model): The model to wrap. + mask_threshold (float): Threshold below which gradients are zeroed. Returns: - defended_model (tensorflow.keras.Model): The model with gradient masking defense. + defended_model (tensorflow.keras.Model): The wrapped model. """ - class GradientMaskingModel(tf.keras.Model): - def __init__(self, base_model, threshold): - super().__init__() - self.base_model = base_model - self.threshold = threshold + mask_threshold = check_float("mask_threshold", mask_threshold, minimum=0.0) + warnings.warn( + "gradient_masking() does not defend against inference-time attacks; " + "it only modifies training gradients. Do not treat it as a robustness " + "measure.", + RuntimeWarning, + stacklevel=2, + ) + return GradientMaskingModel(base_model=model, mask_threshold=mask_threshold) + + +def _temperature_softmax(outputs, temperature, from_logits): + """Temperature-scaled softmax that works for logits and probabilities.""" + if from_logits: + return tf.nn.softmax(outputs / temperature) + probabilities = tf.clip_by_value(outputs, 1e-7, 1.0) + return tf.nn.softmax(tf.math.log(probabilities) / temperature) + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class DistillationModel(tf.keras.Model): + """Knowledge distillation from a (fixed) teacher into a student model.""" + + def __init__(self, student=None, teacher=None, temperature=2.0, + from_logits=None, **kwargs): + super().__init__(**kwargs) + self.student = student + self.teacher = teacher + self.temperature = temperature + self.from_logits = infer_from_logits(student) if from_logits is None else bool(from_logits) + + def call(self, inputs, training=None): + return self.student(inputs, training=training) + + def train_step(self, data): + x, y = data[0], data[1] + teacher_outputs = self.teacher(x, training=False) + soft_targets = _temperature_softmax(teacher_outputs, self.temperature, self.from_logits) + + with tf.GradientTape() as tape: + student_outputs = self.student(x, training=True) + soft_student = _temperature_softmax(student_outputs, self.temperature, self.from_logits) + distillation_loss = tf.reduce_mean( + tf.keras.losses.categorical_crossentropy(soft_targets, soft_student) + ) * (self.temperature ** 2) + task_loss = self.compiled_loss(y, student_outputs) + loss = task_loss + distillation_loss + + trainable = self.student.trainable_variables + gradients = tape.gradient(loss, trainable) + pairs = [(g, v) for g, v in zip(gradients, trainable) if g is not None] + self.optimizer.apply_gradients(pairs) + self.compiled_metrics.update_state(y, student_outputs) + return {m.name: m.result() for m in self.metrics} + + def get_config(self): + config = super().get_config() + config.update( + student=tf.keras.utils.serialize_keras_object(self.student), + teacher=tf.keras.utils.serialize_keras_object(self.teacher), + temperature=self.temperature, + from_logits=self.from_logits, + ) + return config + + @classmethod + def from_config(cls, config): + config = dict(config) + for key in ("student", "teacher"): + config[key] = tf.keras.utils.deserialize_keras_object(config[key]) + return cls(**config) - def train_step(self, data): - x, y = data - with tf.GradientTape() as tape: - y_pred = self.base_model(x, training=True) - loss = self.compiled_loss(y, y_pred) - trainable_vars = self.base_model.trainable_variables - gradients = tape.gradient(loss, trainable_vars) - masked_gradients = [tf.where(tf.abs(g) > self.threshold, g, tf.zeros_like(g)) for g in gradients] - self.optimizer.apply_gradients(zip(masked_gradients, trainable_vars)) - self.compiled_metrics.update_state(y, y_pred) - return {m.name: m.result() for m in self.metrics} +def defensive_distillation(model, teacher_model, temperature=2): + """ + Defensive Distillation defense. - def call(self, x): - return self.base_model(x) + Trains the student to match the temperature-softened predictions of a + teacher, in addition to the ordinary task loss. The distillation term is + scaled by ``temperature ** 2`` and the teacher is evaluated in inference + mode, so its batch-norm statistics and dropout do not corrupt the targets. - return GradientMaskingModel(model, mask_threshold) + Parameters: + model (tensorflow.keras.Model): The student model to defend. + teacher_model (tensorflow.keras.Model): The teacher model. + temperature (float): Distillation temperature (default: 2). -def input_transformation(model, transformation_function=None): + Returns: + defended_model (tensorflow.keras.Model): The distilled student model. """ - Input Transformation defense. + temperature = check_float("temperature", temperature, minimum=1e-6) + return DistillationModel(student=model, teacher=teacher_model, temperature=temperature) + + +@tf.keras.utils.register_keras_serializable(package="deepdefend") +class AdversarialLogitPairingModel(tf.keras.Model): + """Adversarial Logit Pairing: clean and adversarial logits must agree.""" + + def __init__(self, base_model=None, paired_model=None, epsilon=0.01, + num_steps=1, alpha=None, from_logits=None, **kwargs): + super().__init__(**kwargs) + self.base_model = base_model + self.paired_model = paired_model if paired_model is not None else base_model + self.epsilon = epsilon + self.num_steps = num_steps + self.alpha = alpha + self.from_logits = infer_from_logits(base_model) if from_logits is None else bool(from_logits) + + def call(self, inputs, training=None): + return self.base_model(inputs, training=training) + + def _adversarial(self, x, y): + return attacks._iterative_gradient_attack( + self.base_model, x, y, self.epsilon, self.alpha, self.num_steps, + random_start=False, seed=None, targeted=False, + from_logits=self.from_logits, clip_min=0.0, clip_max=1.0, + ) + + def train_step(self, data): + x, y = data[0], data[1] + adversarial = self._adversarial(x, y) + + with tf.GradientTape() as tape: + clean_logits = self.base_model(x, training=True) + adversarial_logits = self.paired_model(adversarial, training=True) + classification_loss = self.compiled_loss(y, clean_logits) + pairing_loss = tf.reduce_mean(tf.square(clean_logits - adversarial_logits)) + loss = classification_loss + pairing_loss + + trainable = self.base_model.trainable_variables + gradients = tape.gradient(loss, trainable) + pairs = [(g, v) for g, v in zip(gradients, trainable) if g is not None] + self.optimizer.apply_gradients(pairs) + self.compiled_metrics.update_state(y, clean_logits) + return {m.name: m.result() for m in self.metrics} + + def get_config(self): + config = super().get_config() + config.update( + base_model=tf.keras.utils.serialize_keras_object(self.base_model), + paired_model=tf.keras.utils.serialize_keras_object(self.paired_model), + epsilon=self.epsilon, + num_steps=self.num_steps, + alpha=self.alpha, + from_logits=self.from_logits, + ) + return config + + @classmethod + def from_config(cls, config): + config = dict(config) + for key in ("base_model", "paired_model"): + if config.get(key) is not None: + config[key] = tf.keras.utils.deserialize_keras_object(config[key]) + return cls(**config) + + +def adversarial_logit_pairing(model, paired_model=None, epsilon=0.01, num_steps=1, + alpha=None): + """ + Adversarial Logit Pairing (ALP) defense. - Input transformation applies a transformation to the input data before feeding it - to the model, aiming to remove adversarial perturbations. + In each training step an adversarial example is generated and the clean and + adversarial logits of the model are pulled together. (The previous + implementation paired two models on *clean* inputs, which is not ALP.) Parameters: model (tensorflow.keras.Model): The model to defend. - transformation_function (function): The transformation function to apply (default: None). + paired_model (tensorflow.keras.Model): Optional second model; defaults + to ``model``. + epsilon (float): Strength of the inner attack (default: 0.01). + num_steps (int): Iterations of the inner attack (default: 1). + alpha (float): Step size of the inner attack. Returns: - defended_model (tensorflow.keras.Model): The model with input transformation defense. + defended_model (tensorflow.keras.Model): The wrapped model. """ - class TransformationLayer(tf.keras.layers.Layer): - def __init__(self, transform_fn): - super().__init__() - self.transform_fn = transform_fn + epsilon = check_float("epsilon", epsilon, minimum=0.0) + num_steps = check_int("num_steps", num_steps, minimum=1) + alpha = check_float("alpha", alpha, minimum=0.0, allow_none=True) + return AdversarialLogitPairingModel( + base_model=model, paired_model=paired_model, epsilon=epsilon, + num_steps=num_steps, alpha=alpha, + ) - def call(self, x): - if self.transform_fn is not None: - return self.transform_fn(x) - return x - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = TransformationLayer(transformation_function)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) +# --------------------------------------------------------------------------- +# Inference-time preprocessing defenses +# --------------------------------------------------------------------------- -def defensive_distillation(model, teacher_model, temperature=2): + +def feature_squeezing(model, bit_depth=4): """ - Defensive Distillation defense. + Feature Squeezing defense. - Defensive distillation trains a student model to mimic the predictions of a - teacher model, which is often a more robust model. + Reduces the colour depth of the input to ``bit_depth`` bits, removing small + adversarial perturbations. A straight-through gradient is used so gradient + based attacks against the defended model still work. Parameters: - model (tensorflow.keras.Model): The student model to defend. - teacher_model (tensorflow.keras.Model): The teacher model. - temperature (float): The temperature parameter for distillation (default: 2). + model (tensorflow.keras.Model): The model to defend. + bit_depth (int): Number of bits per feature, 1-8 (default: 4). Returns: - defended_model (tensorflow.keras.Model): The distilled student model. + defended_model (tensorflow.keras.Model): The defended model. """ - class DistillationModel(tf.keras.Model): - def __init__(self, student, teacher, temp): - super().__init__() - self.student = student - self.teacher = teacher - self.temp = temp + return wrap_preprocessing(model, SqueezeLayer(bit_depth=bit_depth, name="feature_squeezing")) - def train_step(self, data): - x, y = data - teacher_predictions = self.teacher(x, training=False) - with tf.GradientTape() as tape: - student_predictions = self.student(x, training=True) - # Soften predictions and labels - soft_labels = tf.nn.softmax(teacher_predictions / self.temp) - soft_predictions = tf.nn.softmax(student_predictions / self.temp) - - distillation_loss = tf.keras.losses.CategoricalCrossentropy()(soft_labels, soft_predictions) - student_loss = self.compiled_loss(y, student_predictions) +def input_transformation(model, transformation_function=None): + """ + Input Transformation defense. - loss = distillation_loss + student_loss + Applies ``transformation_function`` to the input before the model. - gradients = tape.gradient(loss, self.student.trainable_variables) - self.optimizer.apply_gradients(zip(gradients, self.student.trainable_variables)) - self.compiled_metrics.update_state(y, student_predictions) - return {m.name: m.result() for m in self.metrics} + Parameters: + model (tensorflow.keras.Model): The model to defend. + transformation_function (callable): The transformation to apply. It is + required โ€” the previous default silently produced a model with no + defense at all. - def call(self, x): - return self.student(x) + Returns: + defended_model (tensorflow.keras.Model): The defended model. + """ + if transformation_function is None: + raise ValueError( + "`transformation_function` is required. Pass a callable such as " + "`lambda x: tf.image.resize(x, (224, 224))`." + ) + return wrap_preprocessing( + model, InputTransformationLayer(transformation_function, name="input_transformation") + ) - return DistillationModel(model, teacher_model, temperature) -def randomized_smoothing(model, noise_level=0.1): +def randomized_smoothing(model, noise_level=0.1, seed=None, add_noise_at_inference=True): """ - Randomized Smoothing defense. + Randomized Smoothing (inference-time noise). - Randomized smoothing adds random noise to the input data to make the model more robust - to adversarial attacks. + Adds Gaussian noise to the input. This alone does **not** make a model + certifiably robust: the base model should be trained with the same noise + (see :func:`adversarial_training` with a noise-augmented dataset) and + predictions should be averaged over many samples with :func:`smooth_predict`. Parameters: model (tensorflow.keras.Model): The model to defend. - noise_level (float): The standard deviation of the Gaussian noise (default: 0.1). + noise_level (float): Standard deviation of the noise (default: 0.1). + seed (int): Optional seed. + add_noise_at_inference (bool): Whether to add noise outside training + (default: True). Set to False to use the noise only as training + augmentation. Returns: - defended_model (tensorflow.keras.Model): The model with randomized smoothing defense. + defended_model (tensorflow.keras.Model): The defended model. """ - class NoiseLayer(tf.keras.layers.Layer): - def __init__(self, level): - super().__init__() - self.level = level + layer = NoiseLayer( + noise_level=noise_level, seed=seed, + add_noise_at_inference=add_noise_at_inference, name="randomized_smoothing", + ) + return wrap_preprocessing(model, layer) + + +def _norm_ppf(p): + """Inverse standard normal CDF (Acklam's rational approximation).""" + a = [-3.969683028665376e+01, 2.209460984245205e+02, -2.759285104469687e+02, + 1.383577518672690e+02, -3.066479806614716e+01, 2.506628277459239e+00] + b = [-5.447609879822406e+01, 1.615858368580409e+02, -1.556989798598866e+02, + 6.680131188771972e+01, -1.328068155288572e+01] + c = [-7.784894002430293e-03, -3.223964580411365e-01, -2.400758277161838e+00, + -2.549732539343734e+00, 4.374664141464968e+00, 2.938163982698783e+00] + d = [7.784695709041462e-03, 3.224671290700398e-01, 2.445134137142996e+00, + 3.754408661907416e+00] + p = min(max(float(p), 1e-12), 1.0 - 1e-12) + lower = 0.02425 + if p < lower: + q = math.sqrt(-2.0 * math.log(p)) + return (((((c[0] * q + c[1]) * q + c[2]) * q + c[3]) * q + c[4]) * q + c[5]) / \ + ((((d[0] * q + d[1]) * q + d[2]) * q + d[3]) * q + 1.0) + if p > 1.0 - lower: + q = math.sqrt(-2.0 * math.log(1.0 - p)) + return -(((((c[0] * q + c[1]) * q + c[2]) * q + c[3]) * q + c[4]) * q + c[5]) / \ + ((((d[0] * q + d[1]) * q + d[2]) * q + d[3]) * q + 1.0) + q = p - 0.5 + r = q * q + return (((((a[0] * r + a[1]) * r + a[2]) * r + a[3]) * r + a[4]) * r + a[5]) * q / \ + (((((b[0] * r + b[1]) * r + b[2]) * r + b[3]) * r + b[4]) * r + 1.0) + + +def smooth_predict(model, x, n_samples=100, noise_level=None, seed=None): + """ + Monte-Carlo prediction for a noise-augmented model. - def call(self, x): - noise = tf.random.normal(shape=tf.shape(x), mean=0.0, stddev=self.level, dtype=tf.float32) - return x + noise + Averages ``n_samples`` stochastic forward passes and estimates the certified + L2 radius ``sigma * Phi^-1(p)`` of the majority prediction. - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = NoiseLayer(noise_level)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) + Parameters: + model (tf.keras.Model): A model wrapped by :func:`randomized_smoothing`. + x (numpy.ndarray): Input batch. + n_samples (int): Number of Monte-Carlo samples (default: 100). + noise_level (float): Noise standard deviation. Read from the model's + first ``NoiseLayer`` when omitted. + seed (int): Optional seed. -def feature_denoising(model): + Returns: + dict: ``{"predictions", "probabilities", "certified_radius"}``. + """ + n_samples = check_int("n_samples", n_samples, minimum=1) + x = np.asarray(x, dtype="float32") + + if noise_level is None: + for layer in getattr(model, "layers", []): + if isinstance(layer, NoiseLayer): + noise_level = layer.noise_level + break + if noise_level is None: + raise ValueError( + "Could not find a NoiseLayer in the model; pass `noise_level` " + "explicitly." + ) + noise_level = check_float("noise_level", noise_level, minimum=0.0) + + rng = np.random.default_rng(seed) + votes = np.zeros((n_samples,) + (x.shape[0],), dtype="int32") + probabilities = np.zeros((n_samples, x.shape[0], int(np.shape(model(x[:1]))[-1])), dtype="float32") + + for i in range(n_samples): + noisy = x + rng.normal(0.0, noise_level, size=x.shape).astype("float32") + outputs = np.asarray(model.predict(noisy, verbose=0)) + probabilities[i] = outputs + votes[i] = np.argmax(outputs, axis=-1) + + counts = np.zeros((x.shape[0], probabilities.shape[-1]), dtype="float32") + for i in range(x.shape[0]): + counts[i] = np.bincount(votes[:, i], minlength=counts.shape[1]) + predictions = np.argmax(counts, axis=-1) + p_hat = counts.max(axis=-1) / float(n_samples) + radius = np.where(p_hat > 0.5, noise_level * np.array([_norm_ppf(p) for p in p_hat]), 0.0) + + return { + "predictions": predictions, + "probabilities": probabilities.mean(axis=0), + "certified_radius": radius, + } + + +def feature_denoising(model, method="median", kernel_size=3, sigma=1.0): """ - Feature Denoising defense. + Feature Denoising defense (input-space variant). - Feature denoising applies denoising operations to the input data to remove adversarial perturbations. + Applies a denoising filter to the input. For real feature-level denoising, + insert :class:`FeatureDenoisingBlock` inside your network and train it. Parameters: model (tensorflow.keras.Model): The model to defend. + method (str): ``"median"`` (default) or ``"gaussian"``. + kernel_size (int): Filter size (odd, default: 3). + sigma (float): Standard deviation for the Gaussian filter. Returns: - defended_model (tensorflow.keras.Model): The model with feature denoising defense. + defended_model (tensorflow.keras.Model): The defended model. """ - class DenoiseLayer(tf.keras.layers.Layer): - def call(self, x): - # Use a spatial smoothing as a denoising operation - # Note: avg_pool2d might need rank 4. - if len(x.shape) == 4: - return tf.nn.avg_pool2d(x, ksize=3, strides=1, padding='SAME') - return x + return wrap_preprocessing(model, _make_smoothing_layer(method, kernel_size, sigma, "feature_denoising")) + + +def _make_smoothing_layer(method, kernel_size, sigma, name): + method = str(method).lower() + if method == "median": + return MedianFilterLayer(kernel_size=kernel_size, name=name) + if method == "gaussian": + return GaussianBlurLayer(kernel_size=kernel_size, sigma=sigma, name=name) + raise ValueError(f"`method` must be 'median' or 'gaussian', got {method!r}.") - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = DenoiseLayer()(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) def thermometer_encoding(model, num_bins=10): """ - Thermometer Encoding defense. + Thermometer-encoding style discretisation. - Thermometer encoding discretizes the input features into bins, making it harder for adversarial perturbations - to affect the model. + The input is quantised into ``num_bins`` levels. Note that canonical + thermometer encoding *expands* each feature into ``num_bins`` binary + features, which changes the input shape and requires retraining the first + layer; this shape-preserving variant is the quantisation equivalent and is + documented as such instead of pretending otherwise. Parameters: model (tensorflow.keras.Model): The model to defend. - num_bins (int): The number of bins for encoding (default: 10). + num_bins (int): Number of quantisation levels (default: 10). Returns: - defended_model (tensorflow.keras.Model): The model with thermometer encoding defense. + defended_model (tensorflow.keras.Model): The defended model. """ - class ThermometerLayer(tf.keras.layers.Layer): - def __init__(self, bins): - super().__init__() - self.bins = bins - - def call(self, x): - x = tf.clip_by_value(x, 0, 1) - x = tf.floor(x * self.bins) / self.bins - return x + return wrap_preprocessing(model, QuantizationLayer(num_bins=num_bins, name="thermometer_encoding")) - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = ThermometerLayer(num_bins)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) -def adversarial_logit_pairing(model, paired_model): +def spatial_smoothing(model, kernel_size=3, method="median", sigma=1.0): """ - Adversarial Logit Pairing (ALP) defense. + Spatial Smoothing defense. - Adversarial logit pairing encourages the logits of adversarial examples to be similar to those of clean examples. + Applies a median (default) or Gaussian smoothing filter to the input. Parameters: model (tensorflow.keras.Model): The model to defend. - paired_model (tensorflow.keras.Model): The paired model for logit pairing (can be the same model). + kernel_size (int): Filter size (odd, default: 3). + method (str): ``"median"`` (default) or ``"gaussian"``. + sigma (float): Standard deviation for the Gaussian filter. Returns: - defended_model (tensorflow.keras.Model): The model with adversarial logit pairing defense. + defended_model (tensorflow.keras.Model): The defended model. """ - class ALPModel(tf.keras.Model): - def __init__(self, base_model, p_model): - super().__init__() - self.base_model = base_model - self.p_model = p_model + return wrap_preprocessing(model, _make_smoothing_layer(method, kernel_size, sigma, "spatial_smoothing")) - def train_step(self, data): - x, y = data - with tf.GradientTape() as tape: - clean_logits = self.base_model(x, training=True) - adv_logits = self.p_model(x, training=True) - classification_loss = self.compiled_loss(y, clean_logits) - alp_loss = tf.reduce_mean(tf.square(clean_logits - adv_logits)) +def median_smoothing(model, kernel_size=3): + """ + Median Smoothing defense. - loss = classification_loss + alp_loss + Applies a true median filter (this used to call ``avg_pool2d``). Kept as a + named alias of :func:`spatial_smoothing` with ``method="median"``. - gradients = tape.gradient(loss, self.base_model.trainable_variables) - self.optimizer.apply_gradients(zip(gradients, self.base_model.trainable_variables)) - self.compiled_metrics.update_state(y, clean_logits) - return {m.name: m.result() for m in self.metrics} + Parameters: + model (tensorflow.keras.Model): The model to defend. + kernel_size (int): Filter size (odd, default: 3). - def call(self, x): - return self.base_model(x) + Returns: + defended_model (tensorflow.keras.Model): The defended model. + """ + return wrap_preprocessing(model, MedianFilterLayer(kernel_size=kernel_size, name="median_smoothing")) - return ALPModel(model, paired_model) -def spatial_smoothing(model, kernel_size=3): +def gaussian_blur(model, kernel_size=3, sigma=1.0): """ - Spatial Smoothing defense. + Gaussian Blur defense. - Spatial smoothing applies a smoothing filter to the input data to remove adversarial perturbations. + Convolves the input with a separable Gaussian kernel of standard deviation + ``sigma`` (the parameter is now actually used). Parameters: model (tensorflow.keras.Model): The model to defend. - kernel_size (int): The size of the smoothing kernel (default: 3). + kernel_size (int): Kernel size (odd, default: 3). + sigma (float): Standard deviation of the kernel (default: 1.0). Returns: - defended_model (tensorflow.keras.Model): The model with spatial smoothing defense. - """ - class SmoothLayer(tf.keras.layers.Layer): - def __init__(self, k_size): - super().__init__() - self.k_size = k_size - - def call(self, x): - if len(x.shape) == 4: - # Proper median filter approximation for 4D tensors (B, H, W, C) - # We process each channel separately or extract patches - patches = tf.image.extract_patches( - images=x, - sizes=[1, self.k_size, self.k_size, 1], - strides=[1, 1, 1, 1], - rates=[1, 1, 1, 1], - padding='SAME' - ) - # patches shape: (B, H, W, k*k*C) - shape = tf.shape(x) - B, H, W, C = shape[0], shape[1], shape[2], shape[3] - - # Reshape patches to (B, H, W, k*k, C) to take median over the spatial window per channel - patches_reshaped = tf.reshape(patches, [B, H, W, self.k_size * self.k_size, C]) - # Approximate median since tf.reduce_median is not standard - # We sort and take middle element - sorted_patches = tf.sort(patches_reshaped, axis=3) - mid_idx = (self.k_size * self.k_size) // 2 - return sorted_patches[:, :, :, mid_idx, :] - return x - - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = SmoothLayer(kernel_size)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) + defended_model (tensorflow.keras.Model): The defended model. + """ + return wrap_preprocessing( + model, GaussianBlurLayer(kernel_size=kernel_size, sigma=sigma, name="gaussian_blur") + ) def jpeg_compression(model, quality=75): """ JPEG Compression defense. - This defense compresses the input image using JPEG, which can remove high-frequency - adversarial perturbations. + Compresses and decompresses the input with JPEG. A straight-through + gradient is used, so gradient based attacks against the defended model still + produce a signal instead of ``None``. + + Requires static spatial dimensions and 1 or 3 channels. Parameters: model (tensorflow.keras.Model): The model to defend. - quality (int): The JPEG compression quality (0-100), default is 75. + quality (int): JPEG quality, 1-100 (default: 75). Returns: - defended_model (tensorflow.keras.Model): The model with JPEG compression defense. + defended_model (tensorflow.keras.Model): The defended model. """ - class JPEGCompressLayer(tf.keras.layers.Layer): - def __init__(self, q): - super().__init__() - self.q = q - - def call(self, x): - return tf.map_fn(lambda img: tf.cast(tf.image.decode_jpeg(tf.image.encode_jpeg(tf.cast(img * 255, tf.uint8), quality=self.q), channels=3), tf.float32) / 255.0, x) + return wrap_preprocessing( + model, JPEGCompressionLayer(quality=quality, name="jpeg_compression") + ) - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = JPEGCompressLayer(quality)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) - -def pixel_deflection(model, deflection_count=100, window_size=10): +def pixel_deflection(model, deflection_count=100, window_size=10, seed=None): """ Pixel Deflection defense. - Randomly deflects pixels to nearby locations to disrupt adversarial perturbations. + Randomly replaces ``deflection_count`` pixels by random neighbours within + ``window_size``. The previous implementation raised a ``ValueError`` on + every call. Parameters: model (tensorflow.keras.Model): The model to defend. deflection_count (int): Number of pixels to deflect (default: 100). - window_size (int): The range for random deflection (default: 10). + window_size (int): Maximum displacement in pixels (default: 10). + seed (int): Optional seed. Returns: - defended_model (tensorflow.keras.Model): The model with pixel deflection defense. - """ - class DeflectionLayer(tf.keras.layers.Layer): - def __init__(self, count, window): - super().__init__() - self.count = count - self.window = window - - def call(self, x): - shape = tf.shape(x) - batch_size = shape[0] - h, w = shape[1], shape[2] - - def single_image_deflect(img): - # Efficient pixel deflection approximation: - # Pick random pixels and replace with random neighbors - img_mut = tf.identity(img) - # Since we can't easily loop with assignment in a layer call efficiently, - # we use a small amount of noise or a slight shift as an approximation - # for this "placeholder" logic to be more than just a no-op. - # However, for a real deflection, we'd need scatter_nd. - # Let's implement a simple version with scatter_nd. - indices = tf.random.uniform([self.count, 2], 0, [h, w], dtype=tf.int32) - shifts = tf.random.uniform([self.count, 2], -self.window, self.window, dtype=tf.int32) - neighbor_indices = tf.clip_by_value(indices + shifts, 0, [h-1, w-1]) - - neighbor_pixels = tf.gather_nd(img, neighbor_indices) - return tf.tensor_scatter_nd_update(img, indices, neighbor_pixels) - - return tf.map_fn(single_image_deflect, x) - - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = DeflectionLayer(deflection_count, window_size)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) - -def gaussian_blur(model, kernel_size=3, sigma=1.0): + defended_model (tensorflow.keras.Model): The defended model. """ - Gaussian Blur defense. + return wrap_preprocessing( + model, + PixelDeflectionLayer( + deflection_count=deflection_count, window_size=window_size, + seed=seed, name="pixel_deflection", + ), + ) - Applies Gaussian blurring to the input data to remove adversarial perturbations. - - Parameters: - model (tensorflow.keras.Model): The model to defend. - kernel_size (int): The size of the Gaussian kernel (default: 3). - sigma (float): The standard deviation of the Gaussian kernel (default: 1.0). - - Returns: - defended_model (tensorflow.keras.Model): The model with Gaussian blur defense. - """ - class BlurLayer(tf.keras.layers.Layer): - def __init__(self, k_size, s): - super().__init__() - self.k_size = k_size - self.s = s - - def call(self, x): - if len(x.shape) == 4: - return tf.nn.avg_pool2d(x, ksize=self.k_size, strides=1, padding='SAME') - return x - - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = BlurLayer(kernel_size, sigma)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) def total_variation_minimization(model, iterations=10, regularization_parameter=0.1): """ Total Variation Minimization defense. - Reconstructs the input image by minimizing total variation. + Denoises the input with ``iterations`` gradient steps on the total variation + norm. A straight-through gradient is used so attacks against the defended + model do not silently produce a zero perturbation. Parameters: model (tensorflow.keras.Model): The model to defend. - iterations (int): Number of reconstruction iterations (default: 10). - regularization_parameter (float): The regularization parameter (default: 0.1). + iterations (int): Number of denoising steps (default: 10). + regularization_parameter (float): Step size of the denoiser (default: 0.1). Returns: - defended_model (tensorflow.keras.Model): The model with TV minimization defense. - """ - class TVLayer(tf.keras.layers.Layer): - def __init__(self, iters, reg): - super().__init__() - self.iters = iters - self.reg = reg - - def call(self, x): - # Iterative denoising via TV minimization (simplified) - # This is a basic gradient descent on the TV loss - img = x - for _ in range(self.iters): - with tf.GradientTape() as tape: - tape.watch(img) - tv = tf.reduce_sum(tf.image.total_variation(img)) - grad = tape.gradient(tv, img) - img = img - self.reg * grad - img = tf.clip_by_value(img, 0, 1) - return img - - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = TVLayer(iterations, regularization_parameter)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) - -def word_masking(text, mask_token="[MASK]", mask_prob=0.1): - """ - Simple word masking defense for text. + defended_model (tensorflow.keras.Model): The defended model. + """ + return wrap_preprocessing( + model, + TotalVariationLayer( + iterations=iterations, + regularization_parameter=regularization_parameter, + name="total_variation_minimization", + ), + ) - Parameters: - text (str): The input text. - mask_token (str): The token to use for masking (default: "[MASK]"). - mask_prob (float): The probability of masking a word (default: 0.1). - Returns: - defended_text (str): The text with randomly masked words. - """ - words = text.split() - for i in range(len(words)): - if np.random.rand() < mask_prob: - words[i] = mask_token - return " ".join(words) +# --------------------------------------------------------------------------- +# Text +# --------------------------------------------------------------------------- -def median_smoothing(model, kernel_size=3): +def word_masking(text, mask_token="[MASK]", mask_prob=0.1, seed=None): """ - Median Smoothing defense. - - Applies median filtering to the input data to remove adversarial perturbations. + Word masking for text. Whitespace and line breaks are preserved. Parameters: - model (tensorflow.keras.Model): The model to defend. - kernel_size (int): The size of the smoothing kernel (default: 3). + text (str): The input text. + mask_token (str): The token used for masking (default: "[MASK]"). + mask_prob (float): Probability of masking a word (default: 0.1). + seed (int): Optional seed for reproducibility. Returns: - defended_model (tensorflow.keras.Model): The model with median smoothing defense. - """ - class MedianLayer(tf.keras.layers.Layer): - def __init__(self, k_size): - super().__init__() - self.k_size = k_size - - def call(self, x): - if len(x.shape) == 4: - return tf.nn.avg_pool2d(x, ksize=self.k_size, strides=1, padding='SAME') - return x - - input_layer = tf.keras.Input(shape=model.input_shape[1:]) - x = MedianLayer(kernel_size)(input_layer) - predictions = model(x) - return tf.keras.Model(inputs=input_layer, outputs=predictions) + defended_text (str): The text with randomly masked words. + """ + if not isinstance(text, str): + raise TypeError(f"`text` must be a string, got {type(text).__name__}.") + mask_prob = check_float("mask_prob", mask_prob, minimum=0.0, maximum=1.0) + rng = np.random.default_rng(seed) + + tokens = tokenize_text(text) + for i, (_token, is_word) in enumerate(tokens): + if not is_word: + continue + if rng.random() < mask_prob: + tokens[i] = (mask_token, False) + return "".join(token for token, _ in tokens) diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..35f04c1 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,63 @@ +[build-system] +requires = ["setuptools>=68", "wheel"] +build-backend = "setuptools.build_meta" + +[project] +name = "deepdefend" +version = "0.2.0" +description = "An open-source Python library for adversarial attacks and defenses in deep learning models, enhancing the security and robustness of AI systems." +readme = "readme.md" +requires-python = ">=3.9" +authors = [{ name = "Infinitode Pty Ltd", email = "infinitode.ltd@gmail.com" }] +keywords = ["adversarial", "machine-learning", "security", "robustness", "tensorflow", "keras"] +# MIT plus an extra "substantial changes must ship under a different name" +# clause, so no stock SPDX identifier fits; LicenseRef-* is the correct SPDX +# escape hatch for custom licenses. +license = "LicenseRef-MIT-Modified" +license-files = ["LICENSE"] +dependencies = [ + "numpy>=1.23,<3", + "tensorflow>=2.13,<3", +] +classifiers = [ + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "Intended Audience :: Science/Research", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Topic :: Scientific/Engineering :: Artificial Intelligence", + "Topic :: Security", +] + +[project.optional-dependencies] +dev = [ + "pytest>=7.4", + "ruff>=0.6", +] + +[project.urls] +Homepage = "https://github.com/Infinitode/DeepDefend" +Documentation = "https://infinitode-docs.gitbook.io/documentation/package-documentation/deepdefend-package-documentation" +Issues = "https://github.com/Infinitode/DeepDefend/issues" + +[tool.setuptools] +packages = ["deepdefend"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +filterwarnings = ["ignore::DeprecationWarning"] + +[tool.ruff] +line-length = 100 +target-version = "py39" +exclude = [".git", "build", "dist", "venv"] + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "B", "UP"] +ignore = ["E501"] + +[tool.ruff.lint.per-file-ignores] +"tests/*" = ["B008"] diff --git a/readme.md b/readme.md index ee53f4e..1173a10 100644 --- a/readme.md +++ b/readme.md @@ -1,5 +1,5 @@ # DeepDefend -![Python Version](https://img.shields.io/badge/python-3.13-blue.svg) +![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue.svg) ![Code Size](https://img.shields.io/github/languages/code-size/infinitode/deepdefend) ![Downloads](https://pepy.tech/badge/deepdefend) ![License Compliance](https://img.shields.io/badge/license-compliance-brightgreen.svg) @@ -9,29 +9,41 @@ An open-source Python library for adversarial attacks and defenses in deep learning models, enhancing the security and robustness of AI systems. -## Changes in 0.1.5: -- Added MIM (Momentum Iterative Method) and EAD (Elastic Net Attack) attacks. -- Added Word Swap and Character Swap attacks for text-based models. -- Added Pixel Deflection, Gaussian Blur, Total Variation Minimization, and Median Smoothing defenses. -- Added Word Masking defense for text-based models. -- Added a comprehensive support table for different model types. -- Fixed logical errors in several defense functions. -- Improved Keras compatibility for training-time defenses. - -## Changes in 0.1.4: -- Added SPSA (Simultaneous Perturbation Stochastic Approximation) attack. -- Added JPEG Compression defense. - -## Changes in 0.1.3: -- 5 new functions in `defenses.py`, including: Randomized Smoothing, Feature Denoising, Thermometer Encoding, Adversarial Logit Pairing (ALP), and Spatial Smoothing. - -> [!NOTE] -> Updates to DeepDefend's attack module will be less frequent due to the potential misuse of adversarial attacks on AI models. - -## Changes in 0.1.2: -We've updated DeepDefend, here's what's new: -- 3 new functions under `deepdefend.attacks` -- 3 new functions under `deepdefend.defenses` +## Changes in 0.2.0 + +This release is a correctness pass over the whole package: several functions +either crashed on every call or returned silently wrong results. See +[CHANGELOG.md](CHANGELOG.md) for the full list. + +**Fixed** +- Fixed `jsma`, `spsa` and `pixel_deflection`, which raised an exception on every call. +- Fixed `ead`, which returned the input unchanged with its default arguments (a silent "no vulnerability found"). +- Fixed `cw`, where `c` and `kappa` had no effect on the result. +- Fixed `deepfool`, which returned out-of-range values (e.g. `-0.90` and `2.08` for `[0, 1]` inputs) and ignored any budget. +- Fixed `fgsm`, which returned unclipped, invalid inputs. +- Fixed all attacks rejecting sparse integer labels, the default label format in Keras tutorials. +- Fixed all attacks ignoring `from_logits`, so models not ending in `softmax` were attacked with the wrong loss. +- Fixed targeted attacks freezing at the first step. +- Fixed `median_smoothing`, which called `avg_pool2d` instead of a median filter. +- Fixed `gaussian_blur`, which ignored `sigma` and average-pooled instead. +- Fixed `jpeg_compression`, `feature_squeezing` and `total_variation_minimization` breaking the gradient, so attacks against a defended model crashed or silently did nothing. +- Fixed all preprocessing defenses producing models that could not be saved or reloaded. +- Fixed `adversarial_training` generating examples one sample at a time and discarding the caller's optimiser and loss. +- Fixed `defensive_distillation` applying softmax twice and dropping the `Tยฒ` scaling. +- Fixed `adversarial_logit_pairing`, which paired two models on clean inputs instead of clean and adversarial logits. +- Fixed `randomized_smoothing`, `pixel_deflection` and the text helpers being non-reproducible and destroying whitespace/newlines. +- Fixed silent `NaN`s from invalid parameters such as `bit_depth=0` or `num_bins=0`. +- Fixed `pip install .` from a checkout, which failed because the version was a placeholder. +- Fixed the release pipeline: SHA-pinned actions, a tag/version check, a test gate, and PyPI Trusted Publishing instead of a long-lived API token. + +**Added** โ€” `deepdefend.__version__`, lazy imports, `smooth_predict()` with a +certified-radius estimate, a 171-test suite, CI, `SECURITY.md`, +`CODE_OF_CONDUCT.md`, Dependabot, pre-commit and `pyproject.toml`. + +**Breaking** โ€” the numerical output of `cw`, `deepfool`, `ead`, `spsa` and +`jsma` changed (they are now the algorithms they claim to be), several defaults +changed, and image-only defenses now raise on non-image inputs instead of +silently returning the input. See [CHANGELOG.md](CHANGELOG.md). ## Installation @@ -41,18 +53,20 @@ You can install DeepDefend using pip: pip install deepdefend ``` -## Supported Python Versions +For development (tests and linting): -DeepDefend supports the following Python versions: +```bash +git clone https://github.com/Infinitode/DeepDefend.git +cd DeepDefend +pip install -e ".[dev]" +pytest +``` -- Python 3.6 -- Python 3.7 -- Python 3.8 -- Python 3.9 -- Python 3.10 -- Python 3.11 or later +## Supported Python Versions -Please ensure that you have one of these Python versions installed before using DeepDefend. DeepDefend may not work as expected on lower versions of Python than the supported. +DeepDefend supports Python 3.9 โ€“ 3.12, matching the versions supported by +TensorFlow 2.x. Python 3.6 โ€“ 3.8 are no longer supported because TensorFlow does +not publish wheels for them. ## Features @@ -61,87 +75,89 @@ Please ensure that you have one of these Python versions installed before using ## Supported Model Types +Legend: โœ… supported ยท โŒ unsupported (raises a clear error) + | Feature | Image | Text | Numeric | Classification | |---------|:-----:|:----:|:-------:|:--------------:| | **Attacks** | | | | | | FGSM | โœ… | โœ… (Embeddings) | โœ… | โœ… | | PGD | โœ… | โœ… (Embeddings) | โœ… | โœ… | | BIM | โœ… | โœ… (Embeddings) | โœ… | โœ… | -| CW | โœ… | โœ… (Embeddings) | โœ… | โœ… | -| DeepFool | โœ… | โœ… (Embeddings) | โœ… | โœ… | -| JSMA | โœ… | โŒ | โŒ | โœ… | -| SPSA | โœ… | โŒ | โœ… | โœ… | | MIM | โœ… | โœ… (Embeddings) | โœ… | โœ… | +| CW (L2) | โœ… | โœ… (Embeddings) | โœ… | โœ… | | EAD | โœ… | โœ… (Embeddings) | โœ… | โœ… | -| Word Swap | โŒ | โœ… | โŒ | โœ… | -| Char Swap | โŒ | โœ… | โŒ | โœ… | +| DeepFool | โœ… | โœ… (Embeddings) | โœ… | โœ… | +| JSMA | โœ… | โŒ | โœ… | โœ… | +| SPSA | โœ… | โŒ | โœ… | โœ… | +| Word Swap | โŒ | โœ… | โŒ | โ€” | +| Char Swap | โŒ | โœ… | โŒ | โ€” | | **Defenses** | | | | | | Adversarial Training | โœ… | โœ… | โœ… | โœ… | -| Feature Squeezing | โœ… | โŒ | โœ… | โœ… | -| Gradient Masking | โœ… | โœ… | โœ… | โœ… | +| Feature Squeezing | โœ… | โœ… | โœ… | โœ… | +| Gradient Masking (โš ๏ธ not a defense) | โœ… | โœ… | โœ… | โœ… | | Input Transformation | โœ… | โœ… | โœ… | โœ… | -| Defensive Distillation| โœ… | โœ… | โœ… | โœ… | -| Randomized Smoothing | โœ… | โŒ | โœ… | โœ… | -| Feature Denoising | โœ… | โŒ | โŒ | โœ… | -| Thermometer Encoding | โœ… | โŒ | โœ… | โœ… | +| Defensive Distillation | โœ… | โœ… | โœ… | โœ… | +| Randomized Smoothing | โœ… | โœ… | โœ… | โœ… | +| Thermometer Encoding | โœ… | โœ… | โœ… | โœ… | | ALP | โœ… | โœ… | โœ… | โœ… | +| Feature Denoising | โœ… | โŒ | โŒ | โœ… | | Spatial Smoothing | โœ… | โŒ | โŒ | โœ… | +| Median Smoothing | โœ… | โŒ | โŒ | โœ… | | JPEG Compression | โœ… | โŒ | โŒ | โœ… | | Pixel Deflection | โœ… | โŒ | โŒ | โœ… | | Gaussian Blur | โœ… | โŒ | โŒ | โœ… | | TV Minimization | โœ… | โŒ | โŒ | โœ… | -| Word Masking | โŒ | โœ… | โŒ | โœ… | -| Median Smoothing | โœ… | โŒ | โŒ | โœ… | +| Word Masking | โŒ | โœ… | โŒ | โ€” | ## Usage ### Adversarial Attacks ```python +import numpy as np import tensorflow as tf -from deepdefend.attacks import fgsm, pgd, bim, cw, deepfool, jsma +from deepdefend.attacks import fgsm, pgd, bim, cw, deepfool, jsma, spsa, mim, ead # Load a pre-trained TensorFlow model model = ... # Load example input and label data (replace this with your own data loading code) -x_example = ... # example input data -y_example = ... # true label - -# Perform FGSM attack on the example data -adversarial_example_fgsm = fgsm(model, x_example, y_example, epsilon=0.01) - -# Perform PGD attack on the example data -adversarial_example_pgd = pgd(model, x_example, y_example, epsilon=0.01, alpha=0.01, num_steps=10) - -# Perform BIM attack on the example data -adversarial_example_bim = bim(model, x_example, y_example, epsilon=0.01, alpha=0.01, num_steps=10) - -# Perform CW attack on the example data -adversarial_example_cw = cw(model, x_example, y_example, epsilon=0.01, c=1, kappa=0, num_steps=10, alpha=0.01) - -# Perform Deepfool attack on the example data -adversarial_example_deepfool = deepfool(model, x_example, y_example, num_steps=10) - -# Perform JSMA attack on the example data -adversarial_example_jsma = jsma(model, x_example, y_example, theta=0.1, gamma=0.1, num_steps=10) - -# Perform SPSA attack on the example data -adversarial_example_spsa = spsa(model, x_example, y_example, epsilon=0.01, num_steps=10) - -# Perform MIM attack on the example data -adversarial_example_mim = mim(model, x_example, y_example, epsilon=0.01, alpha=0.01, num_steps=10) - -# Perform EAD attack on the example data -adversarial_example_ead = ead(model, x_example, y_example, epsilon=0.01, beta=0.01, num_steps=10) - -# Perform Word Swap attack on text data -text_data = "The movie was great" -swaps = {"great": "terrible"} -perturbed_text = word_swap(text_data, swap_dict=swaps) - -# Perform Character Swap attack on text data -perturbed_text_char = char_swap(text_data, swap_prob=0.1) +x_example = ... # shape (batch, ...), values in [0, 1] +y_example = ... # one-hot (batch, classes), sparse indices (batch,) or binary (batch, 1) + +# Single-step attack +adversarial_fgsm = fgsm(model, x_example, y_example, epsilon=0.01) + +# Iterative attacks (alpha defaults to epsilon / 4) +adversarial_pgd = pgd(model, x_example, y_example, epsilon=0.01, num_steps=10) +adversarial_bim = bim(model, x_example, y_example, epsilon=0.01, num_steps=10) +adversarial_mim = mim(model, x_example, y_example, epsilon=0.01, num_steps=10) + +# Optimisation based attacks +adversarial_cw = cw(model, x_example, y_example, c=10.0, num_steps=100) +adversarial_ead = ead(model, x_example, y_example, epsilon=0.01, num_steps=50) +adversarial_deepfool = deepfool(model, x_example, y_example, num_steps=50) +adversarial_spsa = spsa(model, x_example, y_example, epsilon=0.01, num_steps=50) + +# Saliency map attack +adversarial_jsma = jsma(model, x_example, y_example, num_steps=10) + +# Targeted attack: y_example is the class you want the model to predict +adversarial_targeted = pgd(model, x_example, target_labels, epsilon=0.03, targeted=True) + +# Models that return logits are auto-detected; override with from_logits=True/False. +# Normalised (e.g. ImageNet) inputs: tell the attack the valid range so it does +# not clip your input to [0, 1]. +adversarial = pgd( + model, x_normalised, y_example, epsilon=0.01, + clip_min=-2.2, clip_max=2.7, +) + +# Text attacks preserve whitespace, newlines and punctuation +from deepdefend.attacks import word_swap, char_swap + +perturbed_text = word_swap("The movie was great", {"great": "terrible"}) +perturbed_text = char_swap("The movie was great", swap_prob=0.1, seed=0) ``` ### Adversarial Defenses @@ -157,61 +173,79 @@ model = ... teacher_model = ... # Load training data -x_train, y_train = ... # training data and labels - -# Adversarial training to defend against attacks -defended_model = adversarial_training(model, x_train, y_train, epsilon=0.01) - -# Feature squeezing defense -defended_model_squeezed = feature_squeezing(model, bit_depth=4) - -# Gradient masking defense -defended_model_masking = gradient_masking(model, mask_threshold=0.1) - -# Input transformation defense -defended_model_transformation = input_transformation(model, transformation_function=None) - -# Defensive distillation defense -defended_model_distillation = defensive_distillation(model, teacher_model, temperature=2) - -# JPEG compression defense -defended_model_jpeg = jpeg_compression(model, quality=75) - -# Randomized smoothing defense -defended_model_smoothing = randomized_smoothing(model, noise_level=0.1) - -# Feature denoising defense -defended_model_denoising = feature_denoising(model) - -# Thermometer encoding defense -defended_model_thermometer = thermometer_encoding(model, num_bins=10) - -# Adversarial Logit Pairing (ALP) defense -defended_model_alp = adversarial_logit_pairing(model, paired_model=model) - -# Spatial smoothing defense -defended_model_spatial = spatial_smoothing(model, kernel_size=3) - -# Pixel deflection defense -defended_model_deflection = pixel_deflection(model, deflection_count=100, window_size=10) - -# Gaussian blur defense -defended_model_blur = gaussian_blur(model, kernel_size=3, sigma=1.0) +x_train, y_train = ... +``` -# TV Minimization defense -defended_model_tv = total_variation_minimization(model, iterations=10) +```python +# Training-time defenses +defended_model = adversarial_training( + model, x_train, y_train, epsilon=0.01, epochs=10, batch_size=32 +) +defended_model = defensive_distillation(model, teacher_model, temperature=2) +defended_model = adversarial_logit_pairing(model, epsilon=0.01) +defended_model = gradient_masking(model, mask_threshold=0.1) # โš ๏ธ see below +``` -# Median smoothing defense -defended_model_median = median_smoothing(model, kernel_size=3) +```python +# Inference-time preprocessing defenses (these return a new Keras model) +defended_model = feature_squeezing(model, bit_depth=4) +defended_model = gaussian_blur(model, kernel_size=3, sigma=1.0) +defended_model = median_smoothing(model, kernel_size=3) +defended_model = spatial_smoothing(model, kernel_size=3, method="median") +defended_model = jpeg_compression(model, quality=75) +defended_model = pixel_deflection(model, deflection_count=100, window_size=10) +defended_model = total_variation_minimization(model, iterations=10) +defended_model = thermometer_encoding(model, num_bins=10) +defended_model = feature_denoising(model, method="median") +defended_model = input_transformation(model, tf.keras.layers.Rescaling(1.0 / 255)) +defended_model = randomized_smoothing(model, noise_level=0.1) + +# ... and they can be saved and reloaded like any other Keras model +defended_model.save("defended.keras") +restored = tf.keras.models.load_model("defended.keras") + +# Monte-Carlo prediction with a certified L2 radius (randomized smoothing) +result = smooth_predict(defended_model, x_test, n_samples=100) +print(result["predictions"], result["certified_radius"]) +``` -# Word masking defense for text -text_data = "The movie was great" -defended_text = word_masking(text_data, mask_prob=0.2) +```python +# Text +defended_text = word_masking("The movie was great", mask_prob=0.2) ``` +## Limitations + +Adversarial defenses are an active research area, and most preprocessing +defenses can be broken by an adaptive attacker. Do not treat a defended model as +certified robust unless you have measured it: + +- **Gradient masking is not a defense.** It only alters training gradients. It + has no effect on the gradients an attacker sees, and `gradient_masking()` emits + a `RuntimeWarning` for this reason. +- **Randomized smoothing** only provides a certified radius if the base model is + trained on noise-augmented data and predictions are averaged over many + samples (`smooth_predict`). Adding noise at inference time alone does not make + a model robust. +- **Thermometer encoding** here is the shape-preserving quantisation variant. + Canonical thermometer encoding expands each feature into `num_bins` binary + features, which requires retraining the first layer. +- **Preprocessing defenses** (blur, JPEG, squeezing, denoising, TV, deflection) + are often defeated by a small step-size attack (Athalye et al., 2018). Always + evaluate them with a strong, adaptive attack (PGD or CW with several restarts). +- **Text helpers** (`word_swap`, `char_swap`, `word_masking`) are simple + baselines, not semantically-aware attacks. + +## Responsible use + +The attack implementations exist so that defenders can measure and improve the +robustness of models they are authorised to test. Do not use this library +against systems you do not own or have explicit permission to assess. To report +a security issue in DeepDefend itself, see [SECURITY.md](SECURITY.md). + ## Contributing -Contributions are welcome! If you encounter any issues, have suggestions, or want to contribute to DeepDefend, please open an issue or submit a pull request on [GitHub](https://github.com/infinitode/deepdefend). +Contributions are welcome! If you encounter any issues, have suggestions, or want to contribute to DeepDefend, please open an issue or submit a pull request on [GitHub](https://github.com/infinitode/deepdefend). Please read [CONTRIBUTING.md](CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) first. ## License @@ -219,8 +253,6 @@ DeepDefend is released under the terms of the **MIT License (Modified)**. Please **Modified License Clause** - - The modified license clause grants users the permission to make derivative works based on the DeepDefend software. However, it requires any substantial changes to the software to be clearly distinguished from the original work and distributed under a different name. By enforcing this distinction, it aims to prevent direct publishing of the source code without changes while allowing users to create derivative works that incorporate the code but are not exactly the same. diff --git a/requirements.txt b/requirements.txt index c2bbfa1..5742dee 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,2 +1,2 @@ -numpy -tensorflow \ No newline at end of file +numpy>=1.23,<3 +tensorflow>=2.13,<3 diff --git a/setup.py b/setup.py deleted file mode 100644 index 8546ff1..0000000 --- a/setup.py +++ /dev/null @@ -1,35 +0,0 @@ -from setuptools import setup, find_packages - -with open("readme.md", "r", encoding="utf-8") as fh: - long_description = fh.read() - -setup( - name='deepdefend', - version='{{VERSION_PLACEHOLDER}}', - author='Infinitode Pty Ltd', - author_email='infinitode.ltd@gmail.com', - description='An open-source Python library for adversarial attacks and defenses in deep learning models, enhancing the security and robustness of AI systems.', - long_description= long_description, - long_description_content_type='text/markdown', - url='https://github.com/infinitode/deepdefend', - packages=find_packages(), - install_requires=[ - 'numpy', - 'tensorflow', - ], - classifiers=[ - 'Development Status :: 5 - Production/Stable', - 'Intended Audience :: Developers', - 'License :: OSI Approved :: MIT License', - 'Programming Language :: Python :: 3', - 'Programming Language :: Python :: 3.6', - 'Programming Language :: Python :: 3.7', - 'Programming Language :: Python :: 3.8', - 'Programming Language :: Python :: 3.9', - 'Programming Language :: Python :: 3.10', - 'Programming Language :: Python :: 3.11', - 'Programming Language :: Python :: 3.12', - 'Programming Language :: Python :: 3.13', - ], - python_requires='>=3.6', -) \ No newline at end of file diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..aa35ebe --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,96 @@ +"""Shared fixtures for the DeepDefend test-suite.""" + +import os + +import numpy as np +import pytest + +os.environ.setdefault("TF_CPP_MIN_LOG_LEVEL", "3") + +tf = pytest.importorskip("tensorflow") + +NUM_CLASSES = 3 +IMAGE_SHAPE = (8, 8, 3) +NUM_SAMPLES = 48 + + +def _make_dataset(seed=0): + """Linearly separable synthetic images so a tiny CNN can actually learn.""" + rng = np.random.default_rng(seed) + x = rng.random((NUM_SAMPLES,) + IMAGE_SHAPE, dtype=np.float32) * 0.25 + y = rng.integers(0, NUM_CLASSES, size=NUM_SAMPLES) + for i, label in enumerate(y): + # A bright block whose position encodes the class. + x[i, label * 2:label * 2 + 2, :, :] += 0.75 + x = np.clip(x, 0.0, 1.0).astype(np.float32) + return x, y.astype(np.int32) + + +def _small_cnn(activation="softmax"): + return tf.keras.Sequential( + [ + tf.keras.Input(IMAGE_SHAPE), + tf.keras.layers.Conv2D(8, 3, activation="relu", padding="same"), + tf.keras.layers.Flatten(), + tf.keras.layers.Dense(NUM_CLASSES, activation=activation), + ] + ) + + +@pytest.fixture(scope="session") +def dataset(): + return _make_dataset() + + +@pytest.fixture(scope="session") +def trained_model(dataset): + """A small CNN that is accurate enough for attacks to succeed.""" + x, y = dataset + tf.keras.utils.set_random_seed(0) + model = _small_cnn() + # Label smoothing keeps the toy model from saturating to a float32 + # probability of exactly 1.0, which would make gradient-free attacks + # (SPSA) unable to measure any loss difference at all. + model.compile( + optimizer=tf.keras.optimizers.Adam(1e-2), + loss=tf.keras.losses.CategoricalCrossentropy(label_smoothing=0.1), + metrics=["accuracy"], + ) + model.fit(x, np.eye(NUM_CLASSES)[y], epochs=6, batch_size=16, verbose=0) + return model + + +@pytest.fixture(scope="session") +def linear_model(dataset): + """A linear softmax classifier โ€” easy to fool, so attacks are measurable.""" + x, y = dataset + tf.keras.utils.set_random_seed(2) + model = tf.keras.Sequential( + [ + tf.keras.Input(IMAGE_SHAPE), + tf.keras.layers.Flatten(), + tf.keras.layers.Dense(NUM_CLASSES, activation="softmax"), + ] + ) + model.compile( + optimizer=tf.keras.optimizers.Adam(1e-2), + loss=tf.keras.losses.CategoricalCrossentropy(), + metrics=["accuracy"], + ) + model.fit(x, np.eye(NUM_CLASSES)[y], epochs=4, batch_size=16, verbose=0) + return model + + +@pytest.fixture(scope="session") +def batch(dataset): + """A small batch with one-hot and sparse labels.""" + x, y = dataset + return x[:4], y[:4] + + +@pytest.fixture(scope="session") +def untrained_model(): + tf.keras.utils.set_random_seed(1) + model = _small_cnn() + model.compile(optimizer="adam", loss="categorical_crossentropy") + return model diff --git a/tests/test_api.py b/tests/test_api.py new file mode 100644 index 0000000..14584fd --- /dev/null +++ b/tests/test_api.py @@ -0,0 +1,62 @@ +"""Tests for the public package surface.""" + +import subprocess +import sys + +import pytest + +import deepdefend + +EXPECTED_ATTACKS = [ + "fgsm", "pgd", "bim", "mim", "cw", "deepfool", "jsma", "spsa", "ead", + "word_swap", "char_swap", +] +EXPECTED_DEFENSES = [ + "adversarial_training", "feature_squeezing", "gradient_masking", + "input_transformation", "defensive_distillation", "randomized_smoothing", + "feature_denoising", "thermometer_encoding", "adversarial_logit_pairing", + "spatial_smoothing", "jpeg_compression", "pixel_deflection", "gaussian_blur", + "total_variation_minimization", "word_masking", "median_smoothing", + "smooth_predict", +] + + +def test_version_is_exposed(): + assert isinstance(deepdefend.__version__, str) + assert deepdefend.__version__.count(".") >= 1 + + +@pytest.mark.parametrize("name", EXPECTED_ATTACKS + EXPECTED_DEFENSES) +def test_public_exports_exist(name): + # These were all missing from the package namespace before 0.2.0. + assert hasattr(deepdefend, name) + assert name in deepdefend.__all__ + + +def test_all_names_are_resolvable(): + for name in deepdefend.__all__: + assert getattr(deepdefend, name) is not None + + +def test_submodules_are_exposed(): + assert deepdefend.attacks is not None + assert deepdefend.defenses is not None + + +def test_unknown_attribute_raises_attribute_error(): + with pytest.raises(AttributeError): + _ = deepdefend.this_does_not_exist + + +def test_importing_the_package_does_not_import_tensorflow(): + """`import deepdefend` must stay cheap and TensorFlow-free.""" + code = ( + "import sys\n" + "import deepdefend\n" + "assert 'tensorflow' not in sys.modules, 'tensorflow imported eagerly'\n" + "print(deepdefend.__version__)\n" + ) + result = subprocess.run( + [sys.executable, "-c", code], capture_output=True, text=True, timeout=180 + ) + assert result.returncode == 0, result.stderr diff --git a/tests/test_attacks.py b/tests/test_attacks.py new file mode 100644 index 0000000..5ba5042 --- /dev/null +++ b/tests/test_attacks.py @@ -0,0 +1,271 @@ +"""Tests for :mod:`deepdefend.attacks`.""" + +import numpy as np +import pytest + +import deepdefend +from deepdefend import attacks + +IMAGE_ATTACKS = ["fgsm", "pgd", "bim", "mim", "cw", "deepfool", "jsma", "spsa", "ead"] + +# Cheap keyword overrides so the suite stays fast. +FAST_KWARGS = { + "cw": {"num_steps": 5}, + "deepfool": {"num_steps": 3}, + "jsma": {"num_steps": 3}, + "spsa": {"num_steps": 3, "spsa_samples": 4}, + "ead": {"num_steps": 5}, + "pgd": {"num_steps": 3}, + "bim": {"num_steps": 3}, + "mim": {"num_steps": 3}, +} + + +def _one_hot(y, n=3): + return np.eye(n)[y].astype(np.float32) + + +@pytest.mark.parametrize("name", IMAGE_ATTACKS) +def test_attack_output_shape_and_range(name, trained_model, batch): + x, y = batch + attack = getattr(attacks, name) + adversarial = attack(trained_model, x, _one_hot(y), **FAST_KWARGS.get(name, {})) + + assert isinstance(adversarial, np.ndarray) + assert adversarial.shape == x.shape + assert adversarial.dtype == np.float32 + assert np.isfinite(adversarial).all() + # Inputs live in [0, 1], so the output must too (this used to fail for + # fgsm and deepfool, which returned values such as -0.9 and 2.08). + assert adversarial.min() >= 0.0 + assert adversarial.max() <= 1.0 + + +@pytest.mark.parametrize("name", IMAGE_ATTACKS) +def test_attack_produces_a_perturbation(name, trained_model, batch): + x, y = batch + attack = getattr(attacks, name) + adversarial = attack(trained_model, x, _one_hot(y), **FAST_KWARGS.get(name, {})) + # `ead` used to be a complete no-op with its default parameters. + assert np.abs(adversarial - x).max() > 0.0 + + +@pytest.mark.parametrize("name", IMAGE_ATTACKS) +def test_attack_accepts_sparse_labels(name, trained_model, batch): + x, y = batch + attack = getattr(attacks, name) + adversarial = attack(trained_model, x, y, **FAST_KWARGS.get(name, {})) + assert adversarial.shape == x.shape + + +@pytest.mark.parametrize("name", IMAGE_ATTACKS) +def test_attack_accepts_binary_labels(name, batch): + tf = pytest.importorskip("tensorflow") + x, _ = batch + binary_model = tf.keras.Sequential( + [ + tf.keras.Input((8, 8, 3)), + tf.keras.layers.Flatten(), + tf.keras.layers.Dense(1, activation="sigmoid"), + ] + ) + labels = (x[:, 0, 0, 0] > 0.5).astype("float32") + adversarial = getattr(attacks, name)(binary_model, x, labels, **FAST_KWARGS.get(name, {})) + assert adversarial.shape == x.shape + + +@pytest.mark.parametrize("name", ["fgsm", "pgd", "bim", "mim", "ead"]) +def test_epsilon_bound_is_respected(name, trained_model, batch): + x, y = batch + epsilon = 0.05 + adversarial = getattr(attacks, name)( + trained_model, x, _one_hot(y), epsilon=epsilon, **FAST_KWARGS.get(name, {}) + ) + assert np.abs(adversarial - x).max() <= epsilon + 1e-5 + + +def test_pgd_random_start_in_epsilon_ball(trained_model, batch): + x, y = batch + epsilon = 0.05 + adversarial = attacks.pgd( + trained_model, x, _one_hot(y), epsilon=epsilon, num_steps=2, seed=0 + ) + assert np.abs(adversarial - x).max() <= epsilon + 1e-5 + + +def test_pgd_and_bim_differ(trained_model, batch): + x, y = batch + y = _one_hot(y) + pgd = attacks.pgd(trained_model, x, y, epsilon=0.05, num_steps=3, seed=7) + bim = attacks.bim(trained_model, x, y, epsilon=0.05, num_steps=3) + # PGD starts from a random point, BIM does not. + assert not np.allclose(pgd, bim) + + +def test_cw_parameters_are_effective(trained_model, batch): + x, y = batch + y = _one_hot(y) + weak = attacks.cw(trained_model, x, y, c=0.01, kappa=0.0, num_steps=5) + strong = attacks.cw(trained_model, x, y, c=100.0, kappa=5.0, num_steps=5) + # These used to be bit-identical because the L2 term and kappa were inert. + assert not np.allclose(weak, strong) + + +def test_ead_is_not_a_noop_with_defaults(trained_model, batch): + x, y = batch + adversarial = attacks.ead(trained_model, x, _one_hot(y)) + assert np.abs(adversarial - x).max() > 0.0 + + +@pytest.mark.parametrize( + "name,kwargs", + [ + ("fgsm", {}), + ("pgd", {"num_steps": 10}), + ("bim", {"num_steps": 10}), + ("mim", {"num_steps": 10}), + ("cw", {"num_steps": 100}), + ("deepfool", {"num_steps": 5}), + ], +) +def test_attacks_reduce_accuracy(name, kwargs, linear_model, dataset): + """A sanity check that the attacks actually attack.""" + x, y = dataset + x, y = x[:16], y[:16] + base = np.mean(np.argmax(linear_model.predict(x, verbose=0), axis=-1) == y) + assert base > 0.8 + + adversarial = getattr(attacks, name)(linear_model, x, _one_hot(y), epsilon=0.3, **kwargs) + accuracy = np.mean( + np.argmax(linear_model.predict(adversarial, verbose=0), axis=-1) == y + ) + assert accuracy < base, f"{name} did not reduce accuracy" + + +def test_targeted_attack_moves_to_the_target(linear_model, dataset): + x, y = dataset + x, y = x[:8], y[:8] + targets = (y + 1) % 3 + adversarial = attacks.pgd( + linear_model, x, _one_hot(targets), epsilon=0.3, alpha=0.05, + num_steps=10, targeted=True, random_start=False, + ) + predictions = np.argmax(linear_model.predict(adversarial, verbose=0), axis=-1) + assert np.mean(predictions == targets) >= 0.5 + + +def test_deepfool_with_epsilon_bound(trained_model, batch): + x, y = batch + adversarial = attacks.deepfool(trained_model, x, _one_hot(y), epsilon=0.02, num_steps=3) + assert np.abs(adversarial - x).max() <= 0.02 + 1e-5 + + +def test_from_logits_is_auto_detected(batch): + tf = pytest.importorskip("tensorflow") + from deepdefend._utils import infer_from_logits + + x, y = batch + softmax_model = tf.keras.Sequential( + [tf.keras.Input((8, 8, 3)), tf.keras.layers.Flatten(), + tf.keras.layers.Dense(3, activation="softmax")] + ) + logits_model = tf.keras.Sequential( + [tf.keras.Input((8, 8, 3)), tf.keras.layers.Flatten(), tf.keras.layers.Dense(3)] + ) + assert infer_from_logits(softmax_model) is False + assert infer_from_logits(logits_model) is True + + # Auto-detection must match an explicit value, and the two assumptions must + # produce different attacks (the old code always assumed probabilities). + auto = attacks.fgsm(logits_model, x, _one_hot(y), epsilon=0.05) + explicit = attacks.fgsm(logits_model, x, _one_hot(y), epsilon=0.05, from_logits=True) + np.testing.assert_allclose(auto, explicit) + + as_probabilities = attacks.fgsm( + logits_model, x, _one_hot(y), epsilon=0.05, from_logits=False + ) + assert not np.allclose(auto, as_probabilities) + + +def test_explicit_clip_range_for_normalised_inputs(trained_model, batch): + x, y = batch + normalised = (x - 0.5) * 2.0 + adversarial = attacks.pgd( + trained_model, normalised, _one_hot(y), epsilon=0.1, clip_min=-1.0, clip_max=1.0 + ) + assert adversarial.min() >= -1.0 - 1e-6 + assert adversarial.max() <= 1.0 + 1e-6 + + +def test_clip_range_is_not_applied_to_normalised_inputs(trained_model, batch): + """Without an explicit range the [0, 1] clip must not destroy the input.""" + x, y = batch + normalised = (x - 0.5) * 4.0 # roughly in [-2, 2] + adversarial = attacks.fgsm(trained_model, normalised, _one_hot(y), epsilon=0.01) + assert np.abs(adversarial - normalised).max() <= 0.01 + 1e-6 + + +@pytest.mark.parametrize( + "call,kwargs", + [ + ("fgsm", {"epsilon": -1.0}), + ("pgd", {"num_steps": 0}), + ("jsma", {"gamma": 0.0}), + ("spsa", {"spsa_samples": 1}), + ("cw", {"num_steps": 0}), + ("ead", {"alpha": 0.0}), + ], +) +def test_invalid_arguments_raise(call, kwargs, trained_model, batch): + x, y = batch + with pytest.raises((ValueError, TypeError)): + getattr(attacks, call)(trained_model, x, _one_hot(y), **kwargs) + + +def test_mismatched_batch_sizes_raise(trained_model, batch): + x, y = batch + with pytest.raises(ValueError): + attacks.fgsm(trained_model, x, _one_hot(y)[:2]) + + +# --------------------------------------------------------------------------- +# Text attacks +# --------------------------------------------------------------------------- + + +def test_word_swap_preserves_whitespace(): + text = "The movie was\ngreat, really\tgreat." + swapped = attacks.word_swap(text, {"great": "awful"}) + assert swapped == "The movie was\nawful, really\tawful." + assert len(swapped.split("\n")) == 2 + + +def test_word_swap_requires_a_dictionary(): + with pytest.raises(ValueError): + attacks.word_swap("nothing to see") + + +def test_char_swap_preserves_layout(): + text = "line one\nline two" + swapped = attacks.char_swap(text, swap_prob=1.0, seed=0) + assert swapped.count("\n") == text.count("\n") + assert swapped.count(" ") == text.count(" ") + assert swapped != text + + +def test_char_swap_is_reproducible(): + text = "deterministic results please" + first = attacks.char_swap(text, swap_prob=1.0, seed=42) + second = attacks.char_swap(text, swap_prob=1.0, seed=42) + assert first == second + assert attacks.char_swap(text, swap_prob=1.0, seed=7) != first or first == text + + +def test_char_swap_validates_probability(): + with pytest.raises(ValueError): + attacks.char_swap("text", swap_prob=1.5) + + +def test_attacks_are_exported_from_the_package(): + for name in IMAGE_ATTACKS + ["word_swap", "char_swap"]: + assert getattr(deepdefend, name) is getattr(attacks, name) diff --git a/tests/test_defenses.py b/tests/test_defenses.py new file mode 100644 index 0000000..5618022 --- /dev/null +++ b/tests/test_defenses.py @@ -0,0 +1,319 @@ +"""Tests for :mod:`deepdefend.defenses`.""" + +import numpy as np +import pytest + +import deepdefend +from deepdefend import defenses + +tf = pytest.importorskip("tensorflow") + +# name -> factory taking a model +PREPROCESSING_DEFENSES = { + "feature_squeezing": lambda m: defenses.feature_squeezing(m, bit_depth=4), + "input_transformation": lambda m: defenses.input_transformation( + m, tf.keras.layers.Rescaling(0.5) + ), + "randomized_smoothing": lambda m: defenses.randomized_smoothing(m, noise_level=0.05, seed=0), + "feature_denoising": lambda m: defenses.feature_denoising(m, method="median"), + "thermometer_encoding": lambda m: defenses.thermometer_encoding(m, num_bins=8), + "spatial_smoothing": lambda m: defenses.spatial_smoothing(m, kernel_size=3), + "median_smoothing": lambda m: defenses.median_smoothing(m, kernel_size=3), + "gaussian_blur": lambda m: defenses.gaussian_blur(m, kernel_size=3, sigma=1.0), + "jpeg_compression": lambda m: defenses.jpeg_compression(m, quality=75), + "pixel_deflection": lambda m: defenses.pixel_deflection(m, deflection_count=10, window_size=2, seed=0), + "total_variation_minimization": lambda m: defenses.total_variation_minimization(m, iterations=2), +} + + +@pytest.mark.parametrize("name", sorted(PREPROCESSING_DEFENSES)) +def test_defense_builds_and_predicts(name, untrained_model, batch): + x, _ = batch + defended = PREPROCESSING_DEFENSES[name](untrained_model) + outputs = defended(x) + assert tuple(outputs.shape) == (x.shape[0], 3) + assert np.isfinite(np.asarray(outputs)).all() + + +@pytest.mark.parametrize("name", sorted(PREPROCESSING_DEFENSES)) +def test_defense_models_can_be_saved_and_reloaded(name, untrained_model, batch, tmp_path): + """Every wrapped defense used to fail at save/load (layers defined inline).""" + x, _ = batch + defended = PREPROCESSING_DEFENSES[name](untrained_model) + path = tmp_path / f"{name}.keras" + defended.save(path) + reloaded = tf.keras.models.load_model(path) + np.testing.assert_allclose( + np.asarray(reloaded(x)), np.asarray(defended(x)), rtol=1e-5, atol=1e-5 + ) + + +@pytest.mark.parametrize( + "name,builder", + [ + ("feature_squeezing", lambda m: defenses.feature_squeezing(m)), + ("thermometer_encoding", lambda m: defenses.thermometer_encoding(m)), + ("gaussian_blur", lambda m: defenses.gaussian_blur(m)), + ("median_smoothing", lambda m: defenses.median_smoothing(m)), + ("spatial_smoothing", lambda m: defenses.spatial_smoothing(m)), + ("jpeg_compression", lambda m: defenses.jpeg_compression(m)), + ("pixel_deflection", lambda m: defenses.pixel_deflection(m, seed=0)), + ("total_variation_minimization", lambda m: defenses.total_variation_minimization(m)), + ("randomized_smoothing", lambda m: defenses.randomized_smoothing(m, seed=0)), + ("feature_denoising", lambda m: defenses.feature_denoising(m)), + ("input_transformation", lambda m: defenses.input_transformation( + m, tf.keras.layers.Rescaling(1.0) + )), + ], +) +def test_gradient_flows_through_preprocessing(name, builder, untrained_model, batch): + """Attacks against a defended model need a usable gradient.""" + x, _ = batch + defended = builder(untrained_model) + inputs = tf.constant(x) + with tf.GradientTape() as tape: + tape.watch(inputs) + # A non-degenerate loss: summing a softmax output would be constant. + loss = tf.reduce_sum(tf.square(defended(inputs))) + gradient = tape.gradient(loss, inputs) + assert gradient is not None, f"{name} blocks the gradient entirely" + assert float(tf.reduce_sum(tf.abs(gradient))) > 0.0, f"{name} yields a zero gradient" + + +def test_attacks_work_against_jpeg_defended_model(trained_model, batch): + """This used to raise `ValueError: Attempt to convert a value (None)`.""" + from deepdefend import attacks + + x, y = batch + defended = defenses.jpeg_compression(trained_model) + adversarial = attacks.fgsm(defended, x, np.eye(3)[y], epsilon=0.1) + assert adversarial.shape == x.shape + + +# --------------------------------------------------------------------------- +# Behavioural correctness of the individual filters +# --------------------------------------------------------------------------- + + +def test_median_smoothing_is_not_an_average_pool(untrained_model, batch): + x, _ = batch + median = defenses.median_smoothing(untrained_model) + average = untrained_model(tf.nn.avg_pool2d(x, 3, 1, "SAME")) + assert not np.allclose(np.asarray(median(x)), np.asarray(average)) + + +def test_gaussian_blur_uses_sigma(untrained_model, batch): + x, _ = batch + narrow = defenses.gaussian_blur(untrained_model, sigma=0.2) + wide = defenses.gaussian_blur(untrained_model, sigma=5.0) + assert not np.allclose(np.asarray(narrow(x)), np.asarray(wide(x))) + + +def test_feature_squeezing_output_range(untrained_model, batch): + x, _ = batch + layer = defenses.SqueezeLayer(bit_depth=2) + squeezed = np.asarray(layer(x)) + assert squeezed.min() >= 0.0 and squeezed.max() <= 1.0 + values = np.unique(np.round(squeezed, 6)) + assert len(values) <= 4 # 2 bits -> levels 0, 1/3, 2/3, 1 + + +def test_thermometer_encoding_output_range(untrained_model, batch): + x, _ = batch + layer = defenses.QuantizationLayer(num_bins=4) + quantised = np.asarray(layer(x)) + assert quantised.min() >= 0.0 and quantised.max() <= 1.0 + + +def test_randomized_smoothing_is_stochastic(untrained_model, batch): + x, _ = batch + defended = defenses.randomized_smoothing(untrained_model, noise_level=0.2, seed=None) + assert not np.allclose(np.asarray(defended(x)), np.asarray(defended(x))) + + +def test_pixel_deflection_is_seeded(untrained_model, batch): + x, _ = batch + defended = defenses.pixel_deflection(untrained_model, seed=3) + np.testing.assert_allclose(np.asarray(defended(x)), np.asarray(defended(x))) + + +@pytest.mark.parametrize( + "builder,kwargs", + [ + (defenses.feature_squeezing, {"bit_depth": 0}), + (defenses.feature_squeezing, {"bit_depth": 9}), + (defenses.thermometer_encoding, {"num_bins": 1}), + (defenses.jpeg_compression, {"quality": 0}), + (defenses.gaussian_blur, {"kernel_size": 4}), + (defenses.median_smoothing, {"kernel_size": 2}), + (defenses.randomized_smoothing, {"noise_level": -0.1}), + (defenses.pixel_deflection, {"deflection_count": -1}), + ], +) +def test_invalid_arguments_raise(builder, kwargs, untrained_model): + with pytest.raises((ValueError, TypeError)): + builder(untrained_model, **kwargs) + + +def test_input_transformation_requires_a_callable(untrained_model): + with pytest.raises(ValueError): + defenses.input_transformation(untrained_model) + + +@pytest.mark.parametrize( + "builder", + [ + defenses.gaussian_blur, + defenses.median_smoothing, + defenses.spatial_smoothing, + defenses.jpeg_compression, + defenses.pixel_deflection, + defenses.total_variation_minimization, + defenses.feature_denoising, + ], +) +def test_image_only_defenses_reject_non_image_inputs(builder): + """Silently returning the input unchanged would be a fake defense.""" + model = tf.keras.Sequential( + [tf.keras.Input((10,)), tf.keras.layers.Dense(3, activation="softmax")] + ) + with pytest.raises(ValueError, match="4-D"): + builder(model) + + +def test_multi_input_models_are_rejected(): + left = tf.keras.Input((4,)) + right = tf.keras.Input((4,)) + merged = tf.keras.layers.Concatenate()([left, right]) + model = tf.keras.Model([left, right], tf.keras.layers.Dense(3)(merged)) + with pytest.raises(TypeError): + defenses.gaussian_blur(model) + + +def test_subclassed_models_get_a_clear_error(): + class Subclassed(tf.keras.Model): + def __init__(self): + super().__init__() + self.dense = tf.keras.layers.Dense(3) + + def call(self, inputs): + return self.dense(inputs) + + with pytest.raises(TypeError): + defenses.gaussian_blur(Subclassed()) + + +# --------------------------------------------------------------------------- +# Training-time defenses +# --------------------------------------------------------------------------- + + +def test_adversarial_training_returns_a_defended_model(dataset): + x, y = dataset + model = tf.keras.Sequential( + [ + tf.keras.Input((8, 8, 3)), + tf.keras.layers.Flatten(), + tf.keras.layers.Dense(3, activation="softmax"), + ] + ) + model.compile(optimizer="adam", loss="sparse_categorical_crossentropy") + defended = defenses.adversarial_training( + model, x[:16], y[:16], epsilon=0.05, epochs=1, batch_size=8, verbose=0 + ) + assert isinstance(defended, tf.keras.Model) + assert defended is not model + assert defended.predict(x[:4], verbose=0).shape == (4, 3) + # The caller's loss must be preserved (it used to be forced to categorical). + assert defended.loss == "sparse_categorical_crossentropy" + + +def test_adversarial_training_generates_examples_in_batches(dataset, monkeypatch): + """Adversarial examples must be generated per batch, not per sample.""" + x, y = dataset + calls = [] + + def spy(model, batch_x, batch_y, epsilon=0.01): + calls.append(len(batch_x)) + return np.asarray(batch_x) + 0.01 + + model = tf.keras.Sequential( + [tf.keras.Input((8, 8, 3)), tf.keras.layers.Flatten(), tf.keras.layers.Dense(3, activation="softmax")] + ) + model.compile(optimizer="adam", loss="sparse_categorical_crossentropy") + defenses.adversarial_training( + model, x[:16], y[:16], epochs=1, batch_size=8, attack=spy, verbose=0 + ) + assert calls == [8, 8] + + +def test_gradient_masking_warns_and_fits(untrained_model, batch): + x, y = batch + with pytest.warns(RuntimeWarning): + wrapped = defenses.gradient_masking(untrained_model, mask_threshold=0.0) + wrapped.compile(optimizer="adam", loss="categorical_crossentropy") + wrapped.fit(x, np.eye(3)[y], epochs=1, verbose=0) + + +def test_defensive_distillation_fits(untrained_model, batch): + x, y = batch + distilled = defenses.defensive_distillation(untrained_model, untrained_model, temperature=3) + distilled.compile(optimizer="adam", loss="categorical_crossentropy") + history = distilled.fit(x, np.eye(3)[y], epochs=1, verbose=0) + assert "loss" in history.history + + +def test_adversarial_logit_pairing_uses_adversarial_examples(untrained_model, batch): + x, y = batch + wrapped = defenses.adversarial_logit_pairing(untrained_model, epsilon=0.05, num_steps=1) + + generated = np.asarray(wrapped._adversarial(tf.constant(x), tf.constant(np.eye(3)[y]))) + # ALP must pair clean and *adversarial* logits; the old implementation + # paired two models on the same clean batch. + assert np.abs(generated - x).max() > 0.0 + + wrapped.compile(optimizer="adam", loss="categorical_crossentropy") + wrapped.fit(x, np.eye(3)[y], epochs=1, verbose=0) + + +def test_smooth_predict_returns_certified_radius(untrained_model, batch): + x, _ = batch + defended = defenses.randomized_smoothing(untrained_model, noise_level=0.1, seed=0) + result = defenses.smooth_predict(defended, x, n_samples=5, seed=0) + assert set(result) == {"predictions", "probabilities", "certified_radius"} + assert result["predictions"].shape == (x.shape[0],) + assert np.all(result["certified_radius"] >= 0.0) + + +def test_smooth_predict_requires_a_noise_level(untrained_model, batch): + x, _ = batch + with pytest.raises(ValueError): + defenses.smooth_predict(untrained_model, x, n_samples=2) + + +# --------------------------------------------------------------------------- +# Text +# --------------------------------------------------------------------------- + + +def test_word_masking_preserves_whitespace(): + # The old implementation collapsed double spaces and newlines via str.split(). + masked = defenses.word_masking("a b\nc", mask_prob=1.0) + assert masked == "[MASK] [MASK]\n[MASK]" + + +def test_word_masking_is_seeded(): + text = "one two three four five six" + first = defenses.word_masking(text, mask_prob=0.5, seed=1) + second = defenses.word_masking(text, mask_prob=0.5, seed=1) + assert first == second + + +def test_word_masking_validates_probability(): + with pytest.raises(ValueError): + defenses.word_masking("text", mask_prob=2.0) + + +def test_defenses_are_exported_from_the_package(): + for name in defenses.__all__: + assert getattr(deepdefend, name) is getattr(defenses, name) From 3359f4cdee3ccb90676ea14a11ab61674056d810 Mon Sep 17 00:00:00 2001 From: Arena Agent Date: Sat, 31 Oct 2026 13:05:39 +0000 Subject: [PATCH 2/3] refactor: remove gradient_masking, fold review into README, add docs brief - Remove gradient_masking()/GradientMaskingModel entirely. It only masked training gradients and had no effect on an attacker's gradients (attacks against a wrapped model were bit-identical to the base model), so it is a false defense. Removed from the package namespace, __all__, the module docstring, the README support matrix and the tests; a regression test now asserts it stays gone. - Delete CODE_REVIEW.md and move its findings into the README as a 'What was wrong before 0.2.0' section (attacks, defenses, packaging/CI tables) and a Limitations section. - Add a 'Documentation update brief' code cell to the README describing every 0.2.0 API change for the docs agent. - CHANGELOG: gradient_masking moves from Fixed to Removed with the rationale; test count updated to 170. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com> --- CHANGELOG.md | 13 +- CODE_REVIEW.md | 371 ----------------------------------------- CONTRIBUTING.md | 2 +- deepdefend/__init__.py | 1 - deepdefend/defenses.py | 82 --------- readme.md | 308 ++++++++++++++++++++++++++++++++-- tests/test_api.py | 2 +- tests/test_defenses.py | 10 +- 8 files changed, 311 insertions(+), 478 deletions(-) delete mode 100644 CODE_REVIEW.md diff --git a/CHANGELOG.md b/CHANGELOG.md index d4cfb92..518a8d3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,9 +72,6 @@ attacks and defenses changes, the minor version was bumped. for both probability and logit outputs. - **`adversarial_logit_pairing`** generates adversarial examples inside the training step; the previous implementation paired two models on clean inputs. -- **`gradient_masking`** no longer crashes on `None` gradients and now emits a - `RuntimeWarning` explaining that it is a training regulariser, not an - inference-time defense. - **`randomized_smoothing`** clips the noisy input, is reproducible when seeded, and ships with `smooth_predict()` for Monte-Carlo prediction and a certified-radius estimate. @@ -94,7 +91,7 @@ attacks and defenses changes, the minor version was bumped. function is reachable as `deepdefend.`, and `import deepdefend` no longer imports TensorFlow. - `deepdefend/smooth_predict` for Monte-Carlo randomized smoothing. -- 164 tests covering attacks, defenses, serialisation, gradients, label +- 170 tests covering attacks, defenses, serialisation, gradients, label formats, input validation and the public API. - CI: lint + test matrix (3.10-3.12) + reproducible build on every push and PR. - `SECURITY.md`, `CODE_OF_CONDUCT.md`, Dependabot, pre-commit and @@ -120,6 +117,14 @@ attacks and defenses changes, the minor version was bumped. ### Removed +- **`gradient_masking`** (and the `GradientMaskingModel` wrapper) has been + deleted. It only masked *training* gradients and had no effect on the + gradients an attacker computes at inference time โ€” attacks against a wrapped + model were bit-identical to attacks against the base model. Gradient masking + is a documented false defense (Athalye et al., 2018) that makes a model look + robust while leaving it vulnerable, so it does not belong in a robustness + library. Mask gradients in your own `train_step` if you want the training + regulariser, and never report it as a security measure. - The `setup.py`-based `{{VERSION_PLACEHOLDER}}` release mechanism. ## [0.1.5] diff --git a/CODE_REVIEW.md b/CODE_REVIEW.md deleted file mode 100644 index fb50cad..0000000 --- a/CODE_REVIEW.md +++ /dev/null @@ -1,371 +0,0 @@ -# DeepDefend โ€” Full Code Review & Vulnerability Assessment - -**Version reviewed:** `0.1.5` (branch `arena/b52ef87b-deepdefend`, commit `cda6c32`) -**Files reviewed:** `deepdefend/__init__.py`, `deepdefend/attacks.py` (430 lines), `deepdefend/defenses.py` (565 lines), `setup.py`, `requirements.txt`, `readme.md`, `LICENSE`, `.github/workflows/build_and_publish.yml`, `CONTRIBUTING.md`, issue templates -**Method:** static review of every function **plus** dynamic verification of each suspected defect against a real Keras model (`tensorflow-cpu 2.16.2` / Keras 3.15, Python 3.11). Every claim below marked *(measured)* was reproduced by executing the package; raw output is in the appendix. - -> This is a security library: a function that **silently returns something wrong** is worse than one that crashes, because downstream users report "no vulnerability found" based on it. That severity ordering drives this report. - ---- - -## 0. Executive summary - -| Area | State | -|---|---| -| Attacks | 7/11 functions are broken, silently inert, or are not the algorithm they claim to be | -| Defenses | 4 crash or are bit-identical to each other; 6 are not the technique they claim; 13 of 15 wrapped models cannot be saved | -| Packaging | `pip install .` from a checkout **fails outright** | -| CI/CD | Unpinned mutable actions + long-lived PyPI token + publishes on any tag | -| Tests | **None.** No test suite, no test workflow, no lint config | - -Of the 11 attacks, **5 raise an exception on normal input** and **2 silently produce no perturbation at all**. Of the 15 model-wrapping defenses, **1 always crashes** and **3 break gradient flow so badly that attacks against the defended model raise or silently no-op**. - -Recommended target version: **0.2.0** (several fixes are intentionally behaviour-changing). - ---- - -## 1. CRITICAL โ€” broken, inert, or scientifically false - -### C1. `attacks.jsma` always raises โ€” implemented incorrectly -`attacks.py:325-376` - -* `saliency_map = np.zeros(input_shape)` with `input_shape` a `TensorShape`; loops `range(input_shape[1..3])` assume a hard-coded 4-D image input. -* `jacobian[0, i, j, k, :]` indexes a rank-4 tensor with 5 indices. -* **Measured:** `InvalidArgumentError: Index out of range using input dim 4; input has only 4 dimensions`. -* `adversarial_example[...] += gamma` mutates an `EagerTensor` in place โ€” illegal even if the above were fixed. -* Conceptually it is not JSMA: it sums the whole Jacobian instead of computing the *saliency map* over (target, other-class) derivative pairs, and applies a fixed `gamma` instead of the `theta` pixel budget. - -**Verdict:** unusable. Needs a rewrite. - -### C2. `attacks.spsa` always raises; hard-codes rank-4 inputs -`attacks.py:378-430` - -* `perturbation = tf.zeros_like(x)` is a **tensor, not a `tf.Variable`**, yet it is passed to `optimizer.apply_gradients(...)` and later `.assign(...)` โ€” both require a Variable. -* `tf.tile(x, [spsa_samples, 1, 1, 1])` hard-codes 4 dimensions โ€” **measured** failure on a `(1, 10)` Dense input, contradicting the README's "Numeric โœ…" matrix entry. -* `y_tiled = tf.tile(y, [128])` produces rank-2 labels for `sparse_categorical_crossentropy` โ€” **measured** failure even on a valid 4-D batch. -* The gradient estimate `mean(loss * delta_x) / delta` is not SPSA: it needs the loss *difference* `(L(x+ฮด) โˆ’ L(xโˆ’ฮด)) / (2ฮด)`, otherwise the absolute loss magnitude dominates the estimate. -* `adversarial_example` is unbound if `num_steps <= 0` โ†’ `UnboundLocalError`. - -### C3. `defenses.pixel_deflection` always raises -`defenses.py:405-451` - -* **Measured:** `ValueError: maxval must be a scalar; got a tensor of shape [2]` from `tf.random.uniform([count, 2], 0, [h, w], ...)`. Any Functional-model construction traces this, so it can never work. -* Also fails inside `tf.function` (no `fn_output_signature`), and duplicate scatter indices silently drop updates. -* The in-code comments admit it is "placeholder logic" and "an approximation". - -### C4. `attacks.ead` is a **complete no-op** with default arguments -`attacks.py:101-145` - -Default `beta=0.01` and `alpha=0.01` give `maximum(|diff| - beta, 0)` where `|diff| == alpha == 0.01` โ†’ **exactly zero**. All `num_steps` are spent producing the input unchanged. - -* **Measured:** `max|ฮด| = 0.0000`, `mean|ฮด| = 0.0000` over 10 steps at `eps=0.02`. -* It is also not EAD: no L2 term, no per-step adversarial objective, no early stop โ€” it is PGD with a shrink applied *after* the update and then clipped, which partially cancels it. - -**Verdict:** a user running EAD concludes their model is already robust. This is the single most dangerous defect in the package. - -### C5. `attacks.cw` โ€” `c` and `kappa` are dead parameters -`attacks.py:252-292` - -`loss + c * tf.norm(x - tf.clip_by_value(x, 0, 1))**2 - kappa` โ€” `x` has already been clipped to `[0,1]` on every previous iteration, so the L2 term is **identically 0** and `kappa` adds a constant whose gradient is **identically 0**. - -* **Measured:** `cw(c=1, kappa=0)` and `cw(c=1000, kappa=50)` produce **bit-identical** output. -* No tanh-space optimisation, no L2 minimisation, no confidence margin (`- kappa` has the wrong sign for C&W anyway), FGSM-style `alpha*sign(grad)` step. - -**Verdict:** it is PGD with two decorative arguments. A C&W benchmark built on this is meaningless. - -### C6. `attacks.deepfool` is unbounded and returns invalid inputs -`attacks.py:294-323` - -* Adds `gradient / ||gradient||` with step size **1.0** and never clips or projects. -* **Measured:** output range `[-0.900, 2.080]` for an input in `[0,1]` โ€” a completely invalid image. -* Has no `epsilon` parameter at all, so the "minimum perturbation" property that defines DeepFool cannot be expressed. -* No per-class linearisation; it is gradient ascent in the loss direction, which is closer to an *anti*-attack than DeepFool. - -### C7. `attacks.fgsm` returns out-of-range data -`attacks.py:21-53` - -* No clipping at all (unlike PGD/BIM/MIM). -* **Measured:** `min=-0.010, max=1.007` for `x โˆˆ [0,1]`, `epsilon=0.01`. Feeding that back into the model gives garbage and it is not a valid image. -* No `gradient is None` handling โ†’ `tf.sign(None)` raises a confusing `AttributeError`. - -### C8. Every attack rejects `sparse_categorical` integer labels -`attacks.py` โ€” all of `fgsm, pgd, bim, mim, cw, ead, deepfool` - -* **Measured:** all seven raise `Arguments target and output must have the same rank (ndim). Received target.shape=(4,), output.shape=(4, 3)`. -* Sparse integer labels are *the* default for MNIST/CIFAR examples in the Keras documentation, i.e. the package's primary audience. -* `pgd`/`bim`/`cw`/`deepfool` hard-code `CategoricalCrossentropy`; `fgsm`/`mim`/`ead` use a fragile heuristic (`y.shape[-1] == 1 or len(y.shape) == 1`) that misclassifies a 1-D integer label tensor as a *binary* target. -* `from_logits` is never exposed, so any model not ending in `softmax` is silently attacked with the wrong loss. - -### C9. Every wrapped defense model is unsaveable -`defenses.py` โ€” all 11 preprocessing wrappers - -Every layer class (`SqueezeLayer`, `NoiseLayer`, `ThermometerLayer`, `JPEGCompressLayer`, `DeflectionLayer`, `BlurLayer`, `TVLayer`, `MedianLayer`, `SmoothLayer`, `DenoiseLayer`, `TransformationLayer`) is **defined inside the function body**, so it has no importable qualified name and no `get_config()`. - -* **Measured:** `model.save(...)` โ†’ `TypeError: could not be deserialized properly` for `feature_squeezing`, `gaussian_blur`, `randomized_smoothing` (3/3). -* `tf.keras.models.clone_model()` on a defended model fails for the same reason. -* This makes the defenses unusable in production: you cannot train today and serve tomorrow. - -### C10. `jpeg_compression` destroys the gradient โ€” attacks on the defended model crash -`defenses.py:396-397` - -`tf.image.encode_jpeg`/`decode_jpeg` are non-differentiable. - -* **Measured:** `tape.gradient(...)` โ†’ `None`. -* **Measured:** `attacks.fgsm(jpeg_compression(model), x, y)` โ†’ `ValueError: Attempt to convert a value (None)`. -* The library's headline use case โ€” "defend, then evaluate the defense with an attack" โ€” therefore fails. Needs a straight-through-estimator wrapper. -* Also hard-codes `channels=3` (breaks grayscale), assumes `x โˆˆ [0,1]`, and cannot run on GPU/TPU. - -### C11. `gradient_masking` provides zero protection at inference -`defenses.py:93-129` - -It masks gradients **during training only**. An attacker queries the model's own gradients, which are untouched. - -* **Measured:** `fgsm(base_model) == fgsm(gradient_masking(base_model))` โ€” bit-identical. -* The docstring claims it "makes them less informative for adversarial attackers", which is false. Gradient masking is a known *false* defense in the literature (Obfuscated Gradients, Athalye et al. 2018). It must be relabelled as a training regulariser, and the README's โœ… row should carry a warning. - -### C12. `median_smoothing` is an average pool โ€” not a median filter -`defenses.py:552-560` - -* **Measured:** output is bit-identical to `tf.nn.avg_pool2d(x, 3, 1, "SAME")`. -* Ironically a real median implementation already exists in `spatial_smoothing` (line 347). This is a copy-paste regression. - -### C13. `gaussian_blur` ignores `sigma` entirely -`defenses.py:467-476` - -* Never builds a Gaussian kernel; calls `avg_pool2d`. -* **Measured:** `gaussian_blur(sigma=0.1)` โ‰ก `gaussian_blur(sigma=9.0)`. - -### C14. `total_variation_minimization` silently kills attack gradients -`defenses.py:503-514` - -* Runs an inner `GradientTape` inside `Layer.call`; the outer tape records the inner optimisation, so the composed gradient w.r.t. the input is **measured as all-zero**. -* Consequence: `fgsm(tv_defended_model, ...)` returns an example that is *not* adversarial, with no error โ€” exactly the silent failure mode this report is about. -* Also un-trainable, adds second-order cost, and cannot be used in a `tf.function` training loop. - ---- - -## 2. HIGH โ€” wrong results, bad defaults, unsafe API - -### H1. `adversarial_training` is crippled and ignores the caller's configuration -`defenses.py:28-60` - -* **Per-sample loop** (`for i in range(len(x))`): one forward+backward pass per image. **Measured: 3.08 s for 200 images** โ†’ ~13 minutes for 50 000 MNIST samples, before training starts. -* Hard-codes `epochs=10`, `batch_size=32`, `optimizer='adam'`, `loss='categorical_crossentropy'`. - **Measured:** a model compiled with `SGD(0.001)` + `KLDivergence` + `mae` comes back with **Adam + categorical_crossentropy + no `mae`** โ€” the user's compile config is silently discarded. -* **Measured:** fails on sparse labels (`target.shape=(1,)`). -* No `verbose`, no `shuffle`, no `validation_data`, no early stop; refuses lists/`tf.data.Dataset` (`len()`/slicing). - -### H2. No input validation anywhere โ†’ NaN or crashes from legal-looking arguments -**Measured:** - -| Call | Result | -|---|---| -| `fgsm(..., epsilon=-1)` | accepted, returns values as low as `-0.9997` | -| `pgd(..., num_steps=0)` | accepted, returns the input silently | -| `feature_squeezing(model, bit_depth=0)` | **NaN** (division by `2**0 - 1 == 0`) | -| `feature_squeezing(model, bit_depth=9)` | `TypeError` (floordiv int32 by `0.5`) | -| `thermometer_encoding(model, num_bins=0)` | **NaN** | -| `spsa(..., num_steps=0)` | `UnboundLocalError` | - -A security library must not emit NaN. Every numeric parameter needs a documented range and a `ValueError`. - -### H3. `feature_squeezing` quantisation is subtly biased and non-differentiable -`defenses.py:85` - -For `bit_depth=4`: `floor(x*255/16)/15`. The level index is divided by `2**b - 1` (15) rather than by `255/16`, so the mapping is stretched by ~6% and `x=0.5 โ†’ 0.4667`. Correct is `round(x*(2**b-1))/(2**b-1)` or a `255`-based round-trip. Also `floor`+`cast` โ‡’ **measured** `gradient is None`, so no attack can be run through it (no straight-through estimator). - -### H4. Every preprocessing wrapper breaks on non-Functional models -`infrastructure of all wrappers, e.g. defenses.py:88, 155, 230, 371, 399, 448, 478, 516, 562` - -* **Measured:** subclassed model โ†’ `AttributeError: 'Sub' object has no attribute 'input_shape'`. -* **Measured:** multi-input model โ†’ `ValueError: Cannot convert '[(None, 4)]' to a shape`. - -Both need an explicit `TypeError`/`ValueError` with a clear message, plus support for subclassed models via `model.inputs`. - -### H5. `randomized_smoothing` is not randomized smoothing -`defenses.py:207-233` - -* Adds inference-time Gaussian noise and nothing else: no noise-augmented training, no Monte-Carlo vote over N samples, no certified radius. Inference-time-only noise is a well-documented *broken* defense. -* No clip to `[0,1]` after adding noise; no `training` gate; no seed. -* **Measured:** two identical calls return different outputs (non-deterministic `predict`). - -### H6. `thermometer_encoding` is plain uniform quantisation -`defenses.py:279-283` - -`floor(x*bins)/bins` is quantisation, not thermometer encoding (which expands `C โ†’ Cร—L` binary features and requires retraining the first layer). Claims in the docstring and README are false. `num_bins=0` โ†’ NaN. - -### H7. `defensive_distillation` applies softmax twice and drops `Tยฒ` -`defenses.py:182-200` - -* `tf.nn.softmax(teacher_predictions / T)` on a model that already ends in `softmax` โ†’ softmax of softmax, then `CategoricalCrossentropy(from_logits=False)` on that. Mathematically not distillation. -* Teacher is called **without `training=False`**, so its BN/dropout is live and the soft targets are noisy. -* Missing the `Tยฒ` scaling factor and the KL formulation; the typical practice is CE against the same temperature-softened distribution. - -### H8. `adversarial_logit_pairing` does not implement ALP -`defenses.py:308-321` - -Both models are called on the **same clean batch** `x`; the pairing loss therefore just forces two models to agree on clean data. ALP requires generating an adversarial example inside the train step and pairing `logits(x)` with `logits(x_adv)`. - -### H9. Packaging and release pipeline (supply chain) - -* **`pip install .` from a checkout FAILS.** `setup.py` ships `version='{{VERSION_PLACEHOLDER}}'`; **measured** `pip install .` โ†’ `Encountered error while generating package metadata`. Only the tag-time `sed` in CI produces a valid version, so local builds, editable installs, and `python -m build` are all broken. Replace with a real static version (or `setuptools_scm` / `importlib.metadata`). -* **`pypa/gh-action-pypi-publish@master` and `actions/checkout@master`** โ€” mutable refs with `PYPI_API_TOKEN` in the environment: a compromise or force-push of the action branch means arbitrary code executes with publishing rights. Pin every action to a commit SHA. -* **`echo ::set-output`** is removed on modern runners โ†’ the version step is broken/warned. -* No `permissions:` block (defaults are broad), no `environment: pypi` protection rule, no tag pattern guard โ†’ **any** tag publishes. -* Long-lived `PYPI_API_TOKEN` secret instead of [PyPI Trusted Publishing] (OIDC `id-token: write`). -* No test/lint step before publish; uses deprecated `python setup.py sdist bdist_wheel` instead of `python -m build`; no `twine check`. -* `install_requires=['numpy','tensorflow']` is fully unbounded, so a future `numpy 3` / `tensorflow 3` breaks the package on install. `tensorflow` is unavailable on many platforms; consider a `tensorflow-cpu` extra. -* Classifiers claim Python **3.6** and `python_requires='>=3.6'`, but TF 2.x requires โ‰ฅ3.9 โ€” the package is installed on interpreters where it cannot import. -* No `license=`/`license_files=`, yet `License :: OSI Approved :: MIT License` is declared while `LICENSE` is "MIT License (Modified)" (not OSI-approved, no SPDX id). Remove the classifier or relicense. -* `Development Status :: 5 - Production/Stable` is not defensible given the above. -* No `pyproject.toml` (PEP 517/518), no `MANIFEST.in`, no `py.typed`, no extras for tests. - -### H10. Public API is incomplete; no `__version__` -`__init__.py` - -* **Measured MISSING:** `deepdefend.mim`, `.ead`, `.word_swap`, `.char_swap`, `.pixel_deflection`, `.gaussian_blur`, `.word_masking`, `.median_smoothing`, `.__version__`. All of these exist in the submodules but are not re-exported, and there is no `__all__`. -* No `__version__` at all makes bug reports unactionable. - ---- - -## 3. MEDIUM โ€” maintainability, fidelity, performance - -* **M1. Duplicated bodies:** `pgd` (188-219) and `bim` (220-251) are byte-identical; `fgsm`/`pgd`/`bim`/`cw`/`deepfool`/`mim`/`ead` each re-implement loss selection and projection. The 11 preprocessing wrappers repeat the same 4-line `Input โ†’ Layer โ†’ model โ†’ Model` block. Extract `_utils` + `_wrap()`. -* **M2. Local layer classes** have no `get_config()`/`compute_output_shape()`, blocking serialization even if moved to module level. -* **M3. Fixed `[0,1]` assumption everywhere.** No `clip_min`/`clip_max` on any attack; real models are fed ImageNet-normalised tensors, where clipping to `[0,1]` silently destroys the input. -* **M4. No PGD `random_start`**, no early stopping on misclassification, no `@tf.function` anywhere (each step is an eager op, ~5-50ร— slower than necessary). -* **M5. Text helpers destroy whitespace.** `" ".join(text.split())` collapses runs of spaces, tabs and newlines. **Measured:** `word_masking("a b\nc", mask_prob=1.0)` โ†’ `"[MASK] [MASK] [MASK]"`; `char_swap("line one\nline two", 1.0)` โ†’ `"lnie noe lien wto"` (newline gone). Tokenise with `re` and preserve separators. -* **M6. Non-reproducible randomness:** `np.random.*` globals in `char_swap`/`word_masking`, `tf.random` without seeds in `randomized_smoothing`/`pixel_deflection`. No `seed` parameter on any function. -* **M7. `spatial_smoothing`** allocates `Bร—Hร—Wร—kยฒร—C` (81ร— the input for `k=9`); mislabelled (it *is* a median filter, not spatial smoothing); crashes on non-4-D input. -* **M8. `input_transformation()` with its default `transformation_function=None` is a silent identity** โ€” returns a model with no defense, named as if defended. -* **M9. Documentation is materially inaccurate:** the support matrix marks SPSA "Numeric โœ…" (fails, H2), and PGD/EAD/DeepFool "Text โœ… (Embeddings)" (unverified); the 0.1.5 changelog claims "Fixed logical errors in several defense functions" while C4/C5/C12/C13 remain; the Python-3.6 claim is impossible with the TF dependency. -* **M10. `attacks.py` calls `model(x)` without `training=False`.** In Keras 3 this defaults to inference, so the impact is currently low (**measured:** deterministic on a dropout model), but the behaviour of `training=None` is context-dependent (inside `fit`'s `train_step`, and in older TF versions, it can be `True`). Pass it explicitly. -* **M11. Circular/pointless imports:** `defenses.py:26` does `from deepdefend import attacks`; `__init__.py:1` does `import deepdefend` inside the package's own `__init__`. Both work only by accident. Note that `import deepdefend` also eagerly pulls in TensorFlow (~5 s, ~600 MB RSS for any consumer, even one that only wants `word_swap`). -* **M12. Zero tests, zero CI checks.** `CONTRIBUTING.md` instructs contributors to "run tests" โ€” there are none, and no workflow runs anything on push/PR. No `ruff`/`flake8`, no pre-commit, no Dependabot, no `SECURITY.md` (notable for a security-focused project), no `CODE_OF_CONDUCT.md`, no `pyproject.toml`. - ---- - -## 4. Function-by-function verdict - -### `deepdefend.attacks` -| Function | Status | Key defect | -|---|---|---| -| `fgsm` | โš ๏ธ wrong | unclipped output (C7); binary/categorical heuristic; no sparse labels | -| `pgd` | โš ๏ธ weak | no `random_start`, no `clip_min/max`, no sparse labels, dup of `bim` | -| `bim` | โš ๏ธ weak | byte-identical to `pgd` | -| `mim` | โš ๏ธ weak | L1-norm is over the whole batch (cross-sample coupling), same as PGD otherwise | -| `cw` | โŒ | `c`/`kappa` inert, not C&W (C5) | -| `ead` | โŒ **no-op** | default params โ‡’ zero perturbation (C4) | -| `deepfool` | โŒ | unbounded, no ฮต, invalid images (C6) | -| `jsma` | โŒ crash | always raises (C1) | -| `spsa` | โŒ crash | not a Variable, rank-4 only, bad estimator (C2) | -| `word_swap` | โš ๏ธ | whitespace loss, silent no-op on `swap_dict=None` | -| `char_swap` | โš ๏ธ | whitespace loss, unseeded, `swap_prob` unvalidated | - -### `deepdefend.defenses` -| Function | Status | Key defect | -|---|---|---| -| `adversarial_training` | โš ๏ธ | per-sample loop, hard-coded compile config (H1) | -| `feature_squeezing` | โš ๏ธ | biased quantiser, kills gradient, `bit_depth` unvalidated (H3) | -| `gradient_masking` | โŒ **no-op** | zero inference-time effect (C11) | -| `input_transformation` | โš ๏ธ | default is a silent identity (M8) | -| `defensive_distillation` | โš ๏ธ wrong | double softmax, no `Tยฒ`, teacher in train mode (H7) | -| `randomized_smoothing` | โŒ wrong | inference-only noise, no certification (H5) | -| `feature_denoising` | โš ๏ธ | an average pool, duplicate of blur | -| `thermometer_encoding` | โŒ wrong | plain quantisation (H6) | -| `adversarial_logit_pairing` | โŒ wrong | pairs on clean inputs only (H8) | -| `spatial_smoothing` | โš ๏ธ | actually a median filter; memory blow-up (M7) | -| `jpeg_compression` | โŒ | breaks gradient; grayscale unsupported (C10) | -| `pixel_deflection` | โŒ crash | `tf.random.uniform` misuse (C3) | -| `gaussian_blur` | โŒ wrong | `sigma` unused, is avg-pool (C13) | -| `total_variation_minimization` | โŒ wrong | zero gradient through, nested tape (C14) | -| `word_masking` | โš ๏ธ | whitespace loss, unseeded | -| `median_smoothing` | โŒ wrong | is avg-pool (C12) | - ---- - -## 5. Proposed remediation plan - -**Phase 0 โ€” guardrails first (so the fixes can be proven)** -1. `tests/` with a tiny deterministic CNN fixture; pytest. -2. `.github/workflows/ci.yml`: pytest on 3.10โ€“3.12 + `ruff check` on every push/PR. -3. `pyproject.toml`: real version `0.2.0`, bounded deps, `[project.optional-dependencies] dev`, ruff/pytest config. -4. `SECURITY.md`, `CODE_OF_CONDUCT.md`, `.pre-commit-config.yaml`, Dependabot. - -**Phase 1 โ€” attack correctness** -5. New `deepdefend/_utils.py`: `_prepare_labels` (one-hot **and** sparse **and** binary), `_loss_fn(model, x, y, from_logits)`, `_project(adv, x, eps, clip_min, clip_max)`, `_validate_*`, `resolve_input_range(model, x)`. -6. Rewrite `fgsm/pgd/bim/mim`: batched, `training=False`, `clip_min/clip_max`, `from_logits`, `random_start` (PGD), sparse labels, `None`-gradient guard. -7. Rewrite `cw` (tanh space, L2 objective, working `c`/`kappa`), `deepfool` (per-class boundaries + ฮต bound), `jsma` (real saliency, vectorised update), `spsa` (Variable + proper finite-difference estimator, rank-agnostic), `ead` (real elastic-net + non-degenerate defaults). -8. Text attacks: `re`-based tokenisation preserving whitespace, `seed` parameter, validated `swap_prob`. - -**Phase 2 โ€” defense correctness** -9. Hoist every layer to module level with `get_config()`/`compute_output_shape()`; add a shared `_wrap_preprocessing()`; raise clear errors for subclassed/multi-input models. **Add a saveโ†’load round-trip test for all wrappers.** -10. `adversarial_training`: batched attack generation, honour `model.optimizer`/`model.loss`, accept `epochs`/`batch_size`/`verbose`. -11. Real median filter for `median_smoothing`; real separable Gaussian for `gaussian_blur` (use `sigma`); de-duplicate `feature_denoising`; make `spatial_smoothing` honest about what it is and memory-safe. -12. Straight-through gradients for `jpeg_compression`, `feature_squeezing` and `total_variation_minimization` (replace the nested tape), so attacks against defended models work โ€” plus an explicit `differentiable=True/False` flag and a test that `grad is not None`. -13. `randomized_smoothing`: `training` gate, clip, seed, and a `predict_smoothed(n_samples)` + certified-accuracy helper; document that the base model must be noise-trained. -14. `thermometer_encoding`: real `C โ†’ Cร—L` expansion (with a documented retraining requirement) or rename to `uniform_quantization`. -15. `defensive_distillation` (logits + `Tยฒ` + KL + `training=False` teacher), `adversarial_logit_pairing` (generate adversarial examples inside the step), `gradient_masking` (keep, but relabel + docstring/README warning that it is not a security defense). -16. `pixel_deflection`: scalar-maxval fix, `fn_output_signature`, unique-index scatter, seed. -17. Validation pass: every numeric arg gets a documented range and a `ValueError` (kills the NaN paths in H2). -18. `input_transformation`: error or documented identity default instead of a silent no-op. - -**Phase 3 โ€” API consistency** -19. `__init__.py`: `__all__`, `__version__`, export everything, remove the self-import, fix the circular import, lazy TF import. -20. Uniform contract: attacks return `np.ndarray` in the input dtype; defenses return a `keras.Model`; text helpers return `str`. Add `seed` everywhere random. - -**Phase 4 โ€” docs, packaging, release** -21. Rewrite the README support matrix from measured behaviour; add per-defense "limitations / not a guarantee" notes and a "dual-use / responsible disclosure" section. -22. Fix `setup.py`/`pyproject.toml` metadata (version, bounded deps, `python_requires>=3.9`, license field, extras). -23. Rewrite the publish workflow: SHA-pinned actions, `permissions: contents: read` + `id-token: write`, `environment: pypi`, Trusted Publishing (no long-lived token), `$GITHUB_OUTPUT`, tag-pattern guard, tests before publish, `python -m build` + `twine check`. - -**Compatibility note:** items 7, 9โ€“17 change numerical output by design. Recommend releasing as **0.2.0** with a `CHANGELOG.md` "Behaviour changes" section rather than pretending to be a patch release. - ---- - -## Appendix โ€” measured evidence (TF 2.16.2 / Keras 3.15 / Python 3.11) - -``` -=== ATTACKS (x in [0,1]) === -[OK] fgsm shape=(4,8,8,3) min=-0.010 max=1.007 <-- outside [0,1] -[FAIL] jsma InvalidArgumentError: Index out of range using input dim 4 -[FAIL] spsa InvalidArgumentError: Tile ... (also fails with batch_size=1) -[OK] deepfool shape=(4,8,8,3) min=-0.900 max=2.080 <-- invalid images - -=== epsilon bound, eps=0.02 === - fgsm max|delta|=0.0200 in[0,1]=False - deepfool FAIL TypeError - ead max|delta|=0.0000 mean|delta|=0.0000 <-- no-op - ---- cw: do c / kappa change anything? --- - cw c/kappa have no effect: True - ---- sparse (integer) labels --- - fgsm / pgd / bim / cw / mim / ead / deepfool -> all FAIL: - "target and output must have the same rank (ndim)" - ---- gradient flow through defenses --- - jpeg_compression grad_present=False <-- attacks impossible - total_variation_minimization grad_present=True grad_nonzero=False - feature_squeezing grad_present=False - ---- serialization --- - feature_squeezing / gaussian_blur / randomized_smoothing -> save+load FAIL - "could not be deserialized properly" - ---- equivalence / dead-parameter checks --- - gaussian_blur(sigma=0.1) == gaussian_blur(sigma=9): True <-- sigma unused - median_smoothing == avg_pool2d: True <-- not a median - fgsm(base) == fgsm(gradient_masking(base)): True <-- no defense - feature_squeezing bit_depth=0 -> NaN: True - thermometer num_bins=0 -> NaN: True - pixel_deflection -> ValueError: maxval must be a scalar - ---- misc --- - fgsm(epsilon=-1) accepted, min=-0.9997 - adversarial_training(200 samples) = 3.08 s (~13 min extrapolated for MNIST) - model compiled with SGD(0.001)/KLDivergence -> returned with Adam/categorical_crossentropy - deepdefend.mim / .ead / .pixel_deflection / .gaussian_blur / .median_smoothing / .__version__ -> MISSING - pip install . -> "Encountered error while generating package metadata" - jsma on subclassed/multi-input models -> AttributeError / ValueError -``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f2b36a0..618badc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -18,7 +18,7 @@ Thank you for considering contributing to **DeepDefend**! Contributions, whether Before and after making changes, run the test-suite and the linter: ```bash - pytest # 171 tests covering attacks, defenses and the public API + pytest # 170 tests covering attacks, defenses and the public API ruff check . # lint ``` diff --git a/deepdefend/__init__.py b/deepdefend/__init__.py index 8dd28a7..6ec5b70 100644 --- a/deepdefend/__init__.py +++ b/deepdefend/__init__.py @@ -47,7 +47,6 @@ "feature_denoising", "feature_squeezing", "gaussian_blur", - "gradient_masking", "input_transformation", "jpeg_compression", "median_smoothing", diff --git a/deepdefend/defenses.py b/deepdefend/defenses.py index 41c9504..5ba57da 100644 --- a/deepdefend/defenses.py +++ b/deepdefend/defenses.py @@ -14,7 +14,6 @@ Available functions: - `adversarial_training(model, x, y, epsilon=0.01)`: Adversarial Training. - `feature_squeezing(model, bit_depth=4)`: Feature Squeezing. -- `gradient_masking(model, mask_threshold=0.1)`: Gradient Masking (training regulariser, *not* a defense). - `input_transformation(model, transformation_function)`: Input Transformation. - `defensive_distillation(model, teacher_model, temperature=2)`: Defensive Distillation. - `randomized_smoothing(model, noise_level=0.1)`: Randomized Smoothing (inference-time noise). @@ -34,7 +33,6 @@ from __future__ import annotations import math -import warnings import numpy as np import tensorflow as tf @@ -56,7 +54,6 @@ "feature_denoising", "feature_squeezing", "gaussian_blur", - "gradient_masking", "input_transformation", "jpeg_compression", "median_smoothing", @@ -582,85 +579,6 @@ def adversarial_training(model, x, y, epsilon=0.01, epochs=10, batch_size=32, return defended_model -@tf.keras.utils.register_keras_serializable(package="deepdefend") -class GradientMaskingModel(tf.keras.Model): - """Training wrapper that drops gradients below ``threshold``. - - .. warning:: - This is a *training regulariser*, not an adversarial defense. It does - not alter the gradients an attacker computes at inference time. - """ - - def __init__(self, base_model=None, mask_threshold=0.1, **kwargs): - super().__init__(**kwargs) - self.base_model = base_model - self.mask_threshold = mask_threshold - - def call(self, inputs, training=None): - return self.base_model(inputs, training=training) - - def train_step(self, data): - x, y = data[0], data[1] - with tf.GradientTape() as tape: - predictions = self.base_model(x, training=True) - loss = self.compiled_loss(y, predictions) - - trainable = self.base_model.trainable_variables - gradients = tape.gradient(loss, trainable) - pairs = [ - (tf.where(tf.abs(g) > self.mask_threshold, g, tf.zeros_like(g)), v) - for g, v in zip(gradients, trainable) if g is not None - ] - self.optimizer.apply_gradients(pairs) - self.compiled_metrics.update_state(y, predictions) - return {m.name: m.result() for m in self.metrics} - - def get_config(self): - config = super().get_config() - config.update( - base_model=tf.keras.utils.serialize_keras_object(self.base_model), - mask_threshold=self.mask_threshold, - ) - return config - - @classmethod - def from_config(cls, config): - config = dict(config) - config["base_model"] = tf.keras.utils.deserialize_keras_object(config["base_model"]) - return cls(**config) - - -def gradient_masking(model, mask_threshold=0.1): - """ - Gradient Masking "defense". - - Zeroes training gradients smaller than ``mask_threshold``. - - .. warning:: - Gradient masking is a well documented **false** defense (Athalye et al., - "Obfuscated Gradients Give a False Sense of Security", 2018). It has no - effect on the gradients an attacker sees, and it can make a model look - robust while remaining vulnerable. Use it only as a training - regulariser, never as evidence of robustness. - - Parameters: - model (tensorflow.keras.Model): The model to wrap. - mask_threshold (float): Threshold below which gradients are zeroed. - - Returns: - defended_model (tensorflow.keras.Model): The wrapped model. - """ - mask_threshold = check_float("mask_threshold", mask_threshold, minimum=0.0) - warnings.warn( - "gradient_masking() does not defend against inference-time attacks; " - "it only modifies training gradients. Do not treat it as a robustness " - "measure.", - RuntimeWarning, - stacklevel=2, - ) - return GradientMaskingModel(base_model=model, mask_threshold=mask_threshold) - - def _temperature_softmax(outputs, temperature, from_logits): """Temperature-scaled softmax that works for logits and probabilities.""" if from_logits: diff --git a/readme.md b/readme.md index 1173a10..051f4e9 100644 --- a/readme.md +++ b/readme.md @@ -37,13 +37,25 @@ either crashed on every call or returned silently wrong results. See - Fixed the release pipeline: SHA-pinned actions, a tag/version check, a test gate, and PyPI Trusted Publishing instead of a long-lived API token. **Added** โ€” `deepdefend.__version__`, lazy imports, `smooth_predict()` with a -certified-radius estimate, a 171-test suite, CI, `SECURITY.md`, +certified-radius estimate, a 170-test suite, CI, `SECURITY.md`, `CODE_OF_CONDUCT.md`, Dependabot, pre-commit and `pyproject.toml`. +**Removed โ€” `gradient_masking`.** The function has been deleted from the +package. It only altered *training* gradients, never the gradients an attacker +computes at inference time, so it had no measurable defensive effect (we +verified that `fgsm(model)` and `fgsm(gradient_masking(model))` produced +bit-identical adversarial examples). Gradient masking is a well documented +**false defense** โ€” it makes a model *look* robust while leaving it vulnerable, +which is exactly the failure mode a robustness tool must not encourage. If you +need it, mask gradients in your own `train_step`; do not report it as a security +measure. Everything else about that class of techniques is covered in +[Limitations](#limitations). + **Breaking** โ€” the numerical output of `cw`, `deepfool`, `ead`, `spsa` and `jsma` changed (they are now the algorithms they claim to be), several defaults -changed, and image-only defenses now raise on non-image inputs instead of -silently returning the input. See [CHANGELOG.md](CHANGELOG.md). +changed, `gradient_masking` was removed, and image-only defenses now raise on +non-image inputs instead of silently returning the input. See +[CHANGELOG.md](CHANGELOG.md). ## Installation @@ -94,7 +106,6 @@ Legend: โœ… supported ยท โŒ unsupported (raises a clear error) | **Defenses** | | | | | | Adversarial Training | โœ… | โœ… | โœ… | โœ… | | Feature Squeezing | โœ… | โœ… | โœ… | โœ… | -| Gradient Masking (โš ๏ธ not a defense) | โœ… | โœ… | โœ… | โœ… | | Input Transformation | โœ… | โœ… | โœ… | โœ… | | Defensive Distillation | โœ… | โœ… | โœ… | โœ… | | Randomized Smoothing | โœ… | โœ… | โœ… | โœ… | @@ -109,6 +120,9 @@ Legend: โœ… supported ยท โŒ unsupported (raises a clear error) | TV Minimization | โœ… | โŒ | โŒ | โœ… | | Word Masking | โŒ | โœ… | โŒ | โ€” | +> `gradient_masking` used to be listed here. It has been removed in 0.2.0 โ€” see +> the note above and [Limitations](#limitations). + ## Usage ### Adversarial Attacks @@ -183,7 +197,6 @@ defended_model = adversarial_training( ) defended_model = defensive_distillation(model, teacher_model, temperature=2) defended_model = adversarial_logit_pairing(model, epsilon=0.01) -defended_model = gradient_masking(model, mask_threshold=0.1) # โš ๏ธ see below ``` ```python @@ -218,11 +231,15 @@ defended_text = word_masking("The movie was great", mask_prob=0.2) Adversarial defenses are an active research area, and most preprocessing defenses can be broken by an adaptive attacker. Do not treat a defended model as -certified robust unless you have measured it: - -- **Gradient masking is not a defense.** It only alters training gradients. It - has no effect on the gradients an attacker sees, and `gradient_masking()` emits - a `RuntimeWarning` for this reason. +certified robust unless you have measured it. + +- **There is no gradient-masking defense in this package, by design.** Zeroing + or obfuscating gradients during training does not change the gradients an + attacker sees; it only produces a false sense of security (Athalye et al., + "Obfuscated Gradients Give a False Sense of Security", 2018). The old + `gradient_masking()` helper was removed in 0.2.0 for that reason. If you want + gradient regularisation as a *training* technique, implement it in your own + `train_step` โ€” and never report it as robustness. - **Randomized smoothing** only provides a certified radius if the base model is trained on noise-augmented data and predictions are averaged over many samples (`smooth_predict`). Adding noise at inference time alone does not make @@ -231,10 +248,277 @@ certified robust unless you have measured it: Canonical thermometer encoding expands each feature into `num_bins` binary features, which requires retraining the first layer. - **Preprocessing defenses** (blur, JPEG, squeezing, denoising, TV, deflection) - are often defeated by a small step-size attack (Athalye et al., 2018). Always - evaluate them with a strong, adaptive attack (PGD or CW with several restarts). + are often defeated by a small step-size attack. Always evaluate them with a + strong, adaptive attack (PGD or CW with several restarts). - **Text helpers** (`word_swap`, `char_swap`, `word_masking`) are simple baselines, not semantically-aware attacks. +- **Iterative attacks stop early.** A sample the model already misclassifies is + returned unchanged (for targeted attacks, a sample already classified as the + target is returned unchanged). +- **Inputs are not silently clipped.** If your data does not lie in `[0, 1]`, + pass `clip_min` / `clip_max`; otherwise attacks will not clip at all rather + than destroy your normalised inputs. + +## What was wrong before 0.2.0 + +0.2.0 fixed defects that made earlier versions actively misleading. This section +is kept so that results produced with 0.1.x can be interpreted correctly, and so +that the reasons behind the API changes are documented. + +Every entry below was reproduced by executing the released code against a real +Keras model, not inferred from reading it. + +### Attacks + +| Function | What it did in 0.1.x | Consequence | +|---|---|---| +| `jsma` | Indexed a rank-4 Jacobian with five indices and mutated an eager tensor in place | Raised on **every** call | +| `spsa` | Passed a tensor (not a `tf.Variable`) to the optimiser and hard-coded rank-4 inputs | Raised on **every** call | +| `ead` | The default `alpha == beta` made the L1 shrinkage cancel every step | Returned the input **unchanged** (`max\|ฮด\| = 0.0000`) โ€” "model is robust" | +| `cw` | Clipped the input before measuring `โ€–x โˆ’ clip(x)โ€–ยฒ`, so the L2 term was always 0 | `c` and `kappa` had **no effect**; not C&W | +| `deepfool` | Added an unnormalised unit step with no budget and no clipping | Returned invalid images in `[-0.90, 2.08]` | +| `fgsm` | Did not clip the output at all | Returned values such as `-0.010` / `1.007` | +| all attacks | Used `CategoricalCrossentropy` only | Rejected sparse integer labels (the default in Keras tutorials) | +| all attacks | Never exposed `from_logits` | Attacked logit models with the wrong loss | +| targeted attacks | Inverted the early-stop mask | Froze at step 0, producing no perturbation | +| `word_swap`, `char_swap` | Round-tripped through `str.split()` / `" ".join()` | Destroyed double spaces, tabs and newlines | + +### Defenses + +| Function | What it did in 0.1.x | Consequence | +|---|---|---| +| all preprocessing defenses | Defined their layer classes inside the function body | **No defended model could be saved or reloaded** | +| `jpeg_compression` | Relied on non-differentiable JPEG decode | `gradient is None`; attacking the defended model raised `ValueError` | +| `total_variation_minimization` | Nested `GradientTape` inside `Layer.call` | Gradient through the defense was **all zeros** | +| `feature_squeezing` | `floor`/`cast` with no straight-through gradient | No gradient; attacks silently did nothing | +| `median_smoothing` | Called `avg_pool2d` | Was not a median filter at all | +| `gaussian_blur` | Called `avg_pool2d` and ignored `sigma` | `sigma=0.1` and `sigma=9.0` were bit-identical | +| `pixel_deflection` | `tf.random.uniform` with a non-scalar `maxval` | Raised on **every** call | +| `gradient_masking` | Masked training gradients only | Bit-identical to having no defense; **removed in 0.2.0** | +| `adversarial_training` | Generated examples one sample at a time with fixed `Adam` + categorical CE | ~500ร— more model calls; discarded the caller's compile config | +| `defensive_distillation` | Applied `softmax` to an already-softmaxed model, dropped `Tยฒ`, ran the teacher in train mode | Not distillation; noisy soft targets | +| `adversarial_logit_pairing` | Paired two models on the *same clean batch* | Not ALP | +| `randomized_smoothing` | Added unclipped inference-time noise only | No certification; non-deterministic `predict` | +| `feature_squeezing`, `thermometer_encoding` | `bit_depth=0` / `num_bins=0` divided by zero | Returned `NaN` instead of raising | + +### Packaging and CI + +- `pip install .` from a checkout **failed** โ€” `setup.py` shipped + `version='{{VERSION_PLACEHOLDER}}'`. +- Dependencies were unbounded, and `python_requires='>=3.6'` claimed support for + interpreters on which TensorFlow does not exist. +- The publish workflow used `actions/checkout@master` and + `pypa/gh-action-pypi-publish@master` (mutable refs) with a long-lived + `PYPI_API_TOKEN`, the removed `::set-output` command, no tag guard, and no + tests before publishing. +- There were **no tests** and no CI checks at all. + +How the fixes are guarded today: 170 tests (`pytest`), `ruff check .`, a CI +workflow on Python 3.10 โ€“ 3.12, and a release workflow that refuses to publish +when the tag and `pyproject.toml` version disagree or the tests fail. + +## Documentation update brief (for the docs agent) + +The block below is a self-contained brief. It is a code cell so it renders as +plain text: paste it into the docs agent as-is after any API change. + +```text +DOCS AGENT BRIEF โ€” DeepDefend 0.2.0 +=================================== + +CONTEXT +------- +Repository: Infinitode/DeepDefend (PyPI package: deepdefend) +Branch merged: arena/b52ef87b-deepdefend, released as version 0.2.0. +Published docs live at: +https://infinitode-docs.gitbook.io/documentation/package-documentation/deepdefend-package-documentation + +0.2.0 is a correctness release. Several public functions were broken or silently +wrong before it; the API changed accordingly. The docs must describe the 0.2.0 +behaviour only. Do not keep pages that document 0.1.x behaviour, except in the +"Migration from 0.1.x" page described below. + +TASKS +----- +1. Add a "What's new in 0.2.0" page, linking to CHANGELOG.md in the repo. +2. Create a "Migration from 0.1.x" page (breaking changes; see section below). +3. Update every per-function page for the attacks and defenses listed. +4. Remove the gradient_masking page entirely and redirect it to the new + "Gradient masking is not a defense" note under Limitations. +5. Update the supported-model-types table so it matches the README exactly. +6. Update the Python version support statement to 3.9-3.12. +7. Add the "Limitations" and "Responsible use" sections verbatim from the README. +8. Add code samples for every function, using 0.2.0 signatures (see below). + +REMOVED IN 0.2.0 +---------------- +gradient_masking(model, mask_threshold=0.1) + - Deleted from the package. hasattr(deepdefend, "gradient_masking") is False. + - Reason: it only masked training gradients, never the gradients an attacker + computes at inference, so it provided no protection (verified: attacks + against a wrapped model were bit-identical to attacks against the base + model). Gradient masking is a documented false defense (Athalye et al., + 2018, "Obfuscated Gradients Give a False Sense of Security"). + - Docs action: delete the page; state clearly that the technique creates a + false sense of security and that gradient regularisation belongs in the + user's own train_step, never in a robustness claim. + +CHANGED SIGNATURES AND DEFAULTS (attacks) +----------------------------------------- +All attacks additionally accept: from_logits=None, clip_min=None, clip_max=None +and return a NumPy array with the same shape and dtype as the input. + +fgsm(model, x, y, epsilon=0.01, *, targeted=False, from_logits=None, + clip_min=None, clip_max=None) + - Output is clipped to the valid data range (used to be unclipped). + - New: targeted. + +pgd(model, x, y, epsilon=0.01, alpha=None, num_steps=10, *, random_start=True, + seed=None, targeted=False, from_logits=None, clip_min=None, clip_max=None) + - alpha now defaults to epsilon / 4 (used to be a fixed 0.01). + - New: random_start, seed, targeted. + +bim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, *, seed=None, + targeted=False, from_logits=None, clip_min=None, clip_max=None) + - Now documented as PGD without a random start (they used to be identical). + +mim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, decay_factor=1.0, *, + seed=None, targeted=False, from_logits=None, clip_min=None, clip_max=None) + - Momentum is now normalised per sample, not across the batch. + +cw(model, x, y, epsilon=None, c=10.0, kappa=0.0, num_steps=100, alpha=0.01, *, + targeted=False, from_logits=None, clip_min=None, clip_max=None) + - This is now the real Carlini & Wagner L2 attack (tanh-space optimisation of + ||delta||^2 + c * margin). + - epsilon defaults to None (unconstrained). If set, it is an L2 budget + expressed as an L-infinity equivalent: radius = epsilon * sqrt(num_features). + - c and kappa now genuinely change the result; document that c trades + distortion against the margin and should be swept to minimise distortion. + - The untargeted target class is the runner-up class of the original input. + +deepfool(model, x, y, num_steps=50, epsilon=None, overshoot=0.02, *, + from_logits=None, clip_min=None, clip_max=None) + - Now computes per-class decision boundaries and respects epsilon / clip range + (used to return out-of-range images and ignore any budget). + +jsma(model, x, y, theta=0.0, gamma=0.2, num_steps=10, *, targeted=False, + from_logits=None, clip_min=None, clip_max=None) + - theta defaults to 0.0 (always pick the most salient feature); raw saliency + values are products of two gradients and therefore tiny. + +spsa(model, x, y, epsilon=0.01, num_steps=50, learning_rate=0.05, delta=None, + spsa_samples=32, *, seed=None, targeted=False, from_logits=None, + clip_min=None, clip_max=None) + - Accepts any batch size and any input rank. + - delta defaults to epsilon / 5. + +ead(model, x, y, epsilon=0.01, beta=0.001, num_steps=50, alpha=0.01, *, c=0.1, + kappa=0.0, targeted=False, from_logits=None, clip_min=None, clip_max=None) + - beta defaults to 0.001 (0.01 made the default attack a no-op). + +word_swap(text, swap_dict=None, max_swaps=None, seed=None) + - swap_dict is now required (ValueError when missing). + - Whitespace, newlines and punctuation are preserved. + +char_swap(text, swap_prob=0.1, seed=None) + - Whitespace/punctuation preserved; adds seed. + +LABEL HANDLING (document as a shared concept) +--------------------------------------------- +Every attack accepts one-hot (batch, classes), sparse integer (batch,) and +binary (batch, 1) targets. from_logits is auto-detected from the last layer and +can be overridden. clip_min/clip_max are inferred: [0, 1] when the batch already +lies in that range, otherwise no clipping (so normalised inputs are not +destroyed). Iterative attacks stop working on a sample once it reaches the goal, +so already-misclassified samples are returned unchanged. + +NEW FUNCTIONS (defenses) +------------------------ +smooth_predict(model, x, n_samples=100, noise_level=None, seed=None) + - Monte-Carlo prediction for randomized_smoothing models. + - Returns dict with keys: predictions, probabilities, certified_radius. + - certified_radius = sigma * Phi^-1(p_hat) for the majority class, or 0. + - Document that it requires a noise-trained base model. + +FeatureDenoisingBlock(kernel_size=3) + - Keras layer to insert inside a network for feature-level denoising + (learned residual box filter, starts as an identity). + +defenses.FeatureDenoisingBlock is importable as +`from deepdefend.defenses import FeatureDenoisingBlock`. + +CHANGED DEFENSE SIGNATURES +-------------------------- +adversarial_training(model, x, y, epsilon=0.01, epochs=10, batch_size=32, + attack=None, optimizer=None, loss=None, verbose=1, + shuffle=True) + - Generates adversarial examples in batches, keeps the caller's optimiser and + loss, returns a NEW model (a clone). + +defensive_distillation(model, teacher_model, temperature=2) + - Distillation loss is scaled by T^2; the teacher runs in inference mode. + +adversarial_logit_pairing(model, paired_model=None, epsilon=0.01, num_steps=1, + alpha=None) + - Generates adversarial examples inside the training step. + +feature_denoising(model, method="median", kernel_size=3, sigma=1.0) +spatial_smoothing(model, kernel_size=3, method="median", sigma=1.0) +median_smoothing(model, kernel_size=3) # alias of spatial_smoothing(median) +gaussian_blur(model, kernel_size=3, sigma=1.0) # sigma is now used +randomized_smoothing(model, noise_level=0.1, seed=None, + add_noise_at_inference=True) +pixel_deflection(model, deflection_count=100, window_size=10, seed=None) +thermometer_encoding(model, num_bins=10) +word_masking(text, mask_token="[MASK]", mask_prob=0.1, seed=None) +input_transformation(model, transformation_function) # now REQUIRED + +BEHAVIOUR THE DOCS MUST STATE +----------------------------- +1. All preprocessing defenses return a tf.keras.Model that can be saved with + model.save() and reloaded with tf.keras.models.load_model() without + custom_objects. +2. Image-only defenses (gaussian_blur, median_smoothing, spatial_smoothing, + jpeg_compression, pixel_deflection, total_variation_minimization, + feature_denoising) raise ValueError on non-4-D inputs instead of silently + returning the input. JPEG requires 1 or 3 channels. +3. Subclassed or multi-input models raise TypeError: these defenses need a + Functional/Sequential model with a known input_shape. +4. Invalid parameters raise ValueError instead of producing NaN: + bit_depth must be 1-8, num_bins >= 2, quality 1-100, kernel sizes odd, + noise levels >= 0, epsilon >= 0, num_steps >= 1. +5. feature_squeezing, jpeg_compression and total_variation_minimization use a + straight-through gradient so gradient-based attacks against a defended model + still work. +6. word_masking, word_swap and char_swap preserve whitespace, newlines and + punctuation exactly. +7. All randomized helpers accept seed= and are reproducible when it is set. + +MIGRATION FROM 0.1.x (new page) +------------------------------- +- gradient_masking() no longer exists; remove it from user code. +- Results produced with cw, deepfool, ead, spsa or jsma under 0.1.x are invalid + and should be re-run. +- cw: the first positional argument after `y` is still epsilon, but its default + is now None and its meaning is an optional L2 budget. +- jsma: theta now defaults to 0.0. +- pgd/bim/mim: alpha now defaults to epsilon / 4; pass alpha explicitly to keep + the old step size. +- spsa: no longer restricted to batch size 1 or rank-4 inputs. +- Attacks now accept sparse labels, so `y.argmax(-1)` no longer needs + converting to one-hot. +- Defended models saved with 0.1.x cannot be loaded; retrain or re-wrap them. +- Python 3.9+ is required. + +TONE / FORMAT +------------- +- Each function page: one-line description, signature block, parameters table, + returns, a runnable example, and a "Caveats" note. +- Be explicit that preprocessing defenses are not robustness guarantees; link to + the Limitations page. +- Never present an attack's success rate as a defense's guarantee. +- Code samples must be copy-pasteable and use only public API. +``` ## Responsible use diff --git a/tests/test_api.py b/tests/test_api.py index 14584fd..c392159 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -12,7 +12,7 @@ "word_swap", "char_swap", ] EXPECTED_DEFENSES = [ - "adversarial_training", "feature_squeezing", "gradient_masking", + "adversarial_training", "feature_squeezing", "input_transformation", "defensive_distillation", "randomized_smoothing", "feature_denoising", "thermometer_encoding", "adversarial_logit_pairing", "spatial_smoothing", "jpeg_compression", "pixel_deflection", "gaussian_blur", diff --git a/tests/test_defenses.py b/tests/test_defenses.py index 5618022..b6e3eac 100644 --- a/tests/test_defenses.py +++ b/tests/test_defenses.py @@ -247,12 +247,10 @@ def spy(model, batch_x, batch_y, epsilon=0.01): assert calls == [8, 8] -def test_gradient_masking_warns_and_fits(untrained_model, batch): - x, y = batch - with pytest.warns(RuntimeWarning): - wrapped = defenses.gradient_masking(untrained_model, mask_threshold=0.0) - wrapped.compile(optimizer="adam", loss="categorical_crossentropy") - wrapped.fit(x, np.eye(3)[y], epochs=1, verbose=0) +def test_gradient_masking_was_removed(): + """Gradient masking is a false defense; it is no longer part of the API.""" + assert not hasattr(defenses, "gradient_masking") + assert not hasattr(deepdefend, "gradient_masking") def test_defensive_distillation_fits(untrained_model, batch): From 85e1f6771f9ace2b14481111aff32ed594a9bc0c Mon Sep 17 00:00:00 2001 From: Johan Beytell <107624055+johanbeytell1@users.noreply.github.com> Date: Sat, 31 Oct 2026 15:28:02 +0200 Subject: [PATCH 3/3] Update readme.md --- readme.md | 205 +----------------------------------------------------- 1 file changed, 1 insertion(+), 204 deletions(-) diff --git a/readme.md b/readme.md index 051f4e9..c1ee7a1 100644 --- a/readme.md +++ b/readme.md @@ -315,210 +315,7 @@ Keras model, not inferred from reading it. How the fixes are guarded today: 170 tests (`pytest`), `ruff check .`, a CI workflow on Python 3.10 โ€“ 3.12, and a release workflow that refuses to publish -when the tag and `pyproject.toml` version disagree or the tests fail. - -## Documentation update brief (for the docs agent) - -The block below is a self-contained brief. It is a code cell so it renders as -plain text: paste it into the docs agent as-is after any API change. - -```text -DOCS AGENT BRIEF โ€” DeepDefend 0.2.0 -=================================== - -CONTEXT -------- -Repository: Infinitode/DeepDefend (PyPI package: deepdefend) -Branch merged: arena/b52ef87b-deepdefend, released as version 0.2.0. -Published docs live at: -https://infinitode-docs.gitbook.io/documentation/package-documentation/deepdefend-package-documentation - -0.2.0 is a correctness release. Several public functions were broken or silently -wrong before it; the API changed accordingly. The docs must describe the 0.2.0 -behaviour only. Do not keep pages that document 0.1.x behaviour, except in the -"Migration from 0.1.x" page described below. - -TASKS ------ -1. Add a "What's new in 0.2.0" page, linking to CHANGELOG.md in the repo. -2. Create a "Migration from 0.1.x" page (breaking changes; see section below). -3. Update every per-function page for the attacks and defenses listed. -4. Remove the gradient_masking page entirely and redirect it to the new - "Gradient masking is not a defense" note under Limitations. -5. Update the supported-model-types table so it matches the README exactly. -6. Update the Python version support statement to 3.9-3.12. -7. Add the "Limitations" and "Responsible use" sections verbatim from the README. -8. Add code samples for every function, using 0.2.0 signatures (see below). - -REMOVED IN 0.2.0 ----------------- -gradient_masking(model, mask_threshold=0.1) - - Deleted from the package. hasattr(deepdefend, "gradient_masking") is False. - - Reason: it only masked training gradients, never the gradients an attacker - computes at inference, so it provided no protection (verified: attacks - against a wrapped model were bit-identical to attacks against the base - model). Gradient masking is a documented false defense (Athalye et al., - 2018, "Obfuscated Gradients Give a False Sense of Security"). - - Docs action: delete the page; state clearly that the technique creates a - false sense of security and that gradient regularisation belongs in the - user's own train_step, never in a robustness claim. - -CHANGED SIGNATURES AND DEFAULTS (attacks) ------------------------------------------ -All attacks additionally accept: from_logits=None, clip_min=None, clip_max=None -and return a NumPy array with the same shape and dtype as the input. - -fgsm(model, x, y, epsilon=0.01, *, targeted=False, from_logits=None, - clip_min=None, clip_max=None) - - Output is clipped to the valid data range (used to be unclipped). - - New: targeted. - -pgd(model, x, y, epsilon=0.01, alpha=None, num_steps=10, *, random_start=True, - seed=None, targeted=False, from_logits=None, clip_min=None, clip_max=None) - - alpha now defaults to epsilon / 4 (used to be a fixed 0.01). - - New: random_start, seed, targeted. - -bim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, *, seed=None, - targeted=False, from_logits=None, clip_min=None, clip_max=None) - - Now documented as PGD without a random start (they used to be identical). - -mim(model, x, y, epsilon=0.01, alpha=None, num_steps=10, decay_factor=1.0, *, - seed=None, targeted=False, from_logits=None, clip_min=None, clip_max=None) - - Momentum is now normalised per sample, not across the batch. - -cw(model, x, y, epsilon=None, c=10.0, kappa=0.0, num_steps=100, alpha=0.01, *, - targeted=False, from_logits=None, clip_min=None, clip_max=None) - - This is now the real Carlini & Wagner L2 attack (tanh-space optimisation of - ||delta||^2 + c * margin). - - epsilon defaults to None (unconstrained). If set, it is an L2 budget - expressed as an L-infinity equivalent: radius = epsilon * sqrt(num_features). - - c and kappa now genuinely change the result; document that c trades - distortion against the margin and should be swept to minimise distortion. - - The untargeted target class is the runner-up class of the original input. - -deepfool(model, x, y, num_steps=50, epsilon=None, overshoot=0.02, *, - from_logits=None, clip_min=None, clip_max=None) - - Now computes per-class decision boundaries and respects epsilon / clip range - (used to return out-of-range images and ignore any budget). - -jsma(model, x, y, theta=0.0, gamma=0.2, num_steps=10, *, targeted=False, - from_logits=None, clip_min=None, clip_max=None) - - theta defaults to 0.0 (always pick the most salient feature); raw saliency - values are products of two gradients and therefore tiny. - -spsa(model, x, y, epsilon=0.01, num_steps=50, learning_rate=0.05, delta=None, - spsa_samples=32, *, seed=None, targeted=False, from_logits=None, - clip_min=None, clip_max=None) - - Accepts any batch size and any input rank. - - delta defaults to epsilon / 5. - -ead(model, x, y, epsilon=0.01, beta=0.001, num_steps=50, alpha=0.01, *, c=0.1, - kappa=0.0, targeted=False, from_logits=None, clip_min=None, clip_max=None) - - beta defaults to 0.001 (0.01 made the default attack a no-op). - -word_swap(text, swap_dict=None, max_swaps=None, seed=None) - - swap_dict is now required (ValueError when missing). - - Whitespace, newlines and punctuation are preserved. - -char_swap(text, swap_prob=0.1, seed=None) - - Whitespace/punctuation preserved; adds seed. - -LABEL HANDLING (document as a shared concept) ---------------------------------------------- -Every attack accepts one-hot (batch, classes), sparse integer (batch,) and -binary (batch, 1) targets. from_logits is auto-detected from the last layer and -can be overridden. clip_min/clip_max are inferred: [0, 1] when the batch already -lies in that range, otherwise no clipping (so normalised inputs are not -destroyed). Iterative attacks stop working on a sample once it reaches the goal, -so already-misclassified samples are returned unchanged. - -NEW FUNCTIONS (defenses) ------------------------- -smooth_predict(model, x, n_samples=100, noise_level=None, seed=None) - - Monte-Carlo prediction for randomized_smoothing models. - - Returns dict with keys: predictions, probabilities, certified_radius. - - certified_radius = sigma * Phi^-1(p_hat) for the majority class, or 0. - - Document that it requires a noise-trained base model. - -FeatureDenoisingBlock(kernel_size=3) - - Keras layer to insert inside a network for feature-level denoising - (learned residual box filter, starts as an identity). - -defenses.FeatureDenoisingBlock is importable as -`from deepdefend.defenses import FeatureDenoisingBlock`. - -CHANGED DEFENSE SIGNATURES --------------------------- -adversarial_training(model, x, y, epsilon=0.01, epochs=10, batch_size=32, - attack=None, optimizer=None, loss=None, verbose=1, - shuffle=True) - - Generates adversarial examples in batches, keeps the caller's optimiser and - loss, returns a NEW model (a clone). - -defensive_distillation(model, teacher_model, temperature=2) - - Distillation loss is scaled by T^2; the teacher runs in inference mode. - -adversarial_logit_pairing(model, paired_model=None, epsilon=0.01, num_steps=1, - alpha=None) - - Generates adversarial examples inside the training step. - -feature_denoising(model, method="median", kernel_size=3, sigma=1.0) -spatial_smoothing(model, kernel_size=3, method="median", sigma=1.0) -median_smoothing(model, kernel_size=3) # alias of spatial_smoothing(median) -gaussian_blur(model, kernel_size=3, sigma=1.0) # sigma is now used -randomized_smoothing(model, noise_level=0.1, seed=None, - add_noise_at_inference=True) -pixel_deflection(model, deflection_count=100, window_size=10, seed=None) -thermometer_encoding(model, num_bins=10) -word_masking(text, mask_token="[MASK]", mask_prob=0.1, seed=None) -input_transformation(model, transformation_function) # now REQUIRED - -BEHAVIOUR THE DOCS MUST STATE ------------------------------ -1. All preprocessing defenses return a tf.keras.Model that can be saved with - model.save() and reloaded with tf.keras.models.load_model() without - custom_objects. -2. Image-only defenses (gaussian_blur, median_smoothing, spatial_smoothing, - jpeg_compression, pixel_deflection, total_variation_minimization, - feature_denoising) raise ValueError on non-4-D inputs instead of silently - returning the input. JPEG requires 1 or 3 channels. -3. Subclassed or multi-input models raise TypeError: these defenses need a - Functional/Sequential model with a known input_shape. -4. Invalid parameters raise ValueError instead of producing NaN: - bit_depth must be 1-8, num_bins >= 2, quality 1-100, kernel sizes odd, - noise levels >= 0, epsilon >= 0, num_steps >= 1. -5. feature_squeezing, jpeg_compression and total_variation_minimization use a - straight-through gradient so gradient-based attacks against a defended model - still work. -6. word_masking, word_swap and char_swap preserve whitespace, newlines and - punctuation exactly. -7. All randomized helpers accept seed= and are reproducible when it is set. - -MIGRATION FROM 0.1.x (new page) -------------------------------- -- gradient_masking() no longer exists; remove it from user code. -- Results produced with cw, deepfool, ead, spsa or jsma under 0.1.x are invalid - and should be re-run. -- cw: the first positional argument after `y` is still epsilon, but its default - is now None and its meaning is an optional L2 budget. -- jsma: theta now defaults to 0.0. -- pgd/bim/mim: alpha now defaults to epsilon / 4; pass alpha explicitly to keep - the old step size. -- spsa: no longer restricted to batch size 1 or rank-4 inputs. -- Attacks now accept sparse labels, so `y.argmax(-1)` no longer needs - converting to one-hot. -- Defended models saved with 0.1.x cannot be loaded; retrain or re-wrap them. -- Python 3.9+ is required. - -TONE / FORMAT -------------- -- Each function page: one-line description, signature block, parameters table, - returns, a runnable example, and a "Caveats" note. -- Be explicit that preprocessing defenses are not robustness guarantees; link to - the Limitations page. -- Never present an attack's success rate as a defense's guarantee. -- Code samples must be copy-pasteable and use only public API. -``` +when the tag and `pyproject.toml` version disagree, or the tests fail. ## Responsible use