Skip to content

Make the Sphinx developer reference formal and strict-build clean #49

Closed

Description

@mgraffg

Context

This issue builds on #48: it adds a docs job to the .github/workflows/test.yml that #48
creates, and its text assumes the badge URL and CLAUDE.md changes of #48 are already in place.

CompStats implements the methodology published in Analysis of systems' performance in natural
language processing competitions
(Nava-Muñoz, Graff and Escalante, Pattern Recognition Letters,
2024, https://doi.org/10.1016/j.patrec.2024.03.010), which extends Comparison of Classifiers in
Challenge Scheme
(MCPR 2023, https://doi.org/10.1007/978-3-031-33783-3_9). Because the package
backs peer-reviewed work, its documentation must read formally and state the methodology exactly.

The documentation has two halves:

  • the user site (Quarto, quarto/), handled in a follow-up issue, and
  • the developer reference (Sphinx, docs/source/), handled here.

State of the developer reference today (docs/source/):

  • conf.py uses the classic theme, manipulates sys.path, and carries unused LaTeX/man/Texinfo
    boilerplate. Its only intersphinx mapping is scikit-learn.

  • index.rst has a badge block, an introduction, an Architecture section (with a Mermaid class
    diagram, keep it), Invariants to preserve, a Citing section that cites only the 2023 paper
    with a malformed DOI (10.1007/978-3-031-33783-3{\_}9/COVER), and an API toctree.

  • There is one page per module (bootstrap_api.rst, interface_api.rst, measurements_api.rst,
    metrics_api.rst, performance_api.rst) but none for CompStats.utils.

  • A strict build (sphinx -n -W) fails with 258 warnings. Measured breakdown:

    Count Warning Where Cause
    91 py:class ... numpy.ndarray mostly metrics.py no intersphinx mapping for NumPy
    30 py:class ... pandas.DataFrame metrics.py, interface.py no mapping for pandas
    ~90 py:attr ... func, func., y_pred., name., args., BiB, correction, measure_names, hy, best utils.py template, metrics.py, interface.py, *.rst parameter names written as :py:attr: cross-references, which point at nothing
    ~30 py:func ... CompStats.metrics.<name>.measure utils.py template, metrics_api.rst, interface.py .measure is an attribute set on a function, not a documentable object
    4 py:class ... CompStats index.rst the package referenced as a class
    5 statsmodels...multipletests, scipy.stats.pearsonr, pd.DataFrame, sns.axisgrid.FacetGrid, StatisticSamples, Perf. interface.py, metrics.py, performance.py, bootstrap_api.rst missing mappings, abbreviated or unqualified names
    11 [docutils] ("Inline strong start-string without end-string", "Block quote ends without a blank line") performance.py docstrings malformed reStructuredText (**kwargs, lists without a blank line before them)

    About 200 of the 258 come from CompStats/metrics.py, and most of those are generated by the
    single docstring template inside metrics_docs in CompStats/utils.py, so fixing the template
    once removes them all.

  • The developer docs are currently hosted on ReadTheDocs (.readthedocs.yaml). Decision already
    taken:
    they move to GitHub Pages under /dev/ of the project site
    (https://ingeotec.github.io/CompStats/dev/) and ReadTheDocs is dropped. The workflow that
    publishes them is added by the follow-up issue; this issue makes the Sphinx source ready for it.

Facts already verified while planning (do not re-derive them):

  • .venv/bin/python (Python 3.13) has sphinx 9.2, furo, sphinxcontrib-mermaid, docutils,
    numpy, pandas, scipy, statsmodels, seaborn installed. Do not install anything.
  • A full Sphinx build takes about 10 seconds plus the download of the intersphinx inventories
    (needs network).
  • Every public class, function, method and property in CompStats/*.py already has a non-empty
    docstring.

Scope

In scope:

  • docs/source/conf.py, every docs/source/*.rst, a new docs/requirements.txt, a new
    docs/source/utils_api.rst.
  • Docstring text in CompStats/bootstrap.py, interface.py, measurements.py, metrics.py,
    performance.py, utils.py, only as far as needed for the strict build and for the short list
    of weak docstrings in step 5.
  • A docs job in .github/workflows/test.yml.
  • Removing .readthedocs.yaml.
  • CompStats/tests/test_docs.py (new).
  • .gitignore, requirements.txt, CLAUDE.md (small additions listed below).

Out of scope:

  • Any change to executable code under CompStats/. Only string literals that are docstrings (and
    the docstring template string inside metrics_docs) may change. After the work,
    git diff develop -- CompStats/*.py must show changes inside docstrings only. Do not rename,
    reorder, add or remove functions, parameters, defaults or imports.
  • quarto/, README.rst, pyproject.toml, .github/workflows/publish.yml,
    .github/workflows/publish-pypi.yml: untouched here (the follow-up issue handles the site, the
    README and the URLs).
  • Running the docstring examples as doctests. They print bootstrap estimates that depend on
    unseeded random draws, so their outputs are illustrative; do not add sphinx -b doctest to CI
    and do not try to make the outputs reproducible.
  • environment.yml, docs/Makefile, the two notebooks docs/CompStats.ipynb and
    docs/CompStats_metrics.ipynb, the .devcontainer/ files.
  • Translating or restyling docstrings beyond what is listed. Keep the existing
    :param name: / :type name: field style; do not convert to NumPy or Google style.

Implementation plan

1. Build tooling

  1. Create docs/requirements.txt:

    # Build-time dependencies for the Sphinx developer reference. CompStats itself is
    # installed normally (pip install -e .) by the workflows that build these docs.
    sphinx>=7
    furo
    sphinxcontrib-mermaid
    
  2. Add a line furo to the root requirements.txt (the devcontainer installs from it).

  3. Add a line docs/build/ to .gitignore, next to the existing docs/_build/ line.

  4. git rm .readthedocs.yaml.

The build command used throughout this issue (strict: -n reports every unresolved
cross-reference, -W turns warnings into errors, --keep-going lists them all in one run):

.venv/bin/python -m sphinx -n -W --keep-going -b html docs/source docs/build/html

To see the warnings without stopping, and to count them while you work:

.venv/bin/python -m sphinx -n -b html docs/source docs/build/html 2> /tmp/sphinx-warnings.txt; grep -c WARNING /tmp/sphinx-warnings.txt

2. Rewrite docs/source/conf.py

Replace the whole file with this (adjust only if the build demands it):

# Configuration file for the Sphinx documentation builder.
#
# For a full list of options see:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup ---------------------------------------------------------
#
# CompStats is installed (editable) by the builds that render these docs
# (.github/workflows/test.yml and website.yml), so conf.py only imports the
# installed package to read its __version__.

import CompStats

# -- Project information -------------------------------------------------

project = 'CompStats'
copyright = '2024, Sergio Nava-Muñoz, Mario Graff, Hugo Jair Escalante'
author = 'Sergio Nava-Muñoz, Mario Graff, Hugo Jair Escalante'
version = CompStats.__version__
release = version

rst_epilog = f"""
.. |compstats_version| replace:: {CompStats.__version__}
"""

# -- General configuration ------------------------------------------------

extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.mathjax',
    'sphinx.ext.viewcode',
    'sphinx.ext.intersphinx',
    'sphinxcontrib.mermaid',
]

intersphinx_mapping = {
    'python': ('https://docs.python.org/3', None),
    'numpy': ('https://numpy.org/doc/stable/', None),
    'pandas': ('https://pandas.pydata.org/docs/', None),
    'scipy': ('https://docs.scipy.org/doc/scipy/', None),
    'sklearn': ('https://scikit-learn.org/stable/', None),
    'statsmodels': ('https://www.statsmodels.org/stable/', None),
    'seaborn': ('https://seaborn.pydata.org/', None),
    'matplotlib': ('https://matplotlib.org/stable/', None),
}

source_suffix = '.rst'
master_doc = 'index'
language = 'en'
exclude_patterns = []

pygments_style = 'sphinx'
highlight_language = 'python'
autodoc_member_order = 'bysource'
autodoc_class_signature = 'separated'
add_function_parentheses = False
add_module_names = False

# -- Options for HTML output ----------------------------------------------

html_theme = 'furo'
html_logo = 'ingeotec.png'
htmlhelp_basename = 'CompStatsdoc'

If one specific external name still cannot be resolved after the mappings above (an inventory that
simply does not list it), add it to a nitpick_ignore = [('py:class', '<name>'), ...] list with a
one-line comment saying why. This is a last resort for at most a handful of names. Never disable
nitpicky mode, never add nitpick_ignore_regex patterns that match whole packages, and never
remove -n or -W from the command.

3. Fix the warnings (docstring text only)

Work category by category, rebuilding after each one. The rules:

  1. External types. The intersphinx mappings resolve numpy.ndarray, pandas.DataFrame,
    statsmodels.stats.multitest.multipletests and scipy.stats.pearsonr with no text change.
    Abbreviated names never resolve: write pandas.DataFrame instead of pd.DataFrame,
    numpy.ndarray instead of np.ndarray, seaborn.FacetGrid instead of
    sns.axisgrid.FacetGrid. A type such as numpy.ndarray[bool] does not resolve either: write
    numpy.ndarray of bool.

  2. Parameter names are literals, not cross-references. Every :py:attr: role whose target is
    a parameter or a local variable (func, y_pred, y_score, y_proba, BiB, name, args,
    correction, measure_names, hy, best, ...) becomes an inline literal with double
    backticks, and a trailing period moves outside the literal. Examples:

    • :py:attr:`func.` -> ``func``.
    • identified with alg-k where k=1 is the first argument included in :py:attr:`y_pred.` ->
      ... included in y_pred.
    • (as :py:attr:`func`) -> (as func)

    Keep :py:attr:/:py:meth: only when the target really is a documented attribute, property
    or method, and then qualify it fully, e.g. :py:attr:`~CompStats.interface.Perf.best` ,
    :py:meth:`~CompStats.interface.Difference.p_value` .

  3. .measure factories are literals. :py:func:`~CompStats.metrics.f1_score.measure` ->
    ``f1_score.measure``. This applies to the template in metrics_docs
    (CompStats/utils.py), where the name is interpolated ({func.__name__}.measure), to
    docs/source/metrics_api.rst and to the Perf class docstring.

  4. The package is not a class. In index.rst, :py:class:`CompStats` -> **CompStats**
    (or plain CompStats).

  5. Unqualified names. :py:class:`Perf.` -> :py:class:`~CompStats.interface.Perf`.;
    :py:class:`StatisticSamples` -> :py:class:`~CompStats.bootstrap.StatisticSamples` .
    References to Difference must use CompStats.interface.Difference (it is not exported at the
    package root, so ~CompStats.Difference does not resolve).

  6. Malformed reStructuredText in CompStats/performance.py (the 11 [docutils] warnings):
    write **kwargs as an inline literal (``**kwargs``), and put a blank line before
    and after every bullet or numbered list and every indented block inside a docstring.

  7. The metrics_docs template (CompStats/utils.py, the f-string assigned to
    func.__doc__). Apply rules 2 and 3 inside it. It is an f-string: literal braces are not
    involved, but keep {func.__name__}, {direction} and {hy_name} exactly as they are. Note
    that the generated :param {hy_name}: line says "included in :py:attr:y_pred." regardless of
    hy_name; make it say ``{hy_name}`` so the sentence names the right parameter.
    The wrappers that do not use the decorator (macro_f1, macro_recall, macro_precision,
    d2_absolute_error_score, pearsonr) have hand-written docstrings with the same patterns:
    fix them the same way.

Stop when the strict build command of step 1 ends with build succeeded. and no warning.
CompStats/tests/test_metrics.py and the rest of the suite must still pass: no test depends on
docstring text, so a failure means code was changed by accident.

4. Restructure and formalize docs/source/index.rst

Rewrite index.rst as the landing page of the developer reference, in this order. Formal,
impersonal, present tense; no marketing language; every statistical claim must be one that appears
below or in the code. Keep the existing reference label .. _CompStats: at the top.

  1. Title and badges. Title CompStats. Keep the badges for: GitHub, tests (test.yml),
    Coveralls, PyPI, the three conda-forge badges. Remove the ReadTheDocs badge and the Colab
    badge (it points at a notebook on another branch). Add the line
    Version |compstats_version|. under the badges.

  2. Note pointing at the user site (keep the existing .. note::, reworded): this is the
    developer reference (architecture and full API); installation, tutorials and the description
    of the methodology for users are on the project site, https://ingeotec.github.io/CompStats/.

  3. Introduction (one or two paragraphs). Content to convey, in your own words: collaborative
    competitions (challenges) evaluate systems on a single held-out test set whose gold standard is
    kept by the organizers; systems are ranked by a performance measure computed on that set;
    organizers have access only to the predictions, a fixed dataset size and a small number of
    submissions, so the classical tests that need the algorithm or several datasets do not apply;
    CompStats implements a bootstrap-based methodology that complements the ranking with standard
    errors, confidence intervals, significance tests of paired differences and corrections for
    multiple comparisons; the same setting covers any comparison of supervised-learning systems on
    a shared test set, classification or regression. Cite the two papers by title with their DOI
    links.

  4. Statistical background (new section). State each concept with its formula and name the
    symbol of the code that computes it. Use .. math:: blocks or :math: roles. Required
    content (this is what the code does; do not embellish):

    • Setting. A gold standard :math:y of size :math:n, predictions
      :math:\hat{y}_1, \ldots, \hat{y}_m of :math:m systems, and a performance measure
      :math:s(y, \hat{y}). A measure is either bigger-is-better (a score) or smaller-is-better
      (an error); the flag BiB carries that direction.
    • Bootstrap. :math:B samples (num_samples, default 500) of :math:n indices are drawn
      with replacement from :math:\{1, \ldots, n\}; the measure is evaluated on every sample for
      every system, giving the bootstrap distribution :math:s_j^{*1}, \ldots, s_j^{*B}
      (:py:class:~CompStats.bootstrap.StatisticSamples). The same index samples are used for
      all systems, which is what makes the comparisons paired.
    • Standard error. The standard deviation of the bootstrap distribution
      (:py:func:~CompStats.measurements.SE, exposed as the se property of
      :py:class:~CompStats.interface.Perf).
    • Confidence interval. The percentile interval
      :math:[q_{\alpha/2}(s_j^{*}),\; q_{1-\alpha/2}(s_j^{*})], with :math:\alpha = 0.05 by
      default (:py:func:~CompStats.measurements.CI, the ci property of Perf).
    • Paired difference. For a reference system :math:A (by default the best one) and another
      system :math:B, the bootstrap distribution of
      :math:\delta^{*} = s^{*}(y, \hat{y}_A) - s^{*}(y, \hat{y}_B) computed on the same samples
      (sign flipped for smaller-is-better measures, so a positive difference always means that the
      reference is better). Built by Perf.difference, which returns a
      :py:class:~CompStats.interface.Difference.
    • Significance. With :math:\delta(x) the difference observed on the test set, the p-value
      of :math:H_0: "the reference is not better" is estimated as the fraction of bootstrap
      differences that are at least :math:2\delta(x)
      (Difference.p_value), following Berg-Kirkpatrick, Burkett and Klein (2012): the bootstrap
      distribution is centred on :math:\delta(x) rather than on zero, hence the factor 2.
    • Multiple comparisons. Comparing the best system against the other :math:m - 1 tests
      several hypotheses at once; the p-values can be adjusted with any method of
      :py:func:statsmodels.stats.multitest.multipletests through the correction argument
      (for example 'bonferroni', 'holm', 'fdr_bh'). With several measures, each measure is
      corrected as its own family.
    • Competitiveness of a competition (legacy functional API,
      :py:func:~CompStats.performance.performance_multiple_metrics): the coefficient of
      variation of the systems' performance :math:CV = 100 \, s_x / \bar{x} (sample standard
      deviation), the possible percentage improvement :math:PPI = 100\,(1 - x_{best}) for scores
      bounded by 1, and the distance between the winner and the median system.
  5. Architecture — keep the existing section (data flow, the Mermaid class diagram, the
    per-class descriptions) and fix its cross-references so they resolve.

  6. Invariants to preserve — keep the three existing notes.

  7. Development (new section, short, commands in .. code:: bash blocks):

    • install: pip install -e ".[test]" and pip install -r docs/requirements.txt;
    • tests: pytest CompStats;
    • developer reference: the strict build command of step 1 and where the HTML lands
      (docs/build/html/index.html);
    • a sentence saying that the same strict build runs in CI and that a warning fails it;
    • releasing: one sentence pointing at the tag-driven workflow (v<version> tag,
      .github/workflows/publish-pypi.yml) and the version gate
      (scripts/check_package_versions.py).
  8. Citing. Replace the current single entry with these two (copy them exactly; do not add
    volume, pages or any field that is not here):

    @article{Nava-Munoz2024,
      author  = {Nava-Mu{\~n}oz, Sergio and Graff, Mario and Escalante, Hugo Jair},
      title   = {Analysis of systems' performance in natural language processing competitions},
      journal = {Pattern Recognition Letters},
      year    = {2024},
      doi     = {10.1016/j.patrec.2024.03.010},
      url     = {https://doi.org/10.1016/j.patrec.2024.03.010}
    }
    
    @inproceedings{Nava-Munoz2023,
      author    = {Nava-Mu{\~n}oz, Sergio and Graff Guerrero, Mario and Escalante, Hugo Jair},
      title     = {Comparison of Classifiers in Challenge Scheme},
      booktitle = {Pattern Recognition},
      series    = {Lecture Notes in Computer Science},
      volume    = {13902},
      pages     = {89--98},
      year      = {2023},
      publisher = {Springer Nature Switzerland},
      doi       = {10.1007/978-3-031-33783-3_9},
      url       = {https://doi.org/10.1007/978-3-031-33783-3_9}
    }

    Say that the first is the reference to cite for the methodology and the second is the work it
    extends. Add, after them, a short "References" list for the works named in the statistical
    background (plain text entries, exactly these):

    • Berg-Kirkpatrick, T., Burkett, D. and Klein, D. (2012). An empirical investigation of
      statistical significance in NLP. EMNLP-CoNLL 2012.
    • Chernick, M. R. and LaBudde, R. A. (2011). An Introduction to Bootstrap Methods with
      Applications to R
      . Wiley.
    • Efron, B. and Tibshirani, R. J. (1994). An Introduction to the Bootstrap. Chapman and
      Hall/CRC.
    • Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian
      Journal of Statistics
      , 6, 65-70.
    • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and
      powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B,
      57, 289-300.
  9. API toctree, in this order: interface_api, metrics_api, measurements_api,
    bootstrap_api, performance_api, utils_api.

5. Module pages

  • Create docs/source/utils_api.rst, same shape as bootstrap_api.rst:

    :mod:`CompStats.utils`
    ==================================
    
    .. automodule:: CompStats.utils
       :members:
  • metrics_api.rst: remove the whole badge block at the top. Keep the worked example (digits
    dataset) and the Multi-measure Perf section, but make them agree with the current API: the
    first call must name its system (score = f1_score(y_val, hy, average='macro', name='Linear SVM')) and the printed tables below it must then show Linear SVM where they show alg-1;
    add from sklearn.ensemble import HistGradientBoostingClassifier to the imports (it is used
    but never imported) and drop the unused from sklearn.base import clone. State once, before
    the first printed output, that the numbers are illustrative because the train/validation split
    and the bootstrap samples are random. Apply rules 2, 3 and 5 of step 3 to its text.

  • performance_api.rst: add one introductory sentence before the automodule saying that
    CompStats.performance is the earlier, functional API that operates on a pandas.DataFrame
    (one gold column plus one column per system), that it is kept for the multi-metric competition
    reports of the paper (coefficient of variation, PPI, distance to the best), and that new code
    should use Perf/Difference.

  • One-line or weak docstrings to improve while you are in the files (text only, same :param:
    style, one or two sentences each, saying what the object is and returns):
    Difference (class summary, currently just "Difference"), difference_p_value,
    StatisticSamples.keys, StatisticSamples.samples, Perf.difference (first line), SE
    (state what is returned; its example ends with a call that shows no output, add an illustrative
    one or remove the dangling call).

6. CI: add the docs job to .github/workflows/test.yml

Append this job to the jobs: mapping of the file created by #48 (same indentation as the other
jobs; do not change the existing ones):

  # Sphinx gate: a strict HTML build of the developer reference. Any broken
  # cross-reference or other warning fails it. The docstring examples are not
  # run (their outputs depend on random bootstrap samples).
  docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install docs and package dependencies
        run: |
          pip install -r docs/requirements.txt
          pip install -e .
      - name: Build the docs strictly
        run: python -m sphinx -n -W --keep-going -b html docs/source docs/build/html

7. CLAUDE.md

In ## Commands, replace the "Build the Sphinx docs" block (cd docs && make html) with the
strict build command of step 1, and say that it is the same command CI runs and that warnings are
errors. Add one sentence to the metrics.py bullet of ## Architecture: in docstrings, parameter
names and .measure factories are written as inline literals (double backticks), not as
:py:attr:/:py:func: cross-references, because the strict Sphinx build rejects references that
do not resolve.

Acceptance criteria

  • .venv/bin/python -m sphinx -n -W --keep-going -b html docs/source docs/build/html ends
    with build succeeded. and zero warnings.
  • git diff develop -- 'CompStats/*.py' contains only docstring changes (no change in any
    line of executable code, signature, default value or import).
  • docs/source/conf.py uses the furo theme, does not touch sys.path, and maps
    intersphinx inventories for python, numpy, pandas, scipy, sklearn, statsmodels, seaborn and
    matplotlib.
  • nitpick_ignore, if present at all, lists at most five individually named targets, each
    with a comment.
  • docs/source/index.rst has, in order: introduction, statistical background, architecture
    (with the Mermaid diagram), invariants, development, citing, API toctree.
  • index.rst contains both DOIs (10.1016/j.patrec.2024.03.010 and
    10.1007/978-3-031-33783-3_9) and no longer contains /COVER.
  • Every module CompStats/<name>.py (bootstrap, interface, measurements, metrics,
    performance, utils) is named by exactly one .. automodule:: directive under docs/source.
  • No file under docs/source contains the string readthedocs; .readthedocs.yaml is gone.
  • docs/requirements.txt exists; .github/workflows/test.yml has a docs job running the
    strict build.
  • docs/build/ is git-ignored and nothing under it is committed.
  • .venv/bin/python -m pytest CompStats is green.

Tests

Add CompStats/tests/test_docs.py (pytest style, English). The tests package is shipped in the
wheel, so everything that needs the repository must skip outside a checkout:

import re
from pathlib import Path

import pytest

ROOT = Path(__file__).resolve().parents[2]
DOCS = ROOT / "docs" / "source"

pytestmark = pytest.mark.skipif(
    not DOCS.is_dir(), reason="needs the repository checkout (docs/source)")

_AUTOMODULE_RE = re.compile(r"^\.\.\s+automodule::\s+(\S+)\s*$", re.MULTILINE)

Required test functions:

  1. test_every_module_is_documented_exactly_once: list CompStats/*.py except __init__.py,
    read every *.rst under DOCS, and assert that CompStats.<stem> appears in exactly one
    automodule directive.
  2. test_public_api_has_docstrings: for each of the six modules, every public (no leading
    underscore) function and class defined in that module (obj.__module__ == module.__name__),
    and every public method and property of those classes, has a non-empty docstring.
  3. test_index_sections_in_order: the text of index.rst contains the section titles
    Statistical background, Architecture, Invariants to preserve, Development, Citing
    and API, each one after the previous.
  4. test_index_cites_both_papers: index.rst contains both DOIs listed above and does not
    contain /COVER.
  5. test_no_readthedocs_in_developer_docs: no *.rst or conf.py under DOCS contains
    readthedocs (case-insensitive), and ROOT / ".readthedocs.yaml" does not exist. Build the
    forbidden word from two fragments ("readthe" + "docs") so the test file does not flag
    itself if a later test scans the whole repository.
  6. test_conf_uses_furo_and_intersphinx: the text of conf.py contains html_theme = 'furo',
    does not contain sys.path, and names each of numpy, pandas, scipy, sklearn,
    statsmodels, seaborn in intersphinx_mapping.
  7. test_docs_job_in_ci: .github/workflows/test.yml contains
    python -m sphinx -n -W --keep-going -b html docs/source docs/build/html (skip if the
    workflow file is absent).

The strict Sphinx build is not run from pytest (it needs the network and the docs
dependencies); it is its own acceptance criterion and its own CI job.

Commands that must be green before committing:

.venv/bin/python -m sphinx -n -W --keep-going -b html docs/source docs/build/html
.venv/bin/python -m pytest CompStats

The first takes about 10 seconds plus inventory downloads and ends with build succeeded.; the
second takes about 2.5 minutes and ends with a line like 89 passed.

Decisions

  • Developer docs go to GitHub Pages under /dev/, ReadTheDocs is dropped (chosen by the
    maintainer). The published article points at compstats.readthedocs.org; redirecting or
    retiring that ReadTheDocs project is a manual step for the maintainer, outside the repository.
  • Strict build (-n -W) is the gate (chosen by the maintainer), fixed by editing docstring
    text, never by loosening Sphinx.
  • No doctest job: the examples print values that depend on unseeded random sampling.
  • furo theme and no sys.path manipulation: same as the sibling projects scjn and
    dofjson; the package is installed before the docs are built.
  • :param:/:type: field style kept: CLAUDE.md requires following the existing docstring
    style, so sphinx.ext.napoleon is not added.
  • Parameter names and .measure factories become inline literals: they are not documentable
    objects, so a cross-reference to them can never resolve.
  • BibTeX entries carry only verified fields: the journal entry has no volume or pages because
    the planning sources did not have them; do not invent them.
  • The statistical background is written in the developer reference as well as on the user
    site
    : contributors need the mapping from each concept to the symbol that implements it.
  • The docs test lives in CompStats/tests/: the project's test command is
    pytest CompStats.

Planned against develop at d2c6feade9eac5eb039b405ac272f70fc66458ce.

Activity

  1. mgraffg commented on Nov 19, 2026

    @mgraffg
    MemberAuthor

    Implemented on branch issues-48-49-50, commit 7d0dfd1.

    • Strict build python -m sphinx -n -W --keep-going -b html docs/source docs/build/html (fresh build) ends with build succeeded. and no warning (was 258). No nitpick_ignore needed.
    • Docstring text only changed under CompStats/ (verified by comparing the ASTs with docstrings stripped: identical, except the metrics_docs template string in utils.py, which is the f-string docstring template).
    • New conf.py (furo, intersphinx for 8 projects), docs/requirements.txt, utils_api.rst; index.rst restructured (introduction, statistical background, architecture, invariants, development, citing, API); metrics_api.rst/performance_api.rst updated; .readthedocs.yaml removed; docs job added to test.yml; CLAUDE.md updated; CompStats/tests/test_docs.py added (7 tests).
    • Tests: .venv/bin/python -m pytest CompStats green (exit 0).
    • Decision: for SE I kept the example call and added an illustrative output (0.1459) instead of removing it.
  2. mgraffg commented on Nov 20, 2026

    @mgraffg
    MemberAuthor

    Review fix-1 (commit f4b315a, branch issues-48-49-50): the "Competitiveness of a competition" paragraph of docs/source/index.rst now follows the article: CV and the winner-median distance measure competitiveness, PPI is presented separately as the potential for improvement. Strict Sphinx build succeeds; full suite: 100 passed.

  3. mgraffg commented on Nov 20, 2026

    @mgraffg
    MemberAuthor

    Implemented in #51, merged into develop. Commit 7d0dfd1.

  4. added 2 commits that reference this issue on Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions