Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# AGENTS.md — Semantic External Query Lookup (SEQL)

Guidance for agents and humans working in this repository. Review findings are
persisted in `REVIEW.md` (defect findings and simplification findings are kept
in separate sections there).

## Project

SEQL is a [Semantic MediaWiki](https://github.com/SemanticMediaWiki/SemanticMediaWiki)
extension that integrates `#ask`/`#show` query results from an external
MediaWiki wiki into a local wiki. PHP ≥ 8.1, MediaWiki ≥ 1.43, Semantic
MediaWiki ≥ 7.0 (see `extension.json` and `composer.json`).

## Repository layout

- `src/` — first-party code, PSR-4 namespace `SEQL\` (registered in
`extension.json` → `AutoloadNamespaces`).
- `tests/phpunit/Unit/`, `tests/phpunit/Integration/` — PHPUnit suites with
JSON fixtures under `tests/phpunit/Integration/ByHttpRequest/Fixtures/`.
- `i18n/` — MediaWiki message files; `en.json` and `qqq.json` must carry every
`seql-*` key (enforced by `I18nJsonFileIntegrityTest` plus translatewiki
practice).
- `build/` — git submodule (`docker-compose-ci`); initialize with
`git submodule update --init --remote` (the `Makefile` does this
automatically when `build/` is missing). Not first-party code.
- `docs/` — end-user configuration documentation.

## Running tests, linters, and builds

Prerequisites: a MediaWiki install with Semantic MediaWiki is required at
runtime; `composer phpunit` resolves the MediaWiki root from `MW_INSTALL_PATH`
(default `../..`). The test bootstrap (`tests/bootstrap.php`) hard-fails if the
SMW test autoloader is not present.

All commands run through Composer scripts defined in `composer.json`:

| Task | Command |
| --- | --- |
| Full test suite (analyze + phpunit) | `composer test` |
| Static analysis only (lint + phpcs + minus-x) | `composer analyze` |
| Unit tests only | `composer unit` |
| Integration tests only | `composer integration` |
| Tests with coverage | `composer test-coverage` |
| PHP parallel lint | `composer lint` |
| Code sniffer (MediaWiki standard) | `composer phpcs` |
| Auto-fix sniffer findings | `composer fix` |
| Executable-bit / shebang check | `composer minus-x` |

CI (GitHub Actions, `.github/workflows/ci.yaml`) runs `make ci` /
`make ci-coverage`, which drives the `docker-compose-ci` submodule with the
versions declared in the `Makefile` (`MW_VERSION`, `PHP_VERSION`, `DB_TYPE`,
`DB_IMAGE`, `SMW_VERSION`).

There is no separate build step; the extension is a Composer package
(`mediawiki/semantic-external-query-lookup`).

## Coding conventions

- Style is the **MediaWiki coding standard**, enforced by
`mediawiki/mediawiki-codesniffer` 48.0.0 via `.phpcs.xml`; tabs for
indentation, `aligned` spacing inside function calls as seen throughout
`src/`.
- Every class file carries the `@license GPL-2.0-or-later` docblock.
- Classes live in `src/` under `SEQL\`; test classes use
`SEQL\Tests\` / `SEQL\ByHttpRequest\Tests\` namespaces (note: the latter is
currently **not** matched by the PSR-4 mappings in `tests/bootstrap.php` —
see REVIEW.md).
- Configuration is declared in `extension.json` with the `seqlg` prefix
(`config_prefix`); code reads it as `$GLOBALS['seqlg…']`.
- User-facing strings are MediaWiki messages in `i18n/`, keys prefixed
`seql-`.
- Behavior changes ship with unit/integration tests and a RELEASE-NOTES entry.

## Project rules

1. **POSIX shell only.** Any shell script added to this repo must be strictly
POSIX (`#!/bin/sh`, `[ ]` tests, quoted variable expansions; `set -eux` for
setup scripts) and be verified with `shellcheck` and `dash -n` where
available. The repo currently contains **no** shell scripts (the Makefile
is not shell); this rule applies to any future ones.
2. **Validate installed versions.** Any installed tool, runtime, or
dependency whose version matters must be validated after installation:
compare installed vs. expected version and fail fast with a clear error on
mismatch. Today this is satisfied declaratively — `composer.json` requires
`php >=8.1.0` and `extension.json` requires MediaWiki ≥ 1.43 / SMW ≥ 7.0
(checked by MediaWiki's extension registry) — keep it that way for any new
tooling.
3. **Funnel dependency installs through one declared place.** PHP dependencies
come exclusively from `composer.json`; CI toolchain versions come
exclusively from the `Makefile` variables passed to the
`docker-compose-ci` submodule. Do not add ad-hoc `install`/`require`
commands scattered in scripts or CI steps.

## Known issues

See `REVIEW.md` for the full, ordered list (defect findings and simplification
findings). The most significant ones at review time (2026-11-13):

- `seqlgExternalQueryEnabledNamespaces` default `[]` **blocks** `source=`
queries in every namespace, contradicting the documented "keep empty to
allow every namespace" (`src/HookRegistry.php:93`, `extension.json`).
- All outbound HTTP requests disable TLS certificate/host verification
(`src/ByHttpRequest/QueryResultFetcher.php`).
- The remote-login flow uses the deprecated `action=login` API and stores its
cookie jar in a `static` shared across instances and endpoints.
- `.gitreview` names the wrong Gerrit project (`SemanticMetaTags`).
18 changes: 18 additions & 0 deletions RELEASE-NOTES.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,30 @@
This file contains the RELEASE-NOTES of the Semantic External Query Lookup (a.k.a. SEQL) extension.

### 2.0.1 (not yet released)

- Fixed `source=` queries failing silently: a transport failure or an
unparseable response from the external repository is now reported as a
query error (`seql-http-request-failed` / `seql-http-response-invalid`)
instead of rendering an empty result without any diagnostic
- A response that does not decode as JSON is no longer written to the
HTTP response cache, so a failed fetch is retried after the error is
fixed instead of being frozen for the cache lifetime
- Added a "Names must align" section to the configuration documentation
explaining that the `source=` value, the `$smwgQuerySources` key, and
the `$seqlgExternalRepositoryEndpoints` prefix must carry the same
name, and that a mismatch (or a later wholesale re-assignment of
`$smwgQuerySources`) silently falls back to the local store

### 2.0.0 (2026-06-08)

- Added support for Semantic MediaWiki 7.0
- Raised the minimum requirement to MediaWiki 1.43
- Raised the minimum requirement to Semantic MediaWiki 7.0
- Raised the minimum requirement to PHP 8.1
- Replaced the `mediawiki/http-request` dependency with MediaWiki core's `HttpRequestFactory`
- Removed the `InterwikiLoadPrefix` hook registration (deprecated since MediaWiki 1.36);
endpoints in `$seqlgExternalRepositoryEndpoints` are now exposed by decorating the
`InterwikiLookup` service via the `MediaWikiServices` hook

### 1.0.0 (2016-??-??)

Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
"composer/installers": "^2|^1.0.1"
},
"require-dev": {
"mediawiki/mediawiki-codesniffer": "48.0.0",
"mediawiki/mediawiki-codesniffer": ">=51.0.1",
"mediawiki/minus-x": "1.1.3",
"php-parallel-lint/php-console-highlighter": "1.0.0",
"php-parallel-lint/php-parallel-lint": "1.4.0"
Expand Down
24 changes: 24 additions & 0 deletions docs/00-configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,30 @@ $GLOBALS['seqlgExternalRepositoryEndpoints'] = array(
)
);
````

#### Names must align

The `source=` value used in a `#ask`/`#show` query, the key in `$GLOBALS['smwgQuerySources']`, and the key in
`$GLOBALS['seqlgExternalRepositoryEndpoints']` (or the interwiki table) must carry **the same name**. In the examples
above that name is `mw-foo`; queries use `{{#ask: ... |source=mw-foo }}`.

The failure modes of a mismatch are silent:

- If `source=X` names no key in `$GLOBALS['smwgQuerySources']`, Semantic MediaWiki resets the source to its default and
runs the query against the **local** store without raising an error. The rendered output is an empty result whenever
the local wiki cannot answer the query, which is indistinguishable from a broken endpoint.
- If `X` names a query source but is not resolvable as an interwiki prefix, SEQL renders the
`seql-interwiki-prefix-is-missing` warning instead of results.

When adding a source with `= array( ... )`, use the array-append operator (`$GLOBALS['smwgQuerySources']['mw-foo'] = ...`)
or merge with existing entries. A wholesale re-assignment in a settings file that is loaded **after** this registration
(for example a farm- or host-level configuration layered on top of `LocalSettings.php`) discards the query source
without any notice.

A quick check for a suspected fallback: `{{#ask: ... |source=X |format=debug }}`. Debug queries are not supported by
external query sources, so a *normal local* debug pane is the signature of the local-store fallback; an error message
about debug queries confirms that the external source was selected.

### Cache

To help limit the amount of request made to an endpoint, SEQL provides:
Expand Down
8 changes: 8 additions & 0 deletions extension.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@
"i18n"
]
},
"Hooks": {
"MediaWikiServices": "main"
},
"HookHandlers": {
"main": {
"class": "SEQL\\MediaWikiServicesHandler"
}
},
"AutoloadNamespaces": {
"SEQL\\": "src/"
},
Expand Down
4 changes: 3 additions & 1 deletion i18n/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,7 @@
"seql-desc": "Allows to integrate query results from a remote [https://semantic-mediawiki.org Semantic MediaWiki] into a local wiki",
"seql-name": "Semantic External Query Lookup",
"seql-interwiki-prefix-is-missing": "The selected \"$1\" source is missing a matching interwiki prefix.",
"seql-debug-query-not-supported": "The display of results for a debug query is not supported when using an external query source."
"seql-debug-query-not-supported": "The display of results for a debug query is not supported when using an external query source.",
"seql-http-request-failed": "The request to the external repository at \"$1\" failed or could not be answered with a usable response.",
"seql-http-response-invalid": "The response from the external repository at \"$1\" could not be decoded."
}
4 changes: 3 additions & 1 deletion i18n/qqq.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,7 @@
"seql-desc": "{{desc|name=Semantic External Query Lookup|url=https://www.mediawiki.org/wiki/Extension:Semantic_External_Query_Lookup}}",
"seql-name": "{{name}}",
"seql-interwiki-prefix-is-missing": "This is an error message.\n\n* $1 holds the label of the configured query source.",
"seql-debug-query-not-supported": "This is an error message."
"seql-debug-query-not-supported": "This is an error message.",
"seql-http-request-failed": "This is an error message.\n\n* $1 holds the URL of the queried external repository endpoint.",
"seql-http-response-invalid": "This is an error message.\n\n* $1 holds the URL of the queried external repository endpoint."
}
34 changes: 30 additions & 4 deletions src/ByHttpRequest/QueryResultFetcher.php
Original file line number Diff line number Diff line change
Expand Up @@ -202,9 +202,20 @@ public function fetchQueryResult( Query $query ) {
$this->doAuthenticateRemoteWiki( $this->credentials );
}

[ $result, $isFromCache ] = $this->doMakeHttpRequestFor( $query );
[ $result, $isFromCache, $failureReason ] = $this->doMakeHttpRequestFor( $query );

if ( $result === null ) {
// The response could not be fetched or decoded; make the failure
// visible instead of silently rendering an empty result.
$query->addErrors( [ wfMessage(
$failureReason === 'decode' ? 'seql-http-response-invalid' : 'seql-http-request-failed',
$this->httpRequestEndpoint
)->text() ] );

if ( $result === [] || $result === false || $result === null ) {
return $this->queryResultFactory->newEmptyQueryResult( $query );
}

if ( $result === [] ) {
return $this->queryResultFactory->newEmptyQueryResult( $query );
}

Expand Down Expand Up @@ -270,12 +281,27 @@ private function doMakeHttpRequestFor( $query ) {
$isFromCache = false;
$response = $this->fetchHttpResponse( $url );

if ( is_string( $response ) && $response !== '' ) {
// Only cache responses that at least decode as JSON so that a
// failed or corrupted fetch is retried on the next request
// instead of being served from the cache.
if ( is_string( $response ) && $response !== '' && is_array( json_decode( $response, true ) ) ) {
$this->cache->set( $key, $response, $this->httpResponseCacheLifetime );
}
}

return [ json_decode( $response ?? '', true ), $isFromCache ];
if ( $response === false || $response === null || $response === '' ) {
// The response could not be fetched at all (transport failure)
return [ null, $isFromCache, 'request' ];
}

$result = json_decode( $response, true );

if ( !is_array( $result ) ) {
// The response was fetched but is no valid JSON
return [ null, $isFromCache, 'decode' ];
}

return [ $result, $isFromCache, '' ];
}

private function fetchHttpResponse( $url ) {
Expand Down
48 changes: 39 additions & 9 deletions src/DynamicInterwikiPrefixLoader.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

namespace SEQL;

use Interwiki;

/**
* Allows to dynamically assign interwiki prefixes without having to
* create an interwiki table entry.
Expand All @@ -28,34 +30,62 @@ public function __construct( array $enabledExternalRepositoryEndpoints = [] ) {
}

/**
* @since 1.0
* @since 2.0.1
*
* @param string $prefix
*/
public function isEnabledPrefixForExternalRepository( $prefix ) {
return isset( $this->enabledExternalRepositoryEndpoints[$prefix] );
}

/**
* @since 2.0.1
*
* @param string $prefix
*
* @return Interwiki|null Interwiki instance for a known external
* repository endpoint or null when the prefix holds no such endpoint
*/
public function newInterwiki( $prefix ) {
if ( !$this->isEnabledPrefixForExternalRepository( $prefix ) ) {
return null;
}

[ $iw_url, $iw_api, $iw_local ] = $this->enabledExternalRepositoryEndpoints[$prefix];

return new Interwiki(
$prefix,
$iw_url,
$iw_api,
$prefix,
$iw_local
);
}

/**
* @since 1.0
* @deprecated since 2.0.1, use newInterwiki() instead. Kept for
* compatibility with the removed InterwikiLoadPrefix hook.
*
* @param string $prefix
* @param array &$interwiki
*
* @return bool
*/
public function tryToLoadIwMapForExternalRepository( $prefix, &$interwiki ) {
if ( !$this->isEnabledPrefixForExternalRepository( $prefix ) ) {
$interwikiObject = $this->newInterwiki( $prefix );

if ( $interwikiObject === null ) {
return true;
}

[ $iw_url, $iw_api, $iw_local ] = $this->enabledExternalRepositoryEndpoints[$prefix];

$interwiki = [
'iw_prefix' => $prefix,
'iw_url' => $iw_url,
'iw_api' => $iw_api,
'iw_wikiid' => $prefix,
'iw_local' => $iw_local,
'iw_trans' => false,
'iw_url' => $interwikiObject->getURL(),
'iw_api' => $interwikiObject->getAPI(),
'iw_wikiid' => $interwikiObject->getWikiID(),
'iw_local' => $interwikiObject->isLocal(),
'iw_trans' => $interwikiObject->isTranscludable()
];

return false;
Expand Down
8 changes: 0 additions & 8 deletions src/HookRegistry.php
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,6 @@ public function getHandlerFor( $name ) {
}

private function addCallbackHandlers( $options ) {
$dynamicInterwikiPrefixLoader = new DynamicInterwikiPrefixLoader(
$options['externalRepositoryEndpoints']
);

$this->handlers['InterwikiLoadPrefix'] = static function ( $prefix, &$interwiki ) use( $dynamicInterwikiPrefixLoader ) {
return $dynamicInterwikiPrefixLoader->tryToLoadIwMapForExternalRepository( $prefix, $interwiki );
};

/**
* Prevents ask parser function with "source" parameter defined from
* being executed outside of allowed namespaces. This supports transclusion too.
Expand Down
Loading
Loading