Repository navigation
chore(release): bump version to 6.2.41 (bundle: authoring API, D14 versions, MetadataExpert, additionalProperties merge) - #595
Merged
Conversation
Egeria replaces a map property whole, even with mergeUpdate true. A Create that found its element (and became an Update), or an Update with Merge Update true, sent only the keys its document gave, so every other key was removed: on laz a SavedQuery's refresh keys were wiped, and a CDO Report's search keys nearly were. - AsyncBaseCommandProcessor._prepare_additional_properties runs once before apply_changes() on an update of an existing element. When the update sends additionalProperties (the document gives Additional Properties, or the processor writes keys of its own: WRITES_ADDITIONAL_PROPERTIES, set on ReportProcessor), it reads the stored map and lays the document's keys over it; the merged map becomes the value every body builder reads. - A replace update (Merge Update false) still replaces, with a warning that names the stored keys it removes. - A merge update whose stored map can't be read (element read without its properties and a re-read by GUID fails) is refused in words (AdditionalPropertiesUnreadable), never sent as a replacement. - upsert_dashboard_asset merges with the existing Dashboard's map too, and its unchanged check compares only the keys given. A Dr.Egeria document has no way to clear one key (a JSON null is parsed to the string "null"); Merge Update false is the way to remove keys. Tests use the laz SavedQuery refresh keys and a 10-key CDO Report. 3.13 micro suite: 3,465 passed. Probe gate (88cf76b, PostgreSQL scratch, 757a5df0): PASS, identical to the baseline. Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
The Report fixture is now the stored additionalProperties of CocoPharma::Report::CDODashboard::MyNinetyDayPlanProgress as read on laz on 2026-11-12 (version 1, LIST): 10 string-valued keys, startFrom included, no metadataElementTypeName. A Create Report block writing only reportSpec, outputFormat TABLE and searchString sends all 10 keys with only outputFormat changed. Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…thout reading A Create Saved Query / Create Report with Additional Properties and no as-is element sends the document's map as given (plus the Report's own parameter keys), never reads a stored element, and is neither merged nor refused (Architect read-back suggestion, 2026-11-12). Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
16 commands wrapping pyegeria/omvs/metadata_expert.py: Create/Update/Delete Type Def, Create/Delete Enum Def, Count Metadata Elements, Publish Metadata Element, and Create/Update/Delete Instance, Link/Unlink Instances, Update Relationship, Classify/Reclassify/Declassify Instance. Adds processors, dispatcher registration, "Count" in STANDARD_VERBS, compact spec, generated report specs, Advanced templates and unit tests. Claude-Session: https://claude.ai/code/session_015pdMuS33mgrngrEv8FTfH1 Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
One typed authoring path for report specs, Reports and dashboard sheets (BRIEF-PYEGERIA-AUTHORING-API-PHASE1-2026-11-13), grown from report_spec_publish (Stage 1a). It never authenticates: every Egeria read uses the caller's client, every write the caller's runner, as the user. - models: SpecDraft / ReportDraft / SheetDraft (Draft union by kind), ValidationResult, Diff, AuthoringGates. - drafts: new_draft, new_from (prototype or template, derived_from recorded), DraftStore with EgeriaDraftStore (D16: the element itself, zoned private to the author, written as the author, prune_drafts(30) as the author, promotion clears the zone; zone name a parameter) and LocalDraftStore (tests, offline). - serialize: deterministic Dr.Egeria markdown; never a Parent ID; spec rows shared with build_publish_document; every Report parameter explicit. - validate: structural, semantic (target type and column attributes from the Properties-class catalog, load-time spec resolution, parameter names and types, placements resolve as the user), then Dr.Egeria validate via a runner. Never raises; says what it did not check. - preview: runs the draft's own FormatSet as the caller via the executor's run dict, never registered; capped and said; sheets resolved without fetching. - diff: breaking = removed column/placement, changed parameter type, changed target type (and a changed analytic routine). - lifecycle: load_held, history, publish_plan / withdraw_plan for all three kinds (consumer folders checked; restricted publish and built-ins refused in words), apply (run_document: confirm + digest). - versions (D14): a pinned spec version resolved as the user from the spec element's history, never through the fetched-name map. - D7: reportParameters and the camelCase copy are both written (switch kept, PYEGERIA_REPORT_CAMEL_PARAMS, default both). - runner: in-process Dr.Egeria runner as the client's user. - mcp_tools: authoring_* tools registered on pyegeria's MCP server; authoring_apply needs confirmed + digest + an as-user credential and refuses the server's service account. Tests: 59 new (CDO map round trip, Stage 1a golden, catalog-health sheet read-back replayed from a scratch capture, D7 consistency, D14, D16, MCP refusals, no Parent ID, preview side effects). Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…ateTime Per ANSWER-DRAFT-PRIVATE-ZONE-2026-11-13 (code-verified on the build laz runs: validateZoneAccess grants a zone named for a user to that user only; the search pushdown excludes other users' drafts): - EgeriaDraftStore zones each draft [<author userId>] and nothing else (no marker zone: a control defined for it later would expose every draft). - Drafts are marked on the element: a spec's contentStatus DRAFT, a Report's additionalProperties authoringState: draft (a sheet has no carrier yet). - A user id that is a GovernanceZone's name, or one of controlled_zones, is refused in words. - prune_drafts removes only marked drafts, by createTime, as the author; unmarked or undated elements are reported, never removed. - Promotion: Declassify Zone Membership, contentStatus ACTIVE / authoringState: published. Drafts are never filed into consumer folders. Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…yped properties, tests) - Mark all 16 commands idempotent:false and chat_allowed:false in the compact spec; the loader now passes both flags through. Validate output says the command is not idempotent. - PURGE / HARD_DELETE on Delete Instance and Unlink Instances are refused unless the new Confirm Delete GUID repeats the GUID; descriptions now say PURGE needs a prior soft delete. - Property values are coerced to the primitive type the instance type's definition (and its super types) declares, instead of guessed from the text: "007" stays a string. Map, array and enum properties, undeclared properties and dict/list values are refused with a message. - Processor-level tests through the real dispatcher: routing, SOFT_DELETE default, purge echo, Replace Properties, refusals, flags. Claude-Session: https://claude.ai/code/session_015pdMuS33mgrngrEv8FTfH1 Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…in search - pyegeria.core.element_shape.as_view_element: a raw OpenMetadataElement (elementGUID, elementProperties.propertyValueMap, classifications list) as the view-bean shape every reader expects (elementHeader with Egeria 6's named classification slots -- anchor, memento, zoneMembership, ... -- and flat, typed properties). A view bean is returned unchanged. - AsyncBaseCommandProcessor.fetch_element: its metadata-expert fallback returned the raw element as is, so the guid, qualifiedName (the ISSUE-59 duplicate check) and the Memento check read as absent, and the merge saw an unknown stored map. It now converts the element. - AssetCatalog.find_in_asset_domain: "*", None or "" is sent as ".*". findAssetsInDomain requires a searchString (OPEN-METADATA-400-022), and the shared find helper turned "*" into null. Tests use real elements read from laz (read only, 002a3ef1): the live openlineage.events Topic and its archived (Memento) twin. 3.13 micro suite: 3,472 passed. Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Files from probe/release-gate 3e80e35 (re-pinned to Egeria 002a3ef1, #9448, baseline 002a3ef1, known gaps trimmed to Project Scope). The probe branch carries its own history from 18178918, so its files are taken as they stand at 3e80e35 rather than merging the tip; none of them was changed on release/next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…one check fails closed Read-back 2026-11-13 F1, F2, F6: - A sheet has no draft marker, so it is no longer treated as marked. Prune deletes a sheet only when its zones are exactly [userId], its qualified name is Dashboard::..., and it belongs to no collection (a filed sheet is published even if its Declassify step didn't run). When its memberships can't be read it is kept. Every kind now needs zones == [userId] exactly. - Age by last update (falling back to creation), not creation. - `removed` lists only real deletes; a delete that found nothing goes to `not_deleted`; each kept-but-old element gets a reason in `kept_reasons`. - The GovernanceZone name check refuses when the lookup errors (only a not-found answer passes), and compares names case-insensitively. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…pdate can't do Read-back 2026-11-13 F3, F4, F4b: - F3: reportSpecVersion is recorded and honoured by validate/preview, but nothing that runs a placed Report reads it. publish_plan adds a step saying so, validate lists it in not_applied, preview notes it. The stale "a pinned version is refused" line in validate's doc is corrected. - F4: a sheet update adds and updates placements in place; it can't remove or reorder them. The plan says so instead of "in order", and diff's removed-placement and order changes carry the same words in `why`. - F4b: mcp_tools documents that the local draft owner is unauthenticated (fine for a per-user stdio server, not a shared one). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…y tile included Read-back 2026-11-13 point 1 / F5. The scratch replay has 6 placements; laz's sheet has 7 (the saved-query tile, LOCAL_DASHBOARDS_SAVED_QUERY_TILE). The new fixture is copied unchanged from the Portal's real-shape capture (egeria-workspaces PyegeriaWebHandler/tests/fixtures/ local_dashboards_real_shape_2026_11_12.json, read-only from laz): the sheet in reader shape and its 6 Report elements. Tests: 7 placements in order, serialize/parse round trip, the Saved Query Result Report's cached-count analytic parameters, and each Report's round trip through the serializer (only the common find-parameter defaults are added). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…blished version Owner decision 2026-11-13 (BRIEF-PYEGERIA-AUTHORING-D14-PUBLISHED-VERSIONS). A new version under a stable identity, not a new element per version: - Publication stamp: every publish writes publishedVersion (held stamp + 1), publishedAt and publishedBy into the element's own Create/Update block, so stamp and content land in one Egeria version. publishedAt is set when the plan runs; the digest leaves it out. Sheets: Create Dashboard Sheet now passes the document's Additional Properties to the Dashboard. - Exact target or refuse: async_publish_plan resolves the element by exact qualifiedName (a Report without one by exact display name), as the user, page 25 (B5). A failed read, two matches or a full page refuses; publish_plan without a resolved target refuses. - No silent overwrite: async_apply re-resolves the target and refuses when its GUID or publishedVersion changed since the plan. - Pins name publications: reportSpecVersion resolves to the publication entry of that publishedVersion; missing or unversioned is refused in words. The entry is the OLDEST one stamped N (the publish write), not the newest: an edit outside publishing keeps the stored map, so it carries the stamp too. - Edited since publication is a note in the plan, validate (not_applied) and preview, never an error; a zone-only change doesn't count. - An element without a stamp is unversioned: its first publish stamps 1. No backfill writes. - Promotion stamps in the Create/Update, before the Declassify. - Nothing in pyegeria.authoring deletes a published element: draft prune keeps it and the draft store's delete refuses. - The stamp, marker and pin keys are bookkeeping, never Report parameters (dashboard_resolver.report_from_element, report_draft_from_element). - MCP: a publish plan now needs as_user (it reads Egeria as the user). Tests: test_authoring_d14_published_versions.py (22, raw history in the laz OpenMetadataElement shape); existing authoring tests plan with a resolved target, and the Stage 1a golden is compared less the stamp row. Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…ort AUTHORING_KEYS
D14 read-back follow-up 1. The oldest entry stamped N is published version N
only if the whole history was read: a capped read could hide the publication
and leave a later edit (still stamped N) as the oldest returned.
- publication.async_read_history pages the element history (100 a page) until
Egeria answers an empty page (a short page isn't the last), up to 50 pages.
Running out of pages, or a failed read, returns the reason instead.
- A pin on such a history is refused in words ("history is too long to read
whole"); publish planning keeps going but says the edited-since check
wasn't done.
- AUTHORING_KEYS and STAMP_KEYS are exported from pyegeria.authoring, a
stable public path for the Portal's by-name report path.
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…etch or tier state Read-back 2026-11-13 F7. The existing preview test fakes exec_report_spec, so it only proves preview.py registers nothing. These run pyegeria's own executor (exec_report_spec -> _run_analytic_function) on the one-spec dict preview builds. The routine is a registered analytic function in an injected module that declares no Egeria client, so no server is needed, and EgeriaTech fails the test if anything builds a client. Checked: the registry, _FETCHED and _TIER are unchanged, the draft label is never registered, the routine ran once per preview with the caller's page size, and the 40 rows came back capped to 5 and said so. Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
A shallow copy shares the inner per-tier entries, so a mutation inside one would compare equal before and after (read-back of 86af9e2). Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Delete Type Def and Update Type Def are refused unless Confirm Type Def Name repeats Type Def Name exactly; Delete Enum Def likewise with Confirm Enum Def Name. The check runs before any call to Egeria, the refusal says the change is platform-wide and nothing was changed, and validate mode reports whether the echo is present and matching without calling Egeria. Create Type Def and Create Enum Def are additive and need no echo. Claude-Session: https://claude.ai/code/session_015pdMuS33mgrngrEv8FTfH1 Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
Delete Type Def, Update Type Def and Delete Enum Def now read the definition behind a supplied Type Def GUID / Enum Def GUID (a read only: Egeria has no read-by-GUID, so the definitions are listed and matched) and refuse, with no write, unless its name equals the Confirm ... Name echo exactly. The refusal names both the GUID's actual name and the echo; an unresolvable GUID is refused too. Validate mode reports the mismatch and writes nothing. Without this, a wrong GUID with the right echo changed a different type platform-wide. Claude-Session: https://claude.ai/code/session_015pdMuS33mgrngrEv8FTfH1 Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
…t belong to the echoed name Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bundled release for pyegeria 6.2.41. Each branch below was reviewed by itself, then integrated and run locally on laz, before this bundle.
What's in it
pyegeria.authoringfor report specs, Reports and dashboard sheets. It covers drafts, validate, preview-as-user, diff and publish, with MCP tools.AUTHORING_KEYScan be imported frompyegeria.authoring.Checks
tests/micro-testson this head: 3721 tests, 0 failures, 0 errors, 4 skipped.Follow-ups (6.2.42)
Signed-off-by: Dan Wolfson dan.wolfson@pdr-associates.com
🤖 Generated with Claude Code