diff --git a/docs/My-Egeria-Doc.md b/docs/My-Egeria-Doc.md index 7a0dc78b..1913fc7f 100644 --- a/docs/My-Egeria-Doc.md +++ b/docs/My-Egeria-Doc.md @@ -31,8 +31,9 @@ All source files named here are in `my_egeria/my_egeria/`. | `team_roles_handler.py`, `MyTeamScreen.py` | `TeamRolesMixin` and the team roster screen | | `bookmarks_handler.py`, `MyBookMarksScreen.py` | `BookmarksMixin` (bookmarks kept in a personal collection) and the bookmarks screen | | `network_handler.py`, `MyNetworkScreen.py` | `NetworkMixin` (peers linked by `Peer` relationships), `MyNetworkScreen` and `ConfirmRemovePeerScreen` | -| `community_handler.py`, `CommunitiesScreen.py` | `CommunitiesMixin` (community membership through `AssignmentScope` relationships), `CommunitiesScreen`, `CreateCommunityScreen` and `ConfirmLeaveCommunityScreen` | -| `CommunityScreen.py` | `CommunityScreen` (one community's members, blog and comments), `NewBlogPostScreen` and `AddCommunityCommentScreen` | +| `community_handler.py`, `CommunitiesScreen.py` | `CommunitiesMixin` (community membership through `AssignmentScope` relationships, resources, feedback on posts and resources), `CommunitiesScreen`, `CreateCommunityScreen` and `ConfirmLeaveCommunityScreen` | +| `CommunityScreen.py` | `CommunityScreen` (one community's members, blog, resources and comments, and joining it), `NewBlogPostScreen`, `AddCommunityCommentScreen`, `ElementFeedbackScreen` (likes, ratings and comments on a blog post or resource) and `RateElementScreen` | +| `paging_support.py` | Egeria's paging contract as the handlers use it: `read_all`/`PagedRows`, the "at least" and "none" words, and the found / absent / unknown guards that writes check first (see [Paging and write guards](#paging-and-write-guards)) | | `UserIdentitiesScreen.py` | Read-only user identity table | | `StatusScreen.py` | Result message with copy-GUID | | `profile_utils.py` | Environment check, Egeria access check (`check_egeria_access`), the shared screen heading (`set_screen_heading`) and data clean-up helpers | @@ -48,7 +49,7 @@ classDiagram class App class MyProfileApp { SCREENS - BINDINGS: q, r, ctrl+f, f3 + BINDINGS: q, ctrl+r, ctrl+f, f3 on_mount() mainline(splash_return) change_user() / change_user_callback() @@ -80,14 +81,18 @@ classDiagram get_peer_details() } class CommunitiesMixin { - show_communities() + show_communities(find) + open_community_row() list_my_communities() search_communities() get_community_members() + get_community_resources() create_community() join_community() / leave_community() get_community_blog_posts() / add_community_blog_post() get_community_comments() / add_community_comment() + get_element_feedback() / add_element_comment() + like_element() / unlike_element() / rate_element() } App <|-- MyProfileApp TechTypesMixin <|-- MyProfileApp @@ -102,17 +107,28 @@ classDiagram - **App bindings:** - `q` quits with `200`. - - `r` re-runs the profile load and table population. + - `ctrl+r` re-runs the profile load and table population. - `ctrl+f` (feedback) and `f3` (feedback log) are *priority* bindings. Textual ignores ordinary App bindings while a modal screen is active, and priority bindings get round that. - **`check_action`** hides and disables `view_feedback_log` unless the current user is the feedback owner (`garygeeke`). - **Screen registration:** most screens are registered by name in `SCREENS`. `EditCollectionsScreen`, `AddCollectionScreen`, `AddUserIdentityScreen`, `FeedbackScreen`, `FeedbackLogScreen` and `ConfirmDeleteScreen` are pushed as instances instead. -- **Mixins and `@on`:** the mixins are plain Python classes, not Textual message pumps. Textual only collects `@on(...)`-decorated handlers from classes built with its own metaclass, so an `@on` handler defined on a mixin is never dispatched. Put any `@on` handler on `MyProfileApp` itself and have it delegate to the mixin, as `_on_roles_row_selected` does. Handlers found by name (`on_worker_state_changed`, `action_*`) and callbacks passed to `push_screen` work on mixins. +- **Mixins and `@on`:** the mixins are plain Python classes, not Textual message pumps. Textual only collects `@on(...)`-decorated handlers from classes built with its own metaclass, so an `@on` handler defined on a mixin is never dispatched. Put any `@on` handler on `MyProfileApp` itself and have it delegate to the mixin, as `_on_roles_row_selected` and `_on_associations_row_selected` do. Handlers found by name (`on_worker_state_changed`, `action_*`) and callbacks passed to `push_screen` work on mixins. - **Screen headings:** every screen with a header calls `set_screen_heading(screen, context)` from `on_mount`. It sets the title to `Egeria - ` (just `Egeria` on the main screen) and the sub-title to `User: , Karma Points: `, read from the app's `user_name` and `karma_points`. The app passes `app=self` when it sets the main screen's heading itself, because `screen.app` isn't always available then. It does that in `_process_profile_data`, once the Karma Points are known, and in `change_user`. Screens opened later read the current values when they mount. - **Diagnostic logging is off.** The app's `self.log(...)` calls are commented out across every module, so the Textual developer console shows only Textual's own messages. Uncomment the ones you need to trace a problem. Where commenting one out would have left a block empty, a `pass` was added. The two "Unexpected table name" messages in `elements_crud_handler.py` are still logged, and also shown to the user with `notify`. - **Entry point:** `main()` runs the app. It is the target of the `my_profile` console script. --- +## Paging and write guards + +Egeria returns a list one page at a time. An empty page can mean the page was filtered empty rather than the end, and on some paths even a null page isn't reliably the end. `paging_support.py` (from pyegeria 6.2.42) wraps pyegeria's pager (`pyegeria.core.paging`) so the handlers follow one rule: never treat one page as the whole set, and never show a total or "none" unless Egeria confirmed the end. + +- **Reading a list.** `read_all(fetch, method=...)` pages `fetch(start, size)` (up to `DISPLAY_MAX_PAGES`, 20) and returns a `PageResult`. `paged_rows(rows, read, what)` wraps the rows a handler builds as `PagedRows`, a plain list that also carries `complete` and, when the end wasn't confirmed, a `note` ("Showing at least N ..."). Screens show `count_text(rows)` ("N" or "at least N") and `none_words(...)` ("No X found", or "No X found in what Egeria returned; Egeria didn't confirm there are no more"). +- **Which reads can be confirmed.** pyegeria's pager decides per method (`TRUSTED_END_METHODS`, `GATED_TRUSTED_METHODS` in `pyegeria.core.paging`). A method in neither pages as "suspect", which is never confirmed. Of the reads My Profile makes, only My Network's `get_related_elements` can be confirmed, and only on a platform carrying odpi/egeria #9454 (Egeria 6.3.0 or later). Everything else is suspect: `find_communities`, `find_collections`, `find_actor_profiles`, `get_collection_members`, and the attached-element reads (comments, likes, ratings, note logs, blog notes). So on most screens the list carries an "at least" note, which the screen shows as a warning, and an empty list says "none found in what Egeria returned" rather than "none". +- **Guards before a write, by qualified name.** Before creating something that must be unique (the bookmarks collection, the feedback log, a community, a community's blog), the handler asks Egeria's exact unique-name read through `by_unique_name` / `async_by_unique_name`. **found** reuses the element; **absent** (Egeria's specific not-found) lets the create go ahead; **unknown** (any other answer or error) refuses with `unique_refusal(...)` words ("Couldn't confirm whether Egeria already holds ... Nothing was created; try again later.") and writes nothing. A handler that refuses inside a write raises `LookupUnconfirmed` and reports its message as a warning. +- **Guards before a write, by GUID.** "Is this already a member/peer?" checks (adding a bookmark, adding someone to My Network) page with `lookup(...)`, which also answers found / absent / unknown; only absent goes on to write, and unknown is refused with `refusal(...)` words. + +--- + ## Startup and Profile Loading ```mermaid @@ -173,8 +189,8 @@ It returns `200` on success and `401` on failure. | Table id | Source | Notes | | :--- | :--- | :--- | -| `associations_table` | Communities from `My-User-MD` | | -| `my_collections_table` | `Egeria.find_collections("*", JSON)`, keeping those whose `elementHeader.versions.createdBy` is the current user, minus the bookmarks collection (`_load_my_collections`) | | +| `associations_table` | Communities from `My-User-MD` | `Enter` on a row opens that community: `_on_associations_row_selected` → `open_community_row` → `CommunityScreen` | +| `my_collections_table` | `Egeria.find_collections("*", JSON)`, paged with `read_all`, keeping those whose `elementHeader.versions.createdBy` is the current user, minus the bookmarks collection (`_load_my_collections`) | | | `roles_table` | Roles from `My-User-MD` | | | `teams_table` | Teams from `My-User-MD` | | | `blogs_table`, `journal_table` | Note Logs from `My-User-MD` | | @@ -185,7 +201,7 @@ It returns `200` on success and `401` on failure. `communities_table` is not on the main screen (communities show in User Associations); `_populate_tables_sync` skips any table that isn't composed. -`MainScreen` tracks the selected table and row through focus, highlight, select and cell events. It uses them for `ctrl+t` (edit table), `ctrl+a` (add comment) and `ctrl+s` (show comments). +`MainScreen` tracks the selected table and row through focus, highlight, select and cell events. It uses them for `ctrl+t` (edit table), `ctrl+n` (add comment), `ctrl+o` (show comments), `ctrl+k` (bookmark the row) and `ctrl+b` (bookmarks), and `Enter` on a User Associations row opens that community. ### Other Functions menu dispatch (`handle_option_selected`) @@ -199,6 +215,7 @@ It returns `200` on success and `401` on failure. | User Bookmarks | `show_my_bookmarks()` | | My Network | `show_my_network()` | | Work with Communities | `show_communities()` | +| Find Community | `show_communities(find=True)`: the same screen, with focus on the search input | | Change User | `ChangeUserScreen()`, callback `change_user_callback` | | Leave Feedback | `action_feedback()` | | Feedback Log | `action_view_feedback_log()` (owner only) | @@ -227,8 +244,8 @@ These are `EDIT_TABLE_ROUTES`, `ADD_TABLE_ROUTES` and `DELETE_METHODS`. Blogs, j ### Edit screens (`BaseEditScreen`) - **On mount:** each subclass names its `SOURCE_TABLE`, and the screen copies that table's columns and rows from the main screen. -- **`a` (add row):** calls `app.add_to_tables(SOURCE_TABLE, row)`, which pushes the Add screen from `ADD_TABLE_ROUTES`. -- **`d` (delete row):** takes the GUID from the row's last column and pushes `ConfirmDeleteScreen`. If the user confirms, it calls `app.delete_element(SOURCE_TABLE, guid)`, which looks up `DELETE_METHODS` and calls Egeria. The row is removed only if that returns `True`. +- **`ctrl+n` (add row):** calls `app.add_to_tables(SOURCE_TABLE, row)`, which pushes the Add screen from `ADD_TABLE_ROUTES`. +- **`ctrl+d` (delete row):** takes the GUID from the row's last column and pushes `ConfirmDeleteScreen`. If the user confirms, it calls `app.delete_element(SOURCE_TABLE, guid)`, which looks up `DELETE_METHODS` and calls Egeria. The row is removed only if that returns `True`. - **`Escape`:** dismisses with `[(row_key, row_values), ...]`. The table's edit callback then writes those rows back to the main table through `_write_back_rows()`. ### Add screens @@ -253,7 +270,7 @@ class AddCollectionScreen(BaseAddScreen): - the `Egeria` client lifecycle, including bearer token, session close and error notices; - the optional link to the user's profile, skipped with a warning if the profile GUID is unknown; - clearing the form after a successful add (a failed add keeps the input so it can be corrected); -- the Add, Quit and `ctrl+a` controls. +- the Add button (`ctrl+s`) and `Escape` to go back. The screen stays open for repeated adds and dismisses with `200`. @@ -315,7 +332,7 @@ sequenceDiagram `EditProfileScreen` pre-fills the person properties and calls `Egeria.update_actor_profile(user_GUID, UpdateElementRequestBody{PersonProperties})`. It returns `200` on success and `401` on failure. -Its `ctrl+c`, `ctrl+i`, `ctrl+r` and `ctrl+t` keys return `"community"`, `"identity"`, `"role"` or `"team"`. `edit_profile_callback` then opens the matching edit screen. +Its `ctrl+g`, `ctrl+l`, `ctrl+r` and `ctrl+t` keys return `"community"`, `"identity"`, `"role"` or `"team"`. `edit_profile_callback` then opens the matching edit screen. --- @@ -323,29 +340,29 @@ Its `ctrl+c`, `ctrl+i`, `ctrl+r` and `ctrl+t` keys return `"community"`, `"ident There are two ways to comment. -**1. From the main screen (`ctrl+a`).** +**1. From the main screen (`ctrl+n`).** 1. `app.add_comment(table, row)` finds the row's GUID column, falling back to a "Qualified Name" column. 2. It pushes `AddCommentScreen`, which dismisses with `[comment, type, guid]`. 3. `add_comment_callback` calls `Egeria.add_comment_to_element(guid, comment, TYPE)`. -**2. From the comments screen (`ctrl+s`).** `ShowCommentsScreen` handles both viewing comments and replying to them: +**2. From the comments screen (`ctrl+o`).** `ShowCommentsScreen` handles both viewing comments and replying to them: ```mermaid flowchart TD - MAIN["MainScreen: select row, ctrl+s"] --> SHOW["show_comments(table, row)"] + MAIN["MainScreen: select row, ctrl+o"] --> SHOW["show_comments(table, row)"] SHOW --> SCS["ShowCommentsScreen(table, row,
selected_comment_guid=None)"] SCS --> LOAD["exec_report_spec('Comment-by-Element',
element_guid = row GUID or comment GUID)"] LOAD --> LIST["Comment table: Comment, GUID, Qualified Name, Name"] - LIST -- "ctrl+a" --> ADD["inline comment + type inputs →
EgeriaTech.add_comment_to_element()"] + LIST -- "ctrl+n" --> ADD["inline comment + type inputs →
EgeriaTech.add_comment_to_element()"] ADD -- "dismiss(200)" --> CB - LIST -- "ctrl+r on a comment" --> RESP["dismiss([250, comment_guid])"] + LIST -- "ctrl+o on a comment" --> RESP["dismiss([250, comment_guid])"] RESP --> CB["show_comments_callback"] CB -- "[250, guid]" --> SHOW2["show_comments(selected_comment_guid=guid)"] SHOW2 --> SCS CB -- "200 / None" --> MAIN ``` -A response is a comment whose anchor is another comment. Each `ctrl+r` re-opens the screen with the selected comment as its target, so threads can nest to any depth. `q` on the comments screen calls `app.pop_screen`. +A response is a comment whose anchor is another comment. Each `ctrl+o` re-opens the screen with the highlighted comment as its target, and `ctrl+n` there adds a response, so threads can nest to any depth. `q` and `Escape` on the comments screen call `app.pop_screen`. --- @@ -366,9 +383,11 @@ sequenceDiagram App->>FS: push_screen(FeedbackScreen(screen, title)) User->>FS: category + text, Submit (ctrl+s) FS-->>App: {screen, screen_title, category, text} - App->>Egeria: _async_get_note_logs_by_name("NoteLog::garygeeke::My-Profile-App-Feedback") - alt log missing + App->>Egeria: unique-name read of "NoteLog::garygeeke::My-Profile-App-Feedback" + alt absent (Egeria's not-found) App->>Egeria: _async_create_note_log(...) + else unknown + App-->>User: warning, nothing written end App->>Egeria: _async_create_note(log_guid, note body) ``` @@ -379,7 +398,9 @@ Each note records: - `description`: the feedback text - `additionalProperties`: `category`, `screen`, `screenTitle`, `submittedBy` and `submittedAt` -**Viewing the log.** `F3` or "Feedback Log" reads the notes with `_async_get_notes_for_note_log`, sorts them newest first and shows them in `FeedbackLogScreen`. +**Finding the log.** The log is found by Egeria's exact unique-name read (`async_by_unique_name`) and created only when that read says not-found. On any other answer the feedback isn't saved, and the user gets the "Couldn't confirm ..." warning, because a second log would split the feedback. + +**Viewing the log.** `F3` or "Feedback Log" pages the notes with `_async_get_notes_for_note_log` (`async_read_all`), sorts them newest first and shows them in `FeedbackLogScreen`. The heading shows the count as "N entries", or "at least N entries" when Egeria didn't confirm the end. **Privacy.** Only the owner (`FEEDBACK_LOG_OWNER = "garygeeke"`) can open the log. The app enforces this, not Egeria. @@ -454,11 +475,11 @@ Only digital products have real sample data. For other rows the sample view show 2. calls `link_agreement_item(subscription, item)` to link the subscription to the chosen item; 3. calls `link_subscriber(profile, subscription)` to link the user's profile as subscriber. If a link fails, the subscription is kept and the user gets a warning naming what couldn't be linked. -- **Viewing subscriptions.** `ViewSubscriptionsScreen` calls `Egeria.find_collections(search_string="*", metadata_element_type_name="DigitalSubscription", JSON)`. It lists those the user created: name, status, description and GUID. +- **Viewing subscriptions.** `ViewSubscriptionsScreen` pages `Egeria.find_collections(search_string="*", metadata_element_type_name="DigitalSubscription", JSON)` with `read_all`. It lists those the user created: name, status, description and GUID. ### Glossary term search -`SearchForTermScreen` runs `exec_report_spec("Glossary-Terms", output_format="MD", search_string=term)` and renders the Markdown. `t` on `ShopForDataScreen` returns `[201]`, which opens it. Going back from it (`g`, result `201`) re-runs `handle_shop_for_data_option()`, so the catalog tables are reloaded rather than shown empty. +`SearchForTermScreen` runs `exec_report_spec("Glossary-Terms", output_format="MD", search_string=term)` and renders the Markdown. `ctrl+g` on `ShopForDataScreen` returns `[201]`, which opens it. Going back from it (`g`, result `201`) re-runs `handle_shop_for_data_option()`, so the catalog tables are reloaded rather than shown empty. --- @@ -510,18 +531,18 @@ pyegeria has no favourites API, so `BookmarksMixin` (`bookmarks_handler.py`) kee | Function | What it does | | :--- | :--- | -| `list_my_bookmarks()` | `get_collections_by_name(qualified name)` → `get_collection_members(guid, JSON)` → `(name, type, GUID)` rows; `[]` if the collection doesn't exist yet, `None` if Egeria can't be reached | +| `list_my_bookmarks()` | Finds the collection with Egeria's unique-name read (`by_unique_name`), then pages `get_collection_members(guid, JSON, graph_query_depth=0)` → `(name, type, GUID)` rows as `PagedRows`; `[]` if the collection doesn't exist yet, `None` if Egeria can't be reached | | `show_my_bookmarks()` | Pushes `MyBookMarksScreen(rows)` | -| `add_my_bookmark(guid)` | Creates the collection on first use (`create_collection`), then `add_to_collection(collection, guid)` | -| `delete_my_bookmark(guid)` | `remove_from_collection(collection, guid)` | -| `bookmark_table_row(table, row_key)` | Reads the row's GUID column (labelled `GUID` or ending ` GUID`) with `profile_utils.row_identity()`. If there's no GUID, it resolves the row's `Qualified Name` with `get_guid_for_name`. Then it calls `add_my_bookmark`. Used by `ctrl+k` on `MainScreen` and `k` on `ShopForDataScreen`. | +| `add_my_bookmark(guid)` | Creates the collection on first use (`create_collection`), only when the unique-name read says not-found. Checks with `lookup` that the element isn't already a member: found is a "already bookmarked" warning, unknown is refused, and only absent goes on to `add_to_collection(collection, guid)` | +| `delete_my_bookmark(guid)` | `remove_from_collection(collection, guid)`; if the collection's unique-name read is neither found nor not-found, nothing is removed and the user is told why | +| `bookmark_table_row(table, row_key)` | Reads the row's GUID column (labelled `GUID` or ending ` GUID`) with `profile_utils.row_identity()`. If there's no GUID, it resolves the row's `Qualified Name` with `get_guid_for_name`. Then it calls `add_my_bookmark`. Used by `ctrl+k` on `MainScreen` and on `ShopForDataScreen`. | `MyBookMarksScreen` lists the rows: - `ctrl+n` adds a bookmark by GUID; -- `d` removes the highlighted bookmark; +- `ctrl+d` removes the highlighted bookmark; - `q` closes the screen. -After each change it re-reads the list through the app. The lookup matches on the exact qualified name, so another user's "My Bookmarks" collection is never used. `profile_utils.element_summary()` flattens raw JSON elements for both bookmarks and My Collections, including member results wrapped in `relatedElement`. +After each change it re-reads the list through the app. The lookup is Egeria's exact unique-name read, so another user's "My Bookmarks" collection is never used. `profile_utils.element_summary()` flattens raw JSON elements for both bookmarks and My Collections, including member results wrapped in `relatedElement`. --- @@ -539,9 +560,9 @@ sequenceDiagram participant EG as Egeria User->>App: Other Functions → My Network - App->>EG: get_related_elements(user_GUID, "Peer") + App->>EG: get_related_elements(user_GUID, "Peer"), paged App->>MNS: push(peer rows) - opt view details (v) + opt view details (ctrl+o) MNS->>App: get_peer_details(peer GUID) App->>EG: get_actor_profile_by_guid(JSON) App->>App: _generate_my_profile_output(DICT, "My-User") @@ -549,13 +570,14 @@ sequenceDiagram end opt add (search, select, Add to Network) MNS->>App: search_people(name) - App->>EG: find_actor_profiles(DICT, "User-Identities", page_size=10) + App->>EG: find_actor_profiles(DICT, "User-Identities"), paged 25 at a time App-->>MNS: (name, GUID, qualified name) rows MNS->>App: link_peer(GUID, nickname, description) - App->>EG: link_peer_person(user_GUID, GUID, PeerProperties) + App->>EG: get_related_elements(user_GUID, "Peer"): is GUID already a peer? + App->>EG: link_peer_person(user_GUID, GUID, PeerProperties) only if not MNS->>App: list_my_peers() end - opt remove (d) + opt remove (ctrl+d) MNS->>Confirm: push(peer name) Confirm-->>MNS: True / False MNS->>App: unlink_peer(GUID) @@ -567,9 +589,9 @@ sequenceDiagram | Function | What it does | | :--- | :--- | | `show_my_network()` | Refuses (with a notice) if `user_GUID` isn't set yet, i.e. the profile hasn't loaded; otherwise pushes `MyNetworkScreen(list_my_peers())` | -| `list_my_peers()` | `get_related_elements(user_GUID, "Peer")`, keeping only elements whose properties are `PersonProperties`. Returns `(qualified name, display name, full name, pronouns, GUID, description)` rows; the GUID comes from the peer's `elementHeader` and the description from `relationshipProperties`. `[]` when Egeria answers "No elements found", `None` if Egeria can't be reached | -| `search_people(search_string)` | `find_actor_profiles(..., ignore_case=True, report_spec="User-Identities", page_size=10)` → `(display name, GUID, qualified name)` rows, excluding the current user. The screen also drops anyone already in the network | -| `link_peer(guid, nickname, description)` | `link_peer_person(user_GUID, guid, NewRelationshipRequestBody)` with `PeerProperties` (`peer_relationship_name` = nickname, `description`) | +| `list_my_peers()` | `get_related_elements(user_GUID, "Peer")`, paged with `read_all` (confirmable on Egeria 6.3.0 or later), keeping only elements whose properties are `PersonProperties`. Returns `(qualified name, display name, full name, pronouns, GUID, description)` rows; the GUID comes from the peer's `elementHeader` and the description from `relationshipProperties`. `[]` when Egeria finds none, `None` if Egeria can't be reached | +| `search_people(search_string)` | `find_actor_profiles(..., ignore_case=True, report_spec="User-Identities", query_depth=0)`, paged `PEOPLE_PAGE` (25) at a time with `read_all` → `(display name, GUID, qualified name)` rows, excluding the current user. The screen also drops anyone already in the network | +| `link_peer(guid, nickname, description)` | First checks with `lookup` over the user's `Peer` links that the person isn't already a peer, because the search only hides peers as far as the network list was read. Found is a warning and unknown is refused; only absent goes on to `link_peer_person(user_GUID, guid, NewRelationshipRequestBody)` with `PeerProperties` (`peer_relationship_name` = nickname, `description`) | | `unlink_peer(guid)` | `unlink_peer_person(user_GUID, guid, DeleteRelationshipRequestBody)`, using the same GUID order as `link_peer_person` | | `get_peer_details(guid)` | `get_actor_profile_by_guid(guid, JSON)`, then `_generate_my_profile_output(..., report_spec="My-User")`. Returns the `Contact Methods` and `Communities` lists, or `None` with a notice if there are no details | @@ -582,7 +604,12 @@ sequenceDiagram ## Work with Communities -A user belongs to a community when an actor is linked to it by an `AssignmentScope` relationship, whose `assignmentType` says how they take part. The actor is either the user's own profile or a role they are appointed to. A community's blog is a `NoteLog` attached to it holding `BlogEntry` notes, and comments are attached to the community directly. `CommunitiesMixin` (`community_handler.py`) makes the Egeria calls; `CommunitiesScreen` and `CommunityScreen` only drive it through `self.app`. Each call builds a fresh `Egeria` client with a bearer token (`_community_client()`) and closes its session afterwards. +A user belongs to a community when an actor is linked to it by an `AssignmentScope` relationship, whose `assignmentType` says how they take part. The actor is either the user's own profile or a role they are appointed to. A community's blog is a `NoteLog` attached to it holding `BlogEntry` notes, and comments are attached to the community directly. Its resources are the elements it lists with `ResourceList` relationships and the elements `ScopedBy` it. `CommunitiesMixin` (`community_handler.py`) makes the Egeria calls; `CommunitiesScreen`, `CommunityScreen` and `ElementFeedbackScreen` only drive it through `self.app`. Each call builds a fresh `Egeria` client with a bearer token (`_community_client()`) and closes its session afterwards. + +There are three ways in: +- **Work with Communities** on the Other Functions menu: `show_communities()`. +- **Find Community** on the same menu: `show_communities(find=True)`, which opens the same screen with focus on the search input (`CommunitiesScreen(..., focus_search=True)`). +- **`Enter` on a User Associations row** on the main screen: `_on_associations_row_selected` → `open_community_row(table, row_key)`, which reads the row's name and GUID columns and pushes `CommunityScreen` directly, with `community_callback`. ```mermaid sequenceDiagram @@ -594,13 +621,14 @@ sequenceDiagram participant Confirm as ConfirmLeaveCommunityScreen participant EG as Egeria - User->>App: Other Functions → Work with Communities + User->>App: Other Functions → Work with Communities / Find Community App->>EG: get_my_profile(JSON) - App->>CS: push(my community rows) + App->>CS: push(my community rows, focus_search) CS->>App: search_communities("*") - App->>EG: find_communities(JSON, graph_query_depth=0) + App->>EG: find_communities(JSON, graph_query_depth=0), paged opt open a community (Enter / ctrl+o) — see the next diagram CS->>CS: push CommunityScreen(GUID, name) + Note over CS: on return, re-read both lists if the user joined from inside it end opt join (ctrl+n) CS->>App: join_community(GUID, assignment type) @@ -611,8 +639,8 @@ sequenceDiagram CS->>Create: push Create-->>CS: {name, description, mission, assignment type} / None CS->>App: create_community(name, description, mission, assignment type) - App->>EG: get_communities_by_name(qualified name) - App->>EG: create_community(NewElementRequestBody, CommunityProperties) + App->>EG: unique-name read of the qualified name + App->>EG: create_community(NewElementRequestBody, CommunityProperties) only on not-found App->>EG: link_assignment_scope(new GUID, user_GUID, AssignmentScopeProperties) unless "Don't join" CS->>App: list_my_communities() and search again, then highlight the new community end @@ -627,22 +655,24 @@ sequenceDiagram | Function | What it does | | :--- | :--- | -| `show_communities()` | Refuses (with a notice) if `user_GUID` isn't set yet; otherwise pushes `CommunitiesScreen(list_my_communities())` | +| `show_communities(find=False)` | Refuses (with a notice) if `user_GUID` isn't set yet; otherwise pushes `CommunitiesScreen(list_my_communities(), focus_search=find)` | +| `open_community_row(table, row_key)` | Opens the community in a main-screen User Associations row `(type, name, description, GUID)`; warns and does nothing if the row has no GUID | | `list_my_communities()` | `get_my_profile(JSON)`, then reads the profile's own `assignmentScope` links and each `performsRoles` appointment's `nestedElements`, keeping only `Community` elements. Returns one `(name, assignment types, linked through, description, GUID)` row per community; "linked through" is `Your profile` and/or `Role: `. `None` if Egeria can't be reached | -| `search_communities(search_string)` | `find_communities(..., ignore_case=True, graph_query_depth=0)` → `(name, description, qualified name, GUID)` rows. The screen drops communities the user already belongs to | -| `join_community(guid, assignment_type)` | `link_assignment_scope(guid, user_GUID, NewRelationshipRequestBody)` with `AssignmentScopeProperties`. The screen offers only `Contributor` and `Observer` | -| `create_community(name, description, mission, assignment_type)` | Builds the qualified name with `__create_qualified_name__("Community", name)` and refuses (a warning, returning `None`) if `get_communities_by_name` already finds it, because Egeria answers a duplicate qualified name only with a server error. Otherwise `create_community(NewElementRequestBody)` with `CommunityProperties` (`mission` only when given), then, unless `assignment_type` is empty, links the user's profile exactly as `join_community` does. Returns the new GUID; a failed link is a warning and still returns the GUID, because the community exists. The form offers `Leader` (the default), `Contributor`, `Observer` and "Don't join" (`NOT_JOINING`, an empty string) | +| `search_communities(search_string)` | Pages `find_communities(..., ignore_case=True, graph_query_depth=0)` with `read_all` → `(name, description, qualified name, GUID)` rows as `PagedRows`. The screen drops communities the user already belongs to, and shows the "at least" note, or "none found in what Egeria returned", when the end wasn't confirmed | +| `join_community(guid, assignment_type)` | `link_assignment_scope(guid, user_GUID, NewRelationshipRequestBody)` with `AssignmentScopeProperties`. Both screens offer only `Contributor` and `Observer` (`JOIN_ASSIGNMENT_TYPES`) | +| `create_community(name, description, mission, assignment_type)` | Builds the qualified name with `__create_qualified_name__("Community", name)` and asks Egeria's unique-name read (`by_unique_name`) whether it exists, because Egeria answers a duplicate qualified name only with a server error. Found is a "find it below to join it" warning; an unclear answer is refused with the "Couldn't confirm ..." words. Both return `None` and create nothing. On not-found it calls `create_community(NewElementRequestBody)` with `CommunityProperties` (`mission` only when given), then, unless `assignment_type` is empty, links the user's profile exactly as `join_community` does. Returns the new GUID; a failed link is a warning and still returns the GUID, because the community exists. The form offers `Leader` (the default), `Contributor`, `Observer` and "Don't join" (`NOT_JOINING`, an empty string) | | `leave_community(guid)` | `detach_assignment_scope(guid, user_GUID, DeleteRelationshipRequestBody)`, using the same GUID order as `link_assignment_scope` (scope first, actor second) | - **Leaving only unlinks the profile.** The screen refuses to leave a community reached only through a role, because the role may be shared and belongs to whoever manages it. - **Refresh.** After a successful create, join or leave the screen re-reads the user's communities and repeats the last search. After a create it also moves the cursor to the new community, in whichever list it appears. +- **The main screen.** A successful join, leave, or create-and-join sets `communities_changed`. When the user returns to the main screen, `communities_callback` or `community_callback` sees the flag, clears it and calls `call_later(action_refresh)`, so User Associations is current. Nothing is refreshed if no membership changed. - **Two ways to create a community.** The User Associations edit screen's `AddCommunityScreen` also creates communities, but doesn't check for a duplicate name or link the user. Work with Communities' **Create Community** does both. - **Errors.** Every Egeria failure is reported with `notify(...)` and the screen stays open; none of them exit the app. -- **Return code.** `CommunitiesScreen` dismisses with `200` on `q`; `communities_callback` just logs it. +- **Return code.** `CommunitiesScreen` dismisses with `200` on `q`. -### A community: members, blog and comments +### A community: members, blog, resources and comments -`CommunityScreen(guid, name)` loads the three lists when it mounts and reloads a list after a successful add. `ctrl+n` opens `AddCommunityCommentScreen` when the comments table has focus, otherwise `NewBlogPostScreen`; each form dismisses with a dict, or `None` on cancel. +`CommunityScreen(guid, name)` loads its four lists when it mounts and reloads a list after a successful add. `ctrl+n` opens `AddCommunityCommentScreen` when the comments table has focus, otherwise `NewBlogPostScreen`; each form dismisses with a dict, or `None` on cancel. `Enter` on a blog post or resource, or `ctrl+o` (the resources table when it has focus, otherwise the highlighted post), opens [`ElementFeedbackScreen`](#feedback-on-a-blog-post-or-resource) for it. ```mermaid sequenceDiagram @@ -656,14 +686,20 @@ sequenceDiagram App->>EG: get_community_by_guid(JSON, graph_query_depth=1) App->>EG: get_actor_role_by_guid(role, include_only_relationships=[PersonRoleAppointment]) for each role CoS->>App: get_community_blog_posts(GUID) - App->>EG: get_attached_note_logs(GUID) - App->>EG: get_notes_for_note_log(log, "BlogEntry") for each log + App->>EG: get_attached_note_logs(GUID), paged + App->>EG: get_notes_for_note_log(log, "BlogEntry") for each log, paged + CoS->>App: get_community_resources(GUID) + App->>EG: get_community_by_guid(JSON, graph_query_depth=1, include_only_relationships=[ResourceList, ScopedBy]) CoS->>App: get_community_comments(GUID) - App->>EG: get_attached_comments(GUID) + App->>EG: get_attached_comments(GUID), paged + opt join (ctrl+w), shown only when the user isn't a member + CoS->>App: join_community(GUID, assignment type) + CoS->>App: get_community_members(GUID) + end opt new blog post (NewBlogPostScreen) CoS->>App: add_community_blog_post(GUID, name, title, text) - App->>EG: get_attached_note_logs(GUID) - App->>EG: create_note_log(GUID, NewAttachmentRequestBody) if the blog doesn't exist + App->>EG: unique-name read of NoteLog::::CommunityBlog + App->>EG: create_note_log(GUID, NewAttachmentRequestBody) only on not-found App->>EG: create_note(log, BlogEntryProperties) end opt add comment (AddCommunityCommentScreen) @@ -675,24 +711,44 @@ sequenceDiagram | Function | What it does | | :--- | :--- | | `get_community_members(guid)` | People only. Reads `assignedActors` from `get_community_by_guid(guid, JSON, graph_query_depth=1)`. A `Person` actor is a direct member. For an actor whose type is or inherits `ActorRole` (checked through `superTypeNames`), it calls `get_actor_role_by_guid(..., graph_query_depth=1, include_only_relationships=["PersonRoleAppointment"])` and takes the `Person` elements in `rolePerformers`. Other actors, such as teams, are skipped. Returns one `(name, assignment types, member through, qualified name, GUID)` row per person, sorted by name; "member through" is `Direct` and/or `Role: `. A role that can't be read is skipped and reported with one warning; `None` if the community itself can't be read | -| `get_community_blog_posts(guid)` | Reads the `BlogEntry` notes from every note log attached to the community, so posts written by other tools are included and the notifications some sample communities' logs hold are not. Returns the `LATEST_POSTS` (20) newest as `(title, author, posted, text, GUID)` rows; author and time come from the element header's `versions` | -| `add_community_blog_post(guid, name, title, text)` | Finds the note log with qualified name `NoteLog::::CommunityBlog` among those attached to the community, creating it (attached, display name "_name_ Blog") on the first post, then `create_note(log, BlogEntryProperties)` | -| `get_community_comments(guid)` | `get_attached_comments(guid, graph_query_depth=0)` → `(comment, type, author, posted, GUID)` rows, newest first | -| `add_community_comment(guid, text, type)` | `add_comment_to_element(guid, text, type)`; the form offers pyegeria's `COMMENT_TYPES` | +| `get_community_blog_posts(guid)` | Pages the `BlogEntry` notes from every note log attached to the community, so posts written by other tools are included and the notifications some sample communities' logs hold are not. Returns the `LATEST_POSTS` (20) newest as `(title, author, posted, text, GUID)` rows; author and time come from the element header's `versions`. When any read wasn't confirmed complete it warns that newer posts may be missing, and an empty blog says "none found in what Egeria returned" rather than "No posts yet" | +| `add_community_blog_post(guid, name, title, text)` | Finds the blog note log, qualified name `NoteLog::::CommunityBlog`, with Egeria's unique-name read. It creates the log (attached, display name "_name_ Blog") only when that read says not-found, then `create_note(log, BlogEntryProperties)`. On any other answer nothing is created or posted (`LookupUnconfirmed`, a warning), because a second blog would split the posts | +| `get_community_resources(guid)` | `get_community_by_guid(guid, JSON, graph_query_depth=1, include_only_relationships=["ResourceList", "ScopedBy"])`. Takes `resourceList` (linked as `Resource`, or `Resource: `) and `relevantToScopes` (linked as `In scope`); `resourceListUsers`, the other end of a `ResourceList` where the community is itself someone's resource, is left out. Returns one `(name, type, linked as, description, GUID)` row per element, sorted by name; `None` if the read fails | +| `get_community_comments(guid)` | Pages `get_attached_comments(guid, graph_query_depth=0)` → `(comment, type, author, posted, GUID)` rows, newest first, as `PagedRows` (via the shared `_comment_rows`); warns with the "at least" note when the end wasn't confirmed | +| `add_community_comment(guid, text, type)` | `add_element_comment(guid, text, type)`; the form offers pyegeria's `COMMENT_TYPES` | -- **Return codes.** `CommunityScreen` dismisses with `200` on `Escape` (back to the communities list) and `210` on `q`; on `210` `CommunitiesScreen` dismisses itself too, so `q` reaches the main screen. -- **Why the role lookups are filtered.** Without `include_only_relationships`, one role can return hundreds of unrelated links. One sample role had 284 `assignmentScope` entries. With the filter, the Sustainability Community's 40 roles resolve in under a second. Asking `get_community_by_guid` for `graph_query_depth=2` instead, to get the appointments in one call, timed out after 30 seconds on the sample data. -- **Names.** People, roles and communities are named by `displayName`, then `fullName`, then `identifier`, then `qualifiedName`. Some sample community roles have only an `identifier`, such as "Community Leader of Data Science special interest group". -- **Membership isn't checked.** Anyone can read and add to a community's blog and comments; Egeria's quickstart deployment enforces no access rules here either. +- **Joining from here.** `_load_members` sets `is_member` when the app's `user_GUID` is among the members (directly or through a role) and shows the join controls (`community_join_controls`: `Contributor`/`Observer` and **Join Community**, `ctrl+w`) only when it is `False`. If the members can't be read, `is_member` stays `None` and joining isn't offered. After a successful join the screen sets `joined`, so `CommunitiesScreen` re-reads both lists when it comes back. +- **Return codes.** `CommunityScreen` dismisses with `200` on `Escape` (back to the communities list, or the main screen if opened from there) and `210` on `q`; on `210` `CommunitiesScreen` dismisses itself too, so `q` reaches the main screen. +- **Why the role lookups are filtered.** Without `include_only_relationships`, one role can return hundreds of unrelated links. One sample role had 284 `assignmentScope` entries. With the filter, the Sustainability Community's 40 roles resolve in under a second. Asking `get_community_by_guid` for `graph_query_depth=2` instead, to get the appointments in one call, timed out after 30 seconds on the sample data. The resources read is filtered to its two relationship types for the same reason. +- **Names.** People, roles, resources and communities are named by `displayName`, then `fullName`, then `identifier`, then `qualifiedName`. Some sample community roles have only an `identifier`, such as "Community Leader of Data Science special interest group". +- **Membership isn't checked.** Anyone can read and add to a community's blog, comments and feedback; Egeria's quickstart deployment enforces no access rules here either. +- **Sample data.** Several of the sample special interest groups (Observability, for example) have only `PersonRole` actors with nobody appointed, so their members list is empty even though they have many roles. + +### Feedback on a blog post or resource + +`ElementFeedbackScreen(guid, name, kind)` shows the feedback on one element: a summary (how many people like it and whether the user does, the average star rating, the comment count), a ratings table `(stars, review, rated by, rated)` and a comments table `(comment, type, author, posted, GUID)`. `kind` ("Blog post" or "Resource") is only for the heading. It reloads after every successful action. + +| Function | What it does | +| :--- | :--- | +| `get_element_feedback(guid)` | Pages `get_attached_comments`, `get_attached_likes` and `get_attached_ratings` (all `graph_query_depth=0`, through `_read_attached`). Returns `comments`, `likes` (count), `i_like` (a like whose `createdBy` is the user), `ratings` (newest first), `average` (mean stars, leaving out `NO_RECOMMENDATION`; `None` if none), `my_rating` (the user's own `(StarRating, review)`), and `complete`/`note`. `complete` is `True` only when all three reads reached a confirmed end; otherwise the screen shows each count as "at least N" and the note underneath. `None` if any read fails | +| `like_element(guid)` | `add_like_to_element(guid, body=...)` with an explicit `NewAttachmentRequestBody`/`LikeProperties` (`emoji` = `LIKE_EMOJI`), because pyegeria's `add_like_to_element`, given an emoji, sent a placeholder emoji and a fixed `ZoneMembership` instead (reported upstream) | +| `unlike_element(guid)` | `remove_like_from_element(guid)` | +| `rate_element(guid, star_rating, review)` | `add_rating_to_element(guid, True, NewAttachmentRequestBody)` with `RatingProperties` (`starRating`, and `review` only when given). Egeria keeps one rating per user, so this replaces any earlier one | +| `add_element_comment(guid, text, type)` | `add_comment_to_element(guid, text, type)`, for any element | + +- **Keys.** `ctrl+l` like/unlike (the button reads **Unlike** when `i_like`), `ctrl+r` rate (`RateElementScreen`, which starts from `my_rating` and dismisses with `{star_rating, review}` or `None`; the button reads **Change Rating** once the user has rated), `ctrl+n` comment (`AddCommunityCommentScreen`). +- **One like and one rating per user.** Confirmed against a live server: liking twice leaves one like, and rating again updates the user's rating in place (its create time is kept). +- **Stars.** `STAR_VALUES` maps Egeria's `StarRating` names (`FIVE_STARS` … `ONE_STAR`, `NO_RECOMMENDATION`) to 5…0; `star_label` shows them as `★★★★☆`, or "No recommendation". +- **Return codes.** `200` on `Escape` (back to the community) and `210` on `q`, which `CommunityScreen` passes on as `210` so `q` reaches the main screen. --- ## Status Screen `StatusScreen(message)` shows a read-only message: -- `c` copies the first `'...'`-quoted substring (normally a GUID) to the clipboard with pyegeria's `copy_to_clipboard`; -- `q` and `Enter` return `200`; -- `b` returns `400`. +- `ctrl+y` copies the first `'...'`-quoted substring (normally a GUID) to the clipboard with pyegeria's `copy_to_clipboard`; +- `Enter`, `Escape` and `q` return `200`; +- `ctrl+x` (Bad Result) returns `400`. The app's `status_callback` then returns to the main screen. @@ -757,14 +813,20 @@ The suite in `tests/micro-tests/my_profile/` has one module per handler and scre **My Network.** `test_network_handler.py` covers `NetworkMixin` and `MyNetworkScreen`, including the remove confirmation; `test_my_egeria.py` checks the menu routing. The feature has also been tested by hand against a live server. -**Work with Communities.** `test_community_handler.py` covers `CommunitiesMixin`, `CommunitiesScreen` and `CommunityScreen`. That includes: +**Work with Communities.** `test_community_handler.py` covers `CommunitiesMixin`, `CommunitiesScreen`, `CommunityScreen` and `ElementFeedbackScreen`. That includes: - memberships through roles, the leave confirmation, and refusing to leave a role membership; -- creating a community: leading it by default, not joining, a duplicate name, a failed create and a failed link; +- creating a community: leading it by default, not joining, a duplicate name (found by the unique-name read), a failed create and a failed link; - the Create Community form: validation, cancel, and highlighting the new community; - members as people only, direct and through roles, merged per person, with an unreadable role skipped; -- creating the blog on the first post and reusing it afterwards, and the post and comment forms. +- creating the blog on the first post and reusing it afterwards, and the post and comment forms; +- resources: `ResourceList` and `ScopedBy` merged per element, the reverse `ResourceList` end left out; +- joining from the community screen (offered only to non-members, not offered when the members can't be read), and the found list moving the community to the user's own list afterwards; +- feedback: paged comment/like/rating reads, "at least" counts, the explicit like body, rating with and without a review, like/unlike, rate (starting from the user's rating), comment, and `Escape`/`q` navigation; +- opening a community from a main-screen row, Find Community starting in the search box, and the main-screen refresh after a membership change (and not otherwise). + +`test_my_egeria.py` checks the menu routing, including **Find Community**. The blog and comment functions were run against a live server on 2026-10-28. Listing and finding communities, `create_community` (with and without joining, and the duplicate check) and `get_community_members` were run there on 2026-10-30. Joining and leaving, `get_community_resources`, and liking, unliking, rating and commenting on a blog post and a resource were run there on 2026-11-17, and the community and feedback screens were rendered against live data. The screens haven't yet been driven by hand end to end on a live server. -`test_my_egeria.py` checks the menu routing. The blog and comment functions were run against a live server on 2026-10-28. Listing and finding communities, `create_community` (with and without joining, and the duplicate check) and `get_community_members` were run there on 2026-10-30. The screens themselves have not been tried by hand there yet. +**Paging.** `test_paging_contract.py` checks the handlers against Egeria's paging contract: no total or "none" without a confirmed end, and no write when a guard's answer is unknown. **Screen headings.** `test_profile_utils.py::TestSetScreenHeading` covers `set_screen_heading`. `test_my_egeria.py` checks that the main screen's heading shows the Karma Points once the profile has loaded, and resets them to 0 on `change_user`. @@ -782,7 +844,7 @@ The user-visible limitations are listed in the [user manual](my_egeria_manual.md - `get_data_product_catalog_table`, `get_guid_for_qualified_name`, `display_glossary_term_details`, `tech_type_processes_details`, `unpack_egeria_data` and `display_selected_data_specification` are defined but never called. - Bookmarks, My Collections, team joining, subscription links and starting processes have only been tested against fakes; they haven't yet been run against a live Egeria server. -- Work with Communities: `join_community` (from the found list), `leave_community` and the screens have only been tested against fakes. The other functions have been run against a live server. +- Work with Communities: every handler function has been run against a live server; the screens have been rendered against live data but only driven end to end against fakes. - Two Add paths create communities: `AddCommunityScreen` on the User Associations edit screen, and `CreateCommunityScreen` in Work with Communities. Only the second checks for a duplicate name and links the user. --- diff --git a/docs/my_egeria_manual.md b/docs/my_egeria_manual.md index 0c6e356a..b4182b5d 100644 --- a/docs/my_egeria_manual.md +++ b/docs/my_egeria_manual.md @@ -151,7 +151,7 @@ The header reads **Egeria — User: _your user ID_, Karma Points: _n_**. Karma P | Panel | Columns | Contents | | :--- | :--- | :--- | -| **User Associations** | Status or Type, Name, Description, GUID | The communities you belong to | +| **User Associations** | Status or Type, Name, Description, GUID | The communities you belong to. Press `Enter` on one (or click it a second time) to [open it](#a-community) | | **My Collections** | Collection Name, Collection Description, Collection GUID | Collections you created (your bookmarks collection is not listed here) | | **Other Functions** | – | A menu of further features ([below](#other-functions-menu)) | | **Roles** | Role Name, Role Type, Description, GUID | Roles you have been appointed to. Press `Enter` on a TeamLeader or TeamMember role (or click it a second time) to see the other people in that team | @@ -291,27 +291,42 @@ Your network is the people linked to your profile in Egeria. The screen lists th Each person in your network is linked to your profile with a `Peer` relationship. The nickname and description are stored on that relationship. ### Work with Communities -The screen lists the communities you belong to (name, assignment type, what links you to it, description and GUID), lets you find others to join, and opens a community to show its members, blog and comments. If your profile has not finished loading, the app tells you to wait and does not open the screen. +The screen lists the communities you belong to (name, assignment type, what links you to it, description and GUID), lets you find others to join, and opens a community to show its members, blog, resources and comments. **Find Community** opens the same screen with the cursor in the search field. If your profile has not finished loading, the app tells you to wait and does not open the screen. When you return to the main screen after joining, leaving or creating-and-joining a community, the main screen reloads so User Associations is up to date. - **Linked Through** is **Your profile** if you joined the community yourself, or **Role: _name_** if you belong through a role you hold. A community you reach both ways is listed once. - **Open a community:** highlight a community in either list and press `Enter`, `ctrl+o` or **Open Community**. See [A community](#a-community) below. -- **Create a community:** press `ctrl+t` (or **Create Community**). Enter a **Name\*** and **Description\***, and optionally a **Mission**, then choose **Your part in it**: **Leader** (the default, since you created it), **Contributor**, **Observer** or **Don't join**. Press `ctrl+s` (or **Create**); **Cancel** or `Escape` discards it. The lists reload and the new community is highlighted: in your communities if you joined it, otherwise in the found list. If a community with that name already exists, you are told to find it and join it instead. -- **Find and join:** type a search term in the search field (`*` for all) and press `Enter`. Communities you already belong to are left out. Choose **Contributor** or **Observer**, highlight a community and press `ctrl+n` (or **Join Community**). Leaders and administrators are assigned by the people who manage the community, not chosen here. +- **Create a community:** press `ctrl+t` (or **Create Community**). Enter a **Name\*** and **Description\***, and optionally a **Mission**, then choose **Your part in it**: **Leader** (the default, since you created it), **Contributor**, **Observer** or **Don't join**. Press `ctrl+s` (or **Create**); **Cancel** or `Escape` discards it. The lists reload and the new community is highlighted: in your communities if you joined it, otherwise in the found list. If a community with that name already exists, you are told to find it and join it instead. If Egeria can't say either way, nothing is created and you are asked to try again later. +- **Find and join:** type a search term in the search field (`*` for all) and press `Enter`. Communities you already belong to are left out. If Egeria doesn't confirm it sent every match, a warning says the list may be incomplete (see [Troubleshooting](#troubleshooting)). Choose **Contributor** or **Observer**, highlight a community and press `ctrl+n` (or **Join Community**). Leaders and administrators are assigned by the people who manage the community, not chosen here. - **Leave:** highlight one of your communities and press `ctrl+d` (or **Leave Community**), then confirm with **Leave**. **Cancel** or `Escape` keeps you in it. Only the link between you and the community is removed. You can't leave a community you belong to through a role; ask whoever manages that role. - `Escape` or `q` returns to the main screen. Membership is an `AssignmentScope` relationship from your profile (or a role) to the community, and the assignment type is stored on that relationship. #### A community -Opening a community shows three lists: its **latest blog posts** (the 20 newest, with title, author and date; the highlighted post's text is shown underneath), its **members** and the **comments** on it, newest first. You don't have to be a member to read or add to them. +Open a community from either list on the Communities screen, or with `Enter` on a row of the main screen's User Associations table. It shows four lists: its **latest blog posts** (the 20 newest, with title, author and date; the highlighted post's text is shown underneath), its **members**, its **resources** and the **comments** on it, newest first. You don't have to be a member to read or add to them. The members list shows **people only**, in name order, not roles. It includes everyone linked to the community directly and everyone appointed to a role that is. **Member Through** says how each person belongs: **Direct**, or **Role: _name_** for each role they hold there. A person who belongs in several ways is listed once. Other kinds of member, such as teams, are not shown. If the people in one of the community's roles can't be read, the rest are still listed and a warning says some members may be missing. +The resources list shows the elements the community lists as its resources (**Linked As** is **Resource**, or **Resource: _use_** when the link says what the resource is for) and the elements in its scope (**In scope**), such as the digital products a special interest group looks after. An element linked both ways is listed once. + +- **Join:** if you are not a member, the members panel offers **Contributor** or **Observer** and a **Join Community** button (`ctrl+w`). Once you have joined, the members list reloads and the button disappears; back on the Communities screen, the community moves from the found list to yours. If the members can't be read, joining isn't offered. - **New blog post:** press `ctrl+n` (or **New Blog Post**), enter a title and the text, then press `ctrl+s` (or **Post**). **Cancel** or `Escape` discards it. - **Add a comment:** select the comments list and press `ctrl+n` (or press **Add Comment**), choose a comment type (Standard Comment, Question, Answer, Suggestion, Usage Experience, Requirement or Other), enter the comment and press `ctrl+s` (or **Add Comment**). -- `Escape` returns to the communities list; `q` returns to the main screen. +- **Feedback on a post or resource:** press `Enter` on a blog post or resource, or highlight one and press `ctrl+o` (or **Feedback on Post** / **Feedback on Resource**). See below. +- `Escape` returns to the communities list (or the main screen, if you opened the community from there); `q` returns to the main screen. + +The community's blog is a note log attached to the community, created when the first post is made (if Egeria can't say whether the blog already exists, nothing is posted and you are asked to try again); each post is a `BlogEntry` in it. Comments are attached to the community itself. Resources are `ResourceList` relationships from the community and `ScopedBy` relationships to it; a `ResourceList` that makes the community someone else's resource is not shown. + +#### Feedback on a blog post or resource +The feedback screen shows how many people like the item (and whether you do), its average star rating, its **ratings** (stars, review, who rated it and when) and its **comments**, newest first. When Egeria doesn't confirm it sent all the likes, ratings and comments, the counts read "at least N" and a note says there may be more (see [Troubleshooting](#troubleshooting)). -The community's blog is a note log attached to the community, created when the first post is made; each post is a `BlogEntry` in it. Comments are attached to the community itself. +- **Like / Unlike:** press `ctrl+l` (or the **Like** button, which reads **Unlike** once you like it). +- **Rate:** press `ctrl+r` (or **Rate**), choose from one to five stars or **No recommendation**, optionally write a review, and press `ctrl+s` (or **Rate**). Egeria keeps one rating per person, so rating again (the button then reads **Change Rating**) replaces your earlier one; the form starts from it. **No recommendation** is not counted in the average. +- **Comment:** press `ctrl+n` (or **Add Comment**), as for a comment on the community. +- `Escape` returns to the community; `q` returns to the main screen. + +### Find Community +Opens [Work with Communities](#work-with-communities) with the cursor in the search field: type a search term (`*` for all) and press `Enter`, then join or open a community from the found list. ### Change User Log in as a different Egeria user and reload the app for them. See [The Change User screen](#the-change-user-screen). @@ -334,20 +349,21 @@ Every screen follows the same rules: | `q` | Main menu / quit the app | Every list screen / Main screen | | `Enter` | Select the highlighted row; continue | Tables and trees; Status screen | | `ctrl+s` | Save / submit the form | Add screens, Add Comment, Create Subscription, Create/Edit Profile, Feedback, New Blog Post, Create Community | -| `ctrl+n` | Add a new item | Edit screens (row), Bookmarks, Comments, Main screen (comment), Communities (join), a community (blog post or comment) | +| `ctrl+n` | Add a new item | Edit screens (row), Bookmarks, Comments, Main screen (comment), Communities (join), a community (blog post or comment), feedback on a post or resource (comment) | | `ctrl+d` | Delete / remove the highlighted item | Edit screens, Bookmarks, My Network, Communities (leave) | -| `ctrl+o` | Open the highlighted item | Main screen (comments), Comments (responses), My Network (person details), Communities (open a community) | +| `ctrl+o` | Open the highlighted item | Main screen (comments), Comments (responses), My Network (person details), Communities (open a community), a community (feedback on the highlighted post or resource) | +| `ctrl+w` | Join the community | A community you are not a member of | | `ctrl+t` | Edit the selected table / create a community | Main screen / Communities | | `ctrl+k` | Bookmark the highlighted row | Main screen, Shop for Data | | `ctrl+b` | Bookmarks, with the highlighted row's GUID ready to add | Main screen, Shop for Data | | `ctrl+u` | Subscribe | Shop for Data, Overview, sample-data view | -| `ctrl+l` | Show sample data | Shop for Data | +| `ctrl+l` | Show sample data / like or unlike | Shop for Data / feedback on a post or resource | | `ctrl+g` | Search glossary terms | Shop for Data | | `ctrl+e` | Expand / collapse the whole tree | Technology Types | | `ctrl+y` | Copy the GUID shown | Status screen | | `ctrl+x` | Close, flagging a bad result | Status screen | | `ctrl+g` / `ctrl+l` / `ctrl+r` / `ctrl+t` | Edit Communities / Identities / Roles / Teams | Edit Profile | -| `ctrl+r` | Refresh all data from Egeria | Main screen | +| `ctrl+r` | Refresh all data from Egeria / rate | Main screen / feedback on a post or resource | | `ctrl+f` | Leave feedback | Any screen | | `F3` | Show the feedback log | Any screen (feedback owner only) | | `ctrl+c` | Force quit (terminal) | Anywhere | @@ -383,17 +399,23 @@ The full list is in `RETURN_CODES.md` in the app folder. - the view server named by `EGERIA_VIEW_SERVER` is running; - `EGERIA_USER` and `EGERIA_USER_PASSWORD` are correct, and the user is allowed to use the view server. If the credentials are the problem, press **Change User** rather than restarting the app. -**Something I added doesn't appear.** Press `ctrl+r` on the main screen. A community added from the User Associations edit screen is not linked to you (see Known Limitations), so it doesn't appear in User Associations until you join it from **Work with Communities**. One made with **Create Community** there, with yourself as a member, appears after `ctrl+r`. +**Something I added doesn't appear.** Press `ctrl+r` on the main screen. A community added from the User Associations edit screen is not linked to you (see Known Limitations), so it doesn't appear in User Associations until you join it from **Work with Communities**. One made with **Create Community** there, with yourself as a member, appears when you return to the main screen. + +**A list says "at least N", or "none found in what Egeria returned".** Egeria sends lists a page at a time, and for most of the lists this app reads it doesn't confirm when it has sent the last page. So when the app can't be sure it has everything, it says so instead of giving a total or saying "none". You may see a warning such as "Showing at least 12 ...", a Feedback Log heading of "at least N entries", or, on a community, "newer posts may be missing" and counts like "at least 2 people like this". The rows shown are real. There may just be more. On Egeria 6.3.0 or later, My Network's list can be confirmed complete. + +**"Couldn't confirm whether Egeria already holds ... Nothing was created; try again later."** Before creating something that must exist only once (your bookmarks collection, the feedback log, a community, a community's blog) or adding a bookmark or a person to your network, the app asks Egeria whether it already exists. If Egeria's answer is anything other than a clear yes or a clear "not found", the app writes nothing rather than risk a duplicate. Try again; if it keeps happening, check that Egeria is healthy. **The app quit unexpectedly.** The app only exits by itself in a few cases: a startup problem (codes `403`, `430`; `402` only when you press Quit on the connection error screen), a failure fetching technology types (`416`), or an error looking up a glossary term (`440`). A failed team lookup is reported as a message instead. ## Known Limitations -As of 2026-10-30, these features are incomplete or don't work as described elsewhere: +As of 2026-11-17, these features are incomplete or don't work as described elsewhere: - **Communities added from the User Associations edit screen** are not linked to you. The community is created but won't appear in User Associations until you join it from **Work with Communities**. **Create Community** in Work with Communities does link you. - **Not yet verified against a live Egeria server:** bookmarks, My Collections, joining a team, subscription links and starting processes. The test server was unavailable when they were added, so they have only been tested against fakes. -- **Work with Communities** has only partly been tried against a live server. Listing your communities, finding communities, creating a community (and joining it as part of that), the members list, and reading and adding blog posts and comments all work there. Leaving a community, joining one from the found list, and the screens themselves have so far only been tested against fakes. +- **Work with Communities** has been tried against a live server: listing your communities, finding communities, creating a community (and joining it as part of that), joining and leaving, the members and resources lists, reading and adding blog posts and comments, and liking, rating and commenting on blog posts and resources all work there (as of 2026-11-17). The screens themselves have been rendered against live data, but driven end-to-end only against fakes. +- **Most lists can't be confirmed complete.** Egeria doesn't yet confirm the last page for most of the lists the app reads, so they show "at least" counts and warnings (see [Troubleshooting](#troubleshooting)). This should improve as Egeria's paging fixes reach more of its APIs. +- **A community's members** are the people linked to it or appointed to its roles. A community whose roles have nobody appointed (several of the sample special interest groups) shows no members. If the people in one of its roles can't be read, you may be offered **Join** for a community you already belong to through that role. ## Running the Tests diff --git a/docs/release-notes/next.md b/docs/release-notes/next.md index fd263bc6..e433cb90 100644 --- a/docs/release-notes/next.md +++ b/docs/release-notes/next.md @@ -43,6 +43,18 @@ notes and are not repeated here. reported "not applied". - **Dr.Egeria Project Scope maps to `ProjectProperties.scope`** (PY-5 / EG-14). +- **My Profile: work with communities directly (#601, Peter Coldicott).** + - Press `Enter` on a community in the main screen's User Associations to open it. + - **Find Community** on the Other Functions menu opens the Communities screen ready to search. + - A community now shows its **resources**: what it lists with `ResourceList`, and what is `ScopedBy` it. + - If you aren't a member, the community screen offers **Join Community** (Contributor or Observer). + - Blog posts and resources have a **feedback** screen: like/unlike, a 1–5 star rating with an optional review + (one per person; rating again replaces it), and comments. + - The main screen reloads after you join, leave or create-and-join a community. + - Feedback reads use the same paging rules as the rest of My Profile: counts say "at least N" unless Egeria + confirms the end. + - See `docs/my_egeria_manual.md` and `docs/My-Egeria-Doc.md`. + ## Planned for pyegeria 6.4 or the next release - **No default password, and no built-in identity outside a named demo profile.** pyegeria 6.4 or the next release drops the quickstart diff --git a/my_egeria/my_egeria/CommunitiesScreen.py b/my_egeria/my_egeria/CommunitiesScreen.py index e9e7b834..886dd70d 100644 --- a/my_egeria/my_egeria/CommunitiesScreen.py +++ b/my_egeria/my_egeria/CommunitiesScreen.py @@ -11,15 +11,12 @@ from textual.screen import ModalScreen from textual.widgets import DataTable, Header, Static, Footer, Input, Button, Select, TextArea -from CommunityScreen import CommunityScreen +from CommunityScreen import CommunityScreen, JOIN_ASSIGNMENT_TYPES from paging_support import is_complete, none_words from profile_utils import set_screen_heading # The "Linked Through" value of a membership held by the user's own profile, which they can leave DIRECT_LINK = "Your profile" -# Assignment types a user can give themselves when joining. Leaders and administrators are -# assigned by the people who manage the community (https://egeria-project.org/concepts/community/) -JOIN_ASSIGNMENT_TYPES = ["Contributor", "Observer"] # Whoever creates a community manages it, so they may also lead it CREATOR_ASSIGNMENT_TYPES = ["Leader"] + JOIN_ASSIGNMENT_TYPES # The "Your part in it" choice for a creator who doesn't join their new community @@ -159,10 +156,12 @@ class CommunitiesScreen(ModalScreen): CSS_PATH = "my_profile.tcss" - def __init__(self, my_communities, *args, **kwargs): + def __init__(self, my_communities, *args, focus_search: bool = False, **kwargs): super().__init__(id="communities_screen", *args, **kwargs) # Rows of (name, assignment type, linked through, description, GUID) self.my_communities_data = list(my_communities or []) + # Start in the search box (the Find Community menu option) rather than the user's own list + self.focus_search = focus_search # The last search, repeated after joining or leaving so the found list stays current self.search_string = "*" @@ -170,7 +169,8 @@ def compose(self) -> ComposeResult: yield Header(show_clock=True) yield Static("Communities", classes="span-3", id="communities_title") yield ScrollableContainer( - Static("The communities you belong to. Enter or ctrl+o opens one (members, blog and comments); " + Static("The communities you belong to. Enter or ctrl+o opens one (members, blog, resources and " + "comments); " "ctrl+d leaves; ctrl+t creates a new community."), DataTable(id="my_communities_table", zebra_stripes=True, cursor_type="row"), Horizontal( @@ -204,6 +204,8 @@ def on_mount(self) -> None: "Name", "Description", "Qualified Name", "GUID") self._fill_my_table() self._search() + if self.focus_search: + self.query_one("#community_search_input", Input).focus() def _fill_my_table(self) -> None: table = self.query_one("#my_communities_table", DataTable) @@ -254,12 +256,17 @@ def action_quit(self) -> None: # --- opening a community ---------------------------------------------- def _open(self, community_guid: str, community_name: str) -> None: + community_screen = CommunityScreen(community_guid, community_name) + def on_close(result) -> None: # q on the community screen goes on to the main screen if result == 210: self.dismiss(200) + elif community_screen.joined: + # Joined from inside the community: it moves from the found list to the user's own + self._reload() - self.app.push_screen(CommunityScreen(community_guid, community_name), callback=on_close) + self.app.push_screen(community_screen, callback=on_close) def action_open_community(self) -> None: """ Open the highlighted community in whichever list has focus """ diff --git a/my_egeria/my_egeria/CommunityScreen.py b/my_egeria/my_egeria/CommunityScreen.py index 825b305f..5b4042b5 100644 --- a/my_egeria/my_egeria/CommunityScreen.py +++ b/my_egeria/my_egeria/CommunityScreen.py @@ -3,8 +3,9 @@ Copyright Contributors to the ODPi Egeria project. This file provides the screen for a single community in My Profile: its members, - the latest posts on its blog and the comments attached to it, plus the forms to - add a blog post or a comment. + the latest posts on its blog, its resources and the comments attached to it, plus + the forms to join it, add a blog post or a comment, and to give feedback (comments, + a like, a star rating) on a blog post or resource. """ from textual import on @@ -23,6 +24,22 @@ def comment_type_label(comment_type: str) -> str: return comment_type.replace("_", " ").title() +# Egeria's StarRating values, best first, with the number of stars each is worth +STAR_VALUES = {"FIVE_STARS": 5, "FOUR_STARS": 4, "THREE_STARS": 3, "TWO_STARS": 2, "ONE_STAR": 1, + "NO_RECOMMENDATION": 0} +# Assignment types a user can give themselves when joining. Leaders and administrators are +# assigned by the people who manage the community (https://egeria-project.org/concepts/community/) +JOIN_ASSIGNMENT_TYPES = ["Contributor", "Observer"] + + +def star_label(star_rating: str) -> str: + """'FOUR_STARS' -> '★★★★☆'; 'NO_RECOMMENDATION' -> 'No recommendation'""" + stars = STAR_VALUES.get(star_rating) + if not stars: + return "No recommendation" if star_rating == "NO_RECOMMENDATION" else star_rating + return "★" * stars + "☆" * (5 - stars) + + class NewBlogPostScreen(ModalScreen): """Collect the title and text of a new post for a community's blog. @@ -139,20 +156,234 @@ def handle_cancel_button(self, event: Button.Pressed) -> None: self.action_cancel() +class RateElementScreen(ModalScreen): + """Collect a star rating and an optional review, starting from the user's current rating if any. + + Dismisses with {"star_rating", "review"}, or None if cancelled. + """ + + BINDINGS = [ + ("escape", "cancel", "Back"), + ("ctrl+s", "submit", "Rate"), + ] + + CSS_PATH = "my_profile.tcss" + + def __init__(self, element_name: str, my_rating: tuple[str, str] | None = None, *args, **kwargs): + super().__init__(*args, **kwargs, id="rate_element_screen") + self.element_name = element_name + self.my_rating = my_rating + + def compose(self) -> ComposeResult: + star_rating, review = self.my_rating or ("FIVE_STARS", "") + if star_rating not in STAR_VALUES: + star_rating = "FIVE_STARS" + yield Header(show_clock=True) + yield Static("Rate", classes="span-3", id="rate_element_title") + yield ScrollableContainer( + Static(f"[b]Rate {self.element_name}[/b]"), + Static("[dim]Rating again replaces your earlier rating[/dim]" if self.my_rating else ""), + Static("Stars"), + Select([(star_label(s), s) for s in STAR_VALUES], value=star_rating, allow_blank=False, + id="star_rating_select"), + Static("Review (optional)"), + TextArea(review, id="rating_review_text"), + Horizontal( + Button("Rate", id="submit_rating_btn", variant="primary"), + Button("Cancel", id="cancel_rating_btn", variant="warning"), + ), + id="rate_element_container", + ) + yield Footer() + + def on_mount(self) -> None: + set_screen_heading(self, "Communities - Rate") + self.query_one("#star_rating_select", Select).focus() + + def action_submit(self) -> None: + self.dismiss({"star_rating": self.query_one("#star_rating_select", Select).value, + "review": self.query_one("#rating_review_text", TextArea).text.strip()}) + + def action_cancel(self) -> None: + self.dismiss(None) + + @on(Button.Pressed, "#submit_rating_btn") + def handle_submit_button(self, event: Button.Pressed) -> None: + self.action_submit() + + @on(Button.Pressed, "#cancel_rating_btn") + def handle_cancel_button(self, event: Button.Pressed) -> None: + self.action_cancel() + + +class ElementFeedbackScreen(ModalScreen): + """The feedback on one blog post or resource: its likes, star ratings and comments, with + actions to like or unlike it, rate it and comment on it. + + The Egeria work is done by the app's CommunitiesMixin (get_element_feedback, like_element, + unlike_element, rate_element, add_element_comment); this screen only drives it. + + Dismisses with 200 to go back to the community, or 210 to go on to the main screen. + """ + + BINDINGS = [ + ("escape", "back", "Back"), + ("q", "quit", "Main Menu"), + ("ctrl+l", "toggle_like", "Like / Unlike"), + ("ctrl+r", "rate", "Rate"), + ("ctrl+n", "add_comment", "Add Comment"), + ] + + CSS_PATH = "my_profile.tcss" + + def __init__(self, element_guid: str, element_name: str, element_kind: str = "", *args, **kwargs): + super().__init__(id="element_feedback_screen", *args, **kwargs) + self.element_guid = element_guid + self.element_name = element_name + # What the element is, for the heading: "Blog post", "Resource", ... + self.element_kind = element_kind + self.feedback: dict = {} + + def compose(self) -> ComposeResult: + heading = f"{self.element_kind}: {self.element_name}" if self.element_kind else self.element_name + yield Header(show_clock=True) + yield Static(f"Feedback - {heading}", classes="span-3", id="element_feedback_title") + yield ScrollableContainer( + Static("", id="feedback_summary"), + Horizontal( + Button("Like", id="toggle_like_btn", variant="primary"), + Button("Rate", id="rate_element_btn", variant="primary"), + Button("Add Comment", id="add_element_comment_btn", variant="primary"), + id="feedback_buttons", + ), + id="feedback_summary_container", + ) + yield ScrollableContainer( + Static("[b]Ratings[/b]"), + DataTable(id="element_ratings_table", zebra_stripes=True, cursor_type="row"), + id="element_ratings_container", + ) + yield ScrollableContainer( + Static("[b]Comments[/b]"), + DataTable(id="element_comments_table", zebra_stripes=True, cursor_type="row"), + id="element_comments_container", + ) + yield Footer() + + def on_mount(self) -> None: + set_screen_heading(self, f"Community - Feedback on {self.element_name}") + self.query_one("#element_ratings_table", DataTable).add_columns("Stars", "Review", "Rated By", "Rated") + self.query_one("#element_comments_table", DataTable).add_columns( + "Comment", "Type", "Author", "Posted", "GUID") + self._load() + + def _load(self) -> None: + feedback = self.app.get_element_feedback(self.element_guid) + if feedback is None: + return + self.feedback = feedback + ratings = self.query_one("#element_ratings_table", DataTable) + ratings.clear() + for row in feedback["ratings"]: + ratings.add_row(*row) + comments = self.query_one("#element_comments_table", DataTable) + comments.clear() + for row in feedback["comments"]: + comments.add_row(*row, key=row[4]) + + # Counts are "at least" unless Egeria confirmed every comment, like and rating was read + complete = feedback.get("complete", True) + at_least = "" if complete else "at least " + likes = feedback["likes"] + like_text = f"{at_least}{likes} {'person likes' if likes == 1 and complete else 'people like'} this" + if feedback["i_like"]: + like_text += " [dim](including you)[/dim]" + average = feedback["average"] + rating_count = len(feedback["ratings"]) + if average is None: + rating_text = ("No star ratings yet" if complete else "No star ratings in what Egeria returned") \ + if not rating_count else f"{at_least}{rating_count} rating(s), no stars given" + else: + rating_text = f"Average rating {average:.1f} of 5 from {at_least}{rating_count} rating(s)" + summary = f"{like_text}\n{rating_text}\n{at_least}{len(feedback['comments'])} comment(s)" + if feedback.get("note"): + summary += f"\n[dim]{feedback['note']}[/dim]" + self.query_one("#feedback_summary", Static).update(summary) + self.query_one("#toggle_like_btn", Button).label = "Unlike" if feedback["i_like"] else "Like" + self.query_one("#rate_element_btn", Button).label = "Change Rating" if feedback["my_rating"] else "Rate" + + def action_back(self) -> None: + """ Back to the community """ + self.dismiss(200) + + def action_quit(self) -> None: + """ Back to the main screen """ + self.dismiss(210) + + # --- likes ----------------------------------------------------------------- + + @on(Button.Pressed, "#toggle_like_btn") + def handle_like_button(self, event: Button.Pressed) -> None: + self.action_toggle_like() + + def action_toggle_like(self) -> None: + if self.feedback.get("i_like"): + done = self.app.unlike_element(self.element_guid) + else: + done = self.app.like_element(self.element_guid) + if done: + self._load() + + # --- ratings --------------------------------------------------------------- + + @on(Button.Pressed, "#rate_element_btn") + def handle_rate_button(self, event: Button.Pressed) -> None: + self.action_rate() + + def action_rate(self) -> None: + def on_rating(rating: dict | None) -> None: + if not rating: + return + if self.app.rate_element(self.element_guid, rating["star_rating"], rating["review"]): + self._load() + + self.app.push_screen(RateElementScreen(self.element_name, self.feedback.get("my_rating")), + callback=on_rating) + + # --- comments -------------------------------------------------------------- + + @on(Button.Pressed, "#add_element_comment_btn") + def handle_add_comment_button(self, event: Button.Pressed) -> None: + self.action_add_comment() + + def action_add_comment(self) -> None: + def on_comment(comment: dict | None) -> None: + if not comment: + return + if self.app.add_element_comment(self.element_guid, comment["text"], comment["comment_type"]): + self._load() + + self.app.push_screen(AddCommunityCommentScreen(self.element_name), callback=on_comment) + + class CommunityScreen(ModalScreen): - """One community: its members, the latest posts on its blog and its comments. + """One community: its members, the latest posts on its blog, its resources and its comments. The Egeria work is done by the app's CommunitiesMixin (get_community_members, - get_community_blog_posts, add_community_blog_post, get_community_comments, - add_community_comment); this screen only drives it. + get_community_blog_posts, add_community_blog_post, get_community_resources, + get_community_comments, add_community_comment, join_community); this screen only drives it. + Feedback on a blog post or resource opens an ElementFeedbackScreen. Dismisses with 200 to go back to the communities list, or 210 to go on to the main screen. + joined is True once the user has joined the community from this screen. """ BINDINGS = [ ("escape", "back", "Back"), ("q", "quit", "Main Menu"), ("ctrl+n", "add_item", "New Post / Comment"), + ("ctrl+o", "feedback", "Feedback on Post / Resource"), + ("ctrl+w", "join", "Join Community"), ] CSS_PATH = "my_profile.tcss" @@ -163,22 +394,43 @@ def __init__(self, community_guid: str, community_name: str, *args, **kwargs): self.community_name = community_name # Full text of each blog post, by GUID, shown below the posts table self.post_text: dict[str, str] = {} + # Whether the user is a member (directly or through a role); None until the members are read + self.is_member: bool | None = None + self.joined = False def compose(self) -> ComposeResult: yield Header(show_clock=True) yield Static(self.community_name, classes="span-3", id="community_title") yield ScrollableContainer( - Static("[b]Latest blog posts[/b] [dim]ctrl+n adds a post[/dim]"), + Static("[b]Latest blog posts[/b] [dim]ctrl+n adds a post; Enter or ctrl+o gives feedback on one[/dim]"), DataTable(id="community_posts_table", zebra_stripes=True, cursor_type="row"), Static("", id="community_post_text"), - Button("New Blog Post", id="new_blog_post_btn", variant="primary"), + Horizontal( + Button("New Blog Post", id="new_blog_post_btn", variant="primary"), + Button("Feedback on Post", id="post_feedback_btn", variant="primary"), + id="community_blog_buttons", + ), id="community_blog_container", ) yield ScrollableContainer( Static("[b]Members[/b] [dim](people linked to the community, directly or through a role they hold)[/dim]"), DataTable(id="community_members_table", zebra_stripes=True, cursor_type="row"), + Static("", id="community_membership_text"), + Horizontal( + Select([(t, t) for t in JOIN_ASSIGNMENT_TYPES], value=JOIN_ASSIGNMENT_TYPES[0], + allow_blank=False, id="community_join_assignment_type"), + Button("Join Community", id="community_join_btn", variant="primary"), + id="community_join_controls", + ), id="community_members_container", ) + yield ScrollableContainer( + Static("[b]Resources[/b] [dim](elements the community lists or looks after; " + "Enter or ctrl+o gives feedback on one)[/dim]"), + DataTable(id="community_resources_table", zebra_stripes=True, cursor_type="row"), + Button("Feedback on Resource", id="resource_feedback_btn", variant="primary"), + id="community_resources_container", + ) yield ScrollableContainer( Static("[b]Comments[/b] [dim]with this list selected, ctrl+n adds a comment[/dim]"), DataTable(id="community_comments_table", zebra_stripes=True, cursor_type="row"), @@ -192,20 +444,31 @@ def on_mount(self) -> None: self.query_one("#community_posts_table", DataTable).add_columns("Title", "Author", "Posted", "GUID") self.query_one("#community_members_table", DataTable).add_columns( "Name", "Assignment Type", "Member Through", "Qualified Name", "GUID") + self.query_one("#community_resources_table", DataTable).add_columns( + "Name", "Type", "Linked As", "Description", "GUID") self.query_one("#community_comments_table", DataTable).add_columns( "Comment", "Type", "Author", "Posted", "GUID") self._load_members() self._load_posts() + self._load_resources() self._load_comments() def _load_members(self) -> None: members = self.app.get_community_members(self.community_guid) if members is None: + # Membership is unknown, so neither offer joining nor claim the user is a member + self.query_one("#community_join_controls", Horizontal).display = False return table = self.query_one("#community_members_table", DataTable) table.clear() for row in members: table.add_row(*row) + user_guid = getattr(self.app, "user_GUID", "") + self.is_member = bool(user_guid) and any(row[4] == user_guid for row in members) + self.query_one("#community_join_controls", Horizontal).display = not self.is_member + self.query_one("#community_membership_text", Static).update( + "[dim]You are a member[/dim]" if self.is_member + else "[dim]You are not a member - join as a contributor or an observer[/dim]") def _load_posts(self) -> None: """Rows of (title, author, posted, text, GUID), newest first.""" @@ -221,6 +484,15 @@ def _load_posts(self) -> None: self.post_text[guid] = text self._show_post_text() + def _load_resources(self) -> None: + resources = self.app.get_community_resources(self.community_guid) + if resources is None: + return + table = self.query_one("#community_resources_table", DataTable) + table.clear() + for row in resources: + table.add_row(*row, key=row[4]) + def _load_comments(self) -> None: comments = self.app.get_community_comments(self.community_guid) if comments is None: @@ -260,6 +532,71 @@ def action_add_item(self) -> None: else: self.action_new_post() + # --- joining --------------------------------------------------------------- + + @on(Button.Pressed, "#community_join_btn") + def handle_join_button(self, event: Button.Pressed) -> None: + self.action_join() + + def action_join(self) -> None: + """ The user joins this community, with the chosen assignment type """ + if self.is_member: + self.notify(f"You are already a member of {self.community_name}", timeout=5) + return + if self.is_member is None: + self.notify("Your membership couldn't be checked, so joining isn't offered", timeout=5, + severity="warning") + return + assignment_type = self.query_one("#community_join_assignment_type", Select).value + if self.app.join_community(self.community_guid, assignment_type): + self.joined = True + self._load_members() + + # --- feedback on posts and resources --------------------------------------- + + def _open_feedback(self, table_id: str, guid_column: int, kind: str) -> None: + table = self.query_one(table_id, DataTable) + if table.row_count == 0: + self.notify(f"There is no {kind.lower()} to give feedback on", timeout=5, severity="warning") + return + row = table.get_row_at(table.cursor_row) + + def on_close(result) -> None: + # q on the feedback screen goes on to the main screen + if result == 210: + self.dismiss(210) + + self.app.push_screen(ElementFeedbackScreen(row[guid_column], row[0], kind), callback=on_close) + + def _open_post_feedback(self) -> None: + self._open_feedback("#community_posts_table", 3, "Blog post") + + def _open_resource_feedback(self) -> None: + self._open_feedback("#community_resources_table", 4, "Resource") + + def action_feedback(self) -> None: + """ Feedback on the highlighted resource if the resources list has focus, otherwise on the post """ + if self.focused is self.query_one("#community_resources_table", DataTable): + self._open_resource_feedback() + else: + self._open_post_feedback() + + @on(Button.Pressed, "#post_feedback_btn") + def handle_post_feedback_button(self, event: Button.Pressed) -> None: + self._open_post_feedback() + + @on(Button.Pressed, "#resource_feedback_btn") + def handle_resource_feedback_button(self, event: Button.Pressed) -> None: + self._open_resource_feedback() + + @on(DataTable.RowSelected, "#community_posts_table") + def handle_post_selected(self, event: DataTable.RowSelected) -> None: + self._open_post_feedback() + + @on(DataTable.RowSelected, "#community_resources_table") + def handle_resource_selected(self, event: DataTable.RowSelected) -> None: + self._open_resource_feedback() + # --- blog ---------------------------------------------------------------- @on(Button.Pressed, "#new_blog_post_btn") diff --git a/my_egeria/my_egeria/MainScreen.py b/my_egeria/my_egeria/MainScreen.py index 7007cf85..9f7f7283 100644 --- a/my_egeria/my_egeria/MainScreen.py +++ b/my_egeria/my_egeria/MainScreen.py @@ -53,7 +53,7 @@ def compose(self) -> ComposeResult: yield Header(show_clock=True, id="main_header") yield ScrollableContainer( - Static("User Associations"), + Static("User Associations [dim](Enter on a community: open it)[/dim]"), DataTable(id="associations_table"), id="main_associations_container" ) @@ -69,6 +69,7 @@ def compose(self) -> ComposeResult: # Option("[] User Identities"), Option("[] My Network"), Option("[] Work with Communities"), + Option("[] Find Community"), Option("[] User Bookmarks"), Option("[] Leave Feedback"), Option("[] Feedback Log"), diff --git a/my_egeria/my_egeria/README.md b/my_egeria/my_egeria/README.md index 5502cffb..ea55749a 100644 --- a/my_egeria/my_egeria/README.md +++ b/my_egeria/my_egeria/README.md @@ -11,8 +11,9 @@ The `my-egeria` Textual TUI application's package root — see the parent | `my_profile.py` | Production My Profile & Catalog TUI Application (`MyProfileApp` / `MyEgeriaApp`, entry point `uv run my_profile` / `python -m my_egeria.my_profile`). | | `main.py` | Entry point for MyEgeria app (`uv run my-egeria` / `python -m my_egeria.main`). | | `my_egeria_app.py` | The MyEgeria browser/navigation Textual `App` class. | -| `*Screen.py` / `*Screens.py` | Production Textual screens (Main, Splash, Profile Creation/Edit, Data Shopping, Bookmarks, Network, Teams, Communities, Feedback, Tech Types, Subscriptions). | +| `*Screen.py` / `*Screens.py` | Production Textual screens (Main, Splash, Profile Creation/Edit, Data Shopping, Bookmarks, Network, Teams, Communities and a single community with feedback on its posts and resources, Feedback, Tech Types, Subscriptions). | | `*_handler.py` | Domain mixins for bookmarks, communities, CRUD operations, feedback, networks, shopping, team roles, and tech types. | +| `paging_support.py` | Egeria's paging contract for the handlers: paged reads (`read_all`/`PagedRows`), "at least" / "none" wording, and the found / absent / unknown checks made before a write. | | `profile_utils.py` | Profile formatting, Mermaid graph sanitization, data structures, and connection utilities. | | `screens/` | Feature screens (`GovernanceOfficer`, `ProductManager`, `ProjectManager`, `collections`, `glossary`, `platform`). | | `widgets/` | Reusable Textual widgets shared across screens. | diff --git a/my_egeria/my_egeria/RETURN_CODES.md b/my_egeria/my_egeria/RETURN_CODES.md index 0373756e..ddbbfd21 100644 --- a/my_egeria/my_egeria/RETURN_CODES.md +++ b/my_egeria/my_egeria/RETURN_CODES.md @@ -14,7 +14,7 @@ In accordance with application conventions: |---|---|---|---| | **`200`** | `SUCCESS_OK` / `DEFAULT_EXIT` | Standard success code. Used for successful operations, standard modal dismissal, backing out of dialogs, cancelling inputs without error, or quitting cleanly. | `my_profile.py`, `MainScreen.py`, `StatusScreen.py`, `CreateProfileScreen.py`, `EditElementsScreens.py`, `AddToElementsScreens.py`, `TechnologyTypeScreens.py`, `tech_types_handler.py`, `shop_for_data_handler.py`, `team_roles_handler.py`, `SelectionOverviewScreen.py`, `GenericDataViewScreen.py`, `MyBookMarksScreen.py`, `MyTeamScreen.py`, `SearchForTermScreen.py`, `ShowCommentScreen.py`, `MyNetworkScreen.py`, `CommunitiesScreen.py`, `CommunityScreen.py` | | **`201`** | `ALT_NAVIGATION` / `NO_MATCH` | Alternative navigation or fallback action code. In search/team screens, indicates no matching terms/roles found and triggers transition to data shopping or alternate view. In tech type selection, represents secondary navigation action. | `SearchForTermScreen.py`, `team_roles_handler.py`, `TechnologyTypeScreens.py`, `MyTeamScreen.py`, `ShopForDataScreen.py` | -| **`210`** | `QUIT_TO_MAIN` | Exit/Quit action code from data shopping / data viewing screens, and from a single community's screen, to dismiss the view and return directly to the main menu screen. | `ShopForDataScreen.py`, `GenericDataViewScreen.py`, `SelectionOverviewScreen.py`, `shop_for_data_handler.py`, `CommunityScreen.py`, `CommunitiesScreen.py` | +| **`210`** | `QUIT_TO_MAIN` | Exit/Quit action code from data shopping / data viewing screens, and from a single community's screen and its feedback screen (likes, ratings and comments on a blog post or resource), to dismiss the view and return directly to the main menu screen. | `ShopForDataScreen.py`, `GenericDataViewScreen.py`, `SelectionOverviewScreen.py`, `shop_for_data_handler.py`, `CommunityScreen.py`, `CommunitiesScreen.py` | | **`211`** | `SUBSCRIBE_ACTION` | Action code indicating the user has requested to create a digital subscription for the selected data source, catalog product, or data element. | `ShopForDataScreen.py`, `GenericDataViewScreen.py`, `SelectionOverviewScreen.py`, `shop_for_data_handler.py` | | **`212`** | `SAMPLE_DATA_ACTION` | Action code indicating the user has requested to view sample data for the selected data source in Shop for Data. | `ShopForDataScreen.py`, `shop_for_data_handler.py` | diff --git a/my_egeria/my_egeria/VIDEO_SCRIPT_DRAFT.md b/my_egeria/my_egeria/VIDEO_SCRIPT_DRAFT.md index 0c0b056f..c25f1c63 100644 --- a/my_egeria/my_egeria/VIDEO_SCRIPT_DRAFT.md +++ b/my_egeria/my_egeria/VIDEO_SCRIPT_DRAFT.md @@ -67,9 +67,9 @@ Camera pans through the structured grid layout: > At the top, my **User Associations** and **Collections** organize the clinical datasets and glossaries I care about. > In the middle, you see my appointed **Roles** and **Teams**. Notice how governance comes alive here: pressing Enter on my Clinical Trials team role instantly opens **My Team**, showing colleagues like Tessa Tube and Tanya Tidie along with their active responsibilities. > Down here is my daily activity stream: team **Blogs**, my personal **Journal**, and my **To-Dos**. -> And at the top right, Egeria tracks my **Karma Points**, recognizing active curation and collaboration across Coco Pharmaceuticals! +> And at the top right, Egeria tracks my **Karma Points**, calculated quietly in the background by Egeria's `Simon` housekeeping connector to recognize active curation and collaboration across Coco Pharmaceuticals! -**[ON-SCREEN CALLOUT]** "Live metadata synchronization • Role & Team drill-down • Automated Karma recognition" +**[ON-SCREEN CALLOUT]** "Live metadata synchronization • Role & Team drill-down • Simon Housekeeping Connector & Karma recognition" --- @@ -83,6 +83,7 @@ Camera pans through the structured grid layout: **CALLIE:** > Data governance only works if metadata is maintained as part of our normal flow of work, not as an afterthought. > Pressing **Ctrl+T** lets me edit any highlighted table in place—adding action items from morning stand-ups or updating project milestones. +> And we don't have to worry about our task lists becoming bloated: behind the scenes, Egeria's `Pasteur` connector automatically archives completed tasks while preserving our full audit lineage, and `Mendel` deduplicates our keyword tags. > With **Ctrl+O** and **Ctrl+N**, threaded discussions attach directly to the metadata asset or task itself. Instead of losing critical context in email threads, Dr. Tube and I keep our clinical trial discussions permanently linked to the data. > And with **Ctrl+K** and **Ctrl+B**, I can bookmark key assets and retrieve them instantly with a single keystroke. @@ -99,19 +100,19 @@ Camera pans through the structured grid layout: - **Business Domains** & **Root Collections** - **Glossaries** (Business concepts curated by Erin Overview & Peter Profile) - **Data Dictionaries** (Physical schemas & clinical data specifications) - - **Digital Product Catalog** (Certified research data products) + - **Digital Product Catalog** (Certified research data products adhering to Bitol ODPS standards) 3. Callie searches for *"Patient Cohort Treatment Efficacy"*, selects the data product, and presses `Ctrl+L` (**Sample Data**). A clean sample data preview opens displaying distribution metrics and confidentiality tags. -4. Confident in the data quality, she presses `Ctrl+U` (**Subscribe**). The **Create Subscription Request** modal opens. She enters her purpose (*"AI oncology treatment response modeling"*), agrees to governance terms, and submits. +4. Confident in the data quality, she presses `Ctrl+U` (**Subscribe**). The **Create Subscription Request** modal opens. She enters her purpose (*"AI oncology treatment response modeling"*), agrees to the Bitol Open Data Contract terms, and submits. 5. She navigates to **Other Functions** → **Subscriptions**, where the new active governed subscription appears. **CALLIE:** > Now for the most powerful feature in my workflow: **Shop for Data**. -> Before Jules Keeper established our open metadata catalog, finding trusted clinical data meant weeks of meetings and manual approvals. Here, Egeria brings together Business Domains, Glossaries, Data Dictionaries, and Digital Product Catalogs into a unified data marketplace. +> Before Jules Keeper established our open metadata catalog, finding trusted clinical data meant weeks of meetings and manual approvals. Here, Egeria brings together Business Domains, Glossaries, Data Dictionaries, and Digital Product Catalogs into a unified data marketplace. External partner hospital data is also seamlessly ingested using standard W3C DCAT version 3 connectors. > I can explore standardized terminology, verify classifications, and hit **Ctrl+L** to **sample** the dataset right in my terminal to ensure it fits my AI model requirements. -> When it's right, **Ctrl+U** lets me submit a formal **Subscription Request**. Egeria ensures Faith Broker's privacy rules and compliance policies are satisfied, giving data owners full visibility into who is using their data and why. +> When it's right, **Ctrl+U** lets me submit a formal **Subscription Request** under Bitol Open Data Contract standards. Egeria ensures Faith Broker's privacy rules and compliance policies are satisfied, and our `Baudot` connector delivers differential delta updates directly to subscribers without redundant event traffic. > In less than sixty seconds, governed access is requested, validated, and tracked in **Subscriptions**. -**[ON-SCREEN CALLOUT]** "Unified Data Marketplace • Instant Data Sampling (`Ctrl+L`) • Governed Subscriptions (`Ctrl+U`)" +**[ON-SCREEN CALLOUT]** "Unified Data Marketplace • Bitol ODCS Contracts • W3C DCAT v3 • Instant Sampling (`Ctrl+L`) • Governed Subscriptions (`Ctrl+U`)" --- @@ -125,10 +126,10 @@ Camera pans through the structured grid layout: **CALLIE:** > My Profile also strengthens our human network across Coco Pharmaceuticals. -> **My Network** visualizes collaboration graphs and reporting relationships across departments. +> **My Network** visualizes collaboration graphs and reporting relationships across departments, using bounded OpenLineage graph queries to keep navigation lightning fast across multi-layer data supply chains. > **Work with Communities** connects us to cross-functional working groups, like our Oncology Research and Sustainability circles. > If I notice something that needs improvement, pressing **Ctrl+F** lets me log feedback directly into Egeria from anywhere in the app. -> And for our integration architects like Bob Nitter, the **Technology Types** screen displays the automated governance action templates Egeria executes behind the scenes. +> And for our integration architects like Bob Nitter, the **Technology Types** screen displays the automated governance action templates and engine connectors Egeria executes behind the scenes. --- @@ -146,6 +147,12 @@ Camera pans through the structured grid layout: - **Egeria Project:** [https://egeria-project.org](https://egeria-project.org) - **Coco Pharmaceuticals Personas:** [https://egeria-project.org/practices/coco-pharmaceuticals/personas/](https://egeria-project.org/practices/coco-pharmaceuticals/personas/) - **PRD Associates & Jules Keeper Video Blogs:** [https://www.prd-associates.com](https://www.prd-associates.com) +- **Mandy Chessell Egeria Technical Blogs (Parts 6–10):** [https://github.com/odpi/egeria-docs/tree/main/blogs](https://github.com/odpi/egeria-docs/tree/main/blogs) + - *Blog 6:* Egeria 6.3: A Metadata Platform Built for the Long Run (Housekeeping & Reversibility) + - *Blog 7:* Supporting Security Professionals: Frameworks, Threat Models & Findings (NIST & STRIDE) + - *Blog 8:* Building the Data Mesh: Products, Subscriptions, Contracts & Dependencies (Bitol ODCS & Darwin) + - *Blog 9:* From Pipeline Runs to Business Flows: Extracting Lineage Layers (OpenLineage in PostgreSQL) + - *Blog 10:* Speaking the Language of Open Data: Exchanging DCAT Documents (W3C DCAT v3) - **Egeria Python SDK & TUI:** [https://github.com/odpi/egeria-python](https://github.com/odpi/egeria-python) (`my_egeria` package) - **Launch Command:** `uv run my_profile` or web interface via `uv run serve_my_profile` - *"Watch Video Blog 3: Establishing Data Governance by Jules Keeper on YouTube"* diff --git a/my_egeria/my_egeria/community_handler.py b/my_egeria/my_egeria/community_handler.py index 14845457..4fe87f4b 100644 --- a/my_egeria/my_egeria/community_handler.py +++ b/my_egeria/my_egeria/community_handler.py @@ -21,6 +21,16 @@ posts are read from every note log attached to the community, so entries written by other tools are shown too. Comments are attached to the community directly. + + A community's resources are the elements it lists with ResourceList + relationships (https://egeria-project.org/types/0/0019-More-Information/) and + those scoped to it with ScopedBy relationships (such as the digital products a + special interest group looks after). The other end of a ResourceList - where + the community is itself someone else's resource - is not shown. + + Feedback on a blog post or a resource is its comments, likes and star ratings. + Egeria keeps one like and one rating per user on an element: liking or rating + again replaces the user's earlier one. """ import sys @@ -40,7 +50,7 @@ from paging_support import (LookupUnconfirmed, PagedRows, by_unique_name, combine, paged_rows, read_all, unique_refusal) from CommunitiesScreen import CommunitiesScreen, DIRECT_LINK, JOIN_ASSIGNMENT_TYPES, CREATOR_ASSIGNMENT_TYPES -from CommunityScreen import comment_type_label +from CommunityScreen import CommunityScreen, comment_type_label, star_label, STAR_VALUES COMMUNITY_TYPE = "Community" PERSON_TYPE = "Person" @@ -52,6 +62,12 @@ BLOG_LOG_QN = "NoteLog::{community_guid}::CommunityBlog" # How many of a community's most recent blog posts are listed LATEST_POSTS = 20 +# The relationships that link a community to its resources, and the "Linked As" text of a scoped element +RESOURCE_RELATIONSHIPS = ["ResourceList", "ScopedBy"] +IN_SCOPE = "In scope" +# Sent with every like: pyegeria's add_like_to_element, given an emoji, sends a placeholder +# text and a fixed zone instead, so the body is always built here +LIKE_EMOJI = "👍" def _type_name(element: dict) -> str: @@ -136,8 +152,9 @@ def list_my_communities(self) -> list[tuple[str, str, str, str, str]] | None: return [(c["name"], ", ".join(c["types"]), "; ".join(c["through"]), c["description"], guid) for guid, c in communities.items()] - def show_communities(self) -> None: - """Show the Work with Communities screen for the current user.""" + def show_communities(self, find: bool = False) -> None: + """Show the Work with Communities screen for the current user. With find, the cursor starts + in the search box (the Find Community menu option).""" if not getattr(self, "user_GUID", ""): self.notify("Your profile has not loaded yet, so your communities can't be shown", timeout=10, severity="warning") @@ -145,11 +162,36 @@ def show_communities(self) -> None: communities = self.list_my_communities() if communities is None: return - self.push_screen(CommunitiesScreen(communities), callback=self.communities_callback) + self.push_screen(CommunitiesScreen(communities, focus_search=find), callback=self.communities_callback) def communities_callback(self, result: Any) -> None: - pass # self.log(f"Communities screen returned: {result}") + self._refresh_if_communities_changed() + + def open_community_row(self, table, row_key) -> None: + """Open the community in a main-screen User Associations row: (type, name, description, GUID).""" + try: + row = table.get_row(row_key) + except Exception: + return + community_guid, community_name = str(row[3] or ""), str(row[1] or "") + if not community_guid: + self.notify("This row has no GUID, so the community can't be opened", timeout=5, severity="warning") + return + self.push_screen(CommunityScreen(community_guid, community_name), callback=self.community_callback) + + def community_callback(self, result: Any) -> None: + # self.log(f"Community screen returned: {result}") + self._refresh_if_communities_changed() + + def _communities_changed(self) -> None: + """Note that the user's memberships changed, so the main screen is re-read when they return to it.""" + self.communities_changed = True + + def _refresh_if_communities_changed(self) -> None: + if getattr(self, "communities_changed", False): + self.communities_changed = False + self.call_later(self.action_refresh) def search_communities(self, search_string: str) -> list[tuple[str, str, str, str]] | None: """Communities matching search_string ('*' for all) as (name, description, qualified name, GUID) rows, @@ -243,6 +285,47 @@ def get_community_members(self, community_guid: str) -> list[tuple[str, str, str return sorted(((p["name"], ", ".join(p["types"]), "; ".join(p["through"]), p["qualified_name"], guid) for guid, p in people.items()), key=lambda row: row[0].lower()) + def get_community_resources(self, community_guid: str) -> list[tuple[str, str, str, str, str]] | None: + """The community's resources as (name, type, linked as, description, GUID) rows, sorted by name, + or None if they could not be retrieved. An element linked both ways is one row.""" + client = None + try: + client = self._community_client() + community = client.get_community_by_guid(community_guid, output_format="JSON", graph_query_depth=1, + include_only_relationships=RESOURCE_RELATIONSHIPS) + except PyegeriaException as e: + # self.log(f"Retrieving the resources of community {community_guid} failed: {e}") + self.notify(f"Could not retrieve the community's resources: {e}", timeout=10, severity="error") + return None + finally: + if client: + client.close_session() + if not isinstance(community, dict): + return [] + + links = [] + # resourceList is the community's own list; resourceListUsers, the other end, is left out + for link in community.get("resourceList") or []: + link_properties = link.get("relationshipProperties") or {} + use = link_properties.get("resourceUse") or link_properties.get("label") or "" + links.append((link, f"Resource: {use}" if use else "Resource")) + links += [(link, IN_SCOPE) for link in community.get("relevantToScopes") or []] + + resources: dict[str, dict] = {} + for link, linked_as in links: + element = link.get("relatedElement") or {} + guid = _guid(element) + if not guid: + continue + properties = element.get("properties") or {} + entry = resources.setdefault(guid, {"name": _name(properties), "type": _type_name(element), + "description": properties.get("description") or "", + "linked_as": []}) + if linked_as not in entry["linked_as"]: + entry["linked_as"].append(linked_as) + return sorted(((r["name"], r["type"], "; ".join(r["linked_as"]), r["description"], guid) + for guid, r in resources.items()), key=lambda row: row[0].lower()) + def get_community_blog_posts(self, community_guid: str) -> list[tuple[str, str, str, str, str]] | None: """The community's latest blog posts, newest first, as (title, author, posted, text, GUID) rows, or None if they could not be retrieved.""" @@ -349,18 +432,37 @@ def add_community_blog_post(self, community_guid: str, community_name: str, titl if client: client.close_session() + @staticmethod + def _comment_rows(read) -> PagedRows: + """(comment, type, author, posted, GUID) rows, newest first, from a paged read of comments""" + rows = [] + for comment in read.items: + if not isinstance(comment, dict): + continue + properties = comment.get("properties") or {} + author, create_time = _created(comment) + comment_type = comment_type_label(properties.get("commentType") or "") + rows.append((create_time, (properties.get("description") or "", comment_type, author, + _posted(create_time), _guid(comment)))) + rows.sort(key=lambda r: r[0], reverse=True) + return paged_rows([row for _, row in rows], read, "comments") + + @staticmethod + def _read_attached(client: Egeria, method: str, element_guid: str): + """Every comment, like or rating (method get_attached_comments/likes/ratings) on an element, paged + to Egeria's end (a PageResult).""" + def fetch(start: int, size: int): + return getattr(client, method)(element_guid, graph_query_depth=0, start_from=start, page_size=size) + + return read_all(fetch, method=method, client=client) + def get_community_comments(self, community_guid: str) -> list[tuple[str, str, str, str, str]] | None: """The comments attached to the community, newest first, as (comment, type, author, posted, GUID) rows, or None if they could not be retrieved.""" client = None try: client = self._community_client() - - def fetch(start: int, size: int): - return client.get_attached_comments(community_guid, graph_query_depth=0, start_from=start, - page_size=size) - - read = read_all(fetch, method="get_attached_comments", client=client) + read = self._read_attached(client, "get_attached_comments", community_guid) except PyegeriaException as e: # self.log(f"Retrieving the comments on community {community_guid} failed: {e}") self.notify(f"Could not retrieve the community's comments: {e}", timeout=10, severity="error") @@ -368,39 +470,127 @@ def fetch(start: int, size: int): finally: if client: client.close_session() - rows = [] - for comment in read.items: - if not isinstance(comment, dict): - continue - properties = comment.get("properties") or {} - author, create_time = _created(comment) - comment_type = comment_type_label(properties.get("commentType") or "") - rows.append((create_time, (properties.get("description") or "", comment_type, author, - _posted(create_time), _guid(comment)))) - rows.sort(key=lambda r: r[0], reverse=True) - result = paged_rows([row for _, row in rows], read, "comments") + result = self._comment_rows(read) if result.note: self.notify(result.note, timeout=10, severity="warning") return result - def add_community_comment(self, community_guid: str, text: str, comment_type: str) -> bool: - """Attach a comment to the community. comment_type is one of pyegeria's COMMENT_TYPES. - Returns True on success.""" + def add_element_comment(self, element_guid: str, text: str, comment_type: str) -> bool: + """Attach a comment to an element (a community, blog post or resource). comment_type is one of + pyegeria's COMMENT_TYPES. Returns True on success.""" client = None try: client = self._community_client() - response = client.add_comment_to_element(community_guid, text, comment_type) - # self.log(f"Commented on community {community_guid}: {response}") + response = client.add_comment_to_element(element_guid, text, comment_type) + # self.log(f"Commented on element {element_guid}: {response}") self.notify("Your comment has been added", timeout=10, severity="information") return True except PyegeriaException as e: - # self.log(f"Commenting on community {community_guid} failed: {e}") + # self.log(f"Commenting on element {element_guid} failed: {e}") self.notify(f"Adding the comment failed: {e}", timeout=10, severity="error") return False finally: if client: client.close_session() + def add_community_comment(self, community_guid: str, text: str, comment_type: str) -> bool: + """Attach a comment to the community. Returns True on success.""" + return self.add_element_comment(community_guid, text, comment_type) + + # --- feedback on a blog post or resource ----------------------------------- + + def get_element_feedback(self, element_guid: str) -> dict | None: + """The feedback on an element, or None if it could not be retrieved: + comments (comment, type, author, posted, GUID) rows, newest first + likes how many people like it + i_like whether the user is one of them + ratings (stars, review, rated by, rated) rows, newest first + average mean stars over the ratings that give some, or None + my_rating the user's own (StarRating value, review), or None + complete whether Egeria confirmed every comment, like and rating was read; when not, the + counts are "at least" and note says so + note the "at least" words, or None + """ + client = None + try: + client = self._community_client() + comments = self._read_attached(client, "get_attached_comments", element_guid) + likes = self._read_attached(client, "get_attached_likes", element_guid) + ratings = self._read_attached(client, "get_attached_ratings", element_guid) + except PyegeriaException as e: + # self.log(f"Retrieving the feedback on element {element_guid} failed: {e}") + self.notify(f"Could not retrieve the feedback: {e}", timeout=10, severity="error") + return None + finally: + if client: + client.close_session() + + like_items = [like for like in likes.items if isinstance(like, dict)] + rating_rows, stars, my_rating = [], [], None + for rating in ratings.items: + if not isinstance(rating, dict): + continue + properties = rating.get("properties") or {} + star_rating = properties.get("starRating") or "" + review = properties.get("review") or "" + rated_by, create_time = _created(rating) + rating_rows.append((create_time, (star_label(star_rating), review, rated_by, _posted(create_time)))) + if STAR_VALUES.get(star_rating): + stars.append(STAR_VALUES[star_rating]) + if rated_by == self.user_name: + my_rating = (star_rating, review) + rating_rows.sort(key=lambda r: r[0], reverse=True) + complete = combine(comments, likes, ratings) + note = None if complete else ("Egeria didn't confirm every comment, like and rating was read, " + "so there may be more than shown") + return { + "comments": self._comment_rows(comments), + "likes": len(like_items), + "i_like": any(_created(like)[0] == self.user_name for like in like_items), + "ratings": [row for _, row in rating_rows], + "average": sum(stars) / len(stars) if stars else None, + "my_rating": my_rating, + "complete": complete, + "note": note, + } + + def _feedback_call(self, action, done: str, failed: str) -> bool: + client = None + try: + client = self._community_client() + action(client) + self.notify(done, timeout=10, severity="information") + return True + except PyegeriaException as e: + # self.log(f"{failed}: {e}") + self.notify(f"{failed}: {e}", timeout=10, severity="error") + return False + finally: + if client: + client.close_session() + + def like_element(self, element_guid: str) -> bool: + """Like an element. Returns True on success.""" + body = {"class": "NewAttachmentRequestBody", + "properties": {"class": "LikeProperties", "emoji": LIKE_EMOJI}} + return self._feedback_call(lambda client: client.add_like_to_element(element_guid, body=body), + "You like this", "Adding your like failed") + + def unlike_element(self, element_guid: str) -> bool: + """Remove the user's like from an element. Returns True on success.""" + return self._feedback_call(lambda client: client.remove_like_from_element(element_guid), + "Your like has been removed", "Removing your like failed") + + def rate_element(self, element_guid: str, star_rating: str, review: str = "") -> bool: + """Give an element a star rating (one of STAR_VALUES) and an optional review, replacing any + earlier rating by the user. Returns True on success.""" + properties = {"class": "RatingProperties", "starRating": star_rating} + if review: + properties["review"] = review + body = {"class": "NewAttachmentRequestBody", "properties": properties} + return self._feedback_call(lambda client: client.add_rating_to_element(element_guid, True, body), + f"You rated it {star_label(star_rating)}", "Adding your rating failed") + def create_community(self, name: str, description: str, mission: str = "", assignment_type: str = CREATOR_ASSIGNMENT_TYPES[0]) -> str | None: """Create a community, then link the user's profile to it with assignment_type (none if empty). @@ -445,6 +635,7 @@ def create_community(self, name: str, description: str, mission: str = "", # self.log(f"Joining new community {community_guid} failed: {e}") self.notify(f"Community {name} created, but joining it failed: {e}", timeout=10, severity="warning") return community_guid + self._communities_changed() self.notify(f"Community {name} created, and you have joined it as its {assignment_type}", timeout=10, severity="information") return community_guid @@ -469,6 +660,7 @@ def join_community(self, community_guid: str, assignment_type: str = JOIN_ASSIGN try: client = self._community_client() client.link_assignment_scope(community_guid, self.user_GUID, body) + self._communities_changed() self.notify(f"You have joined the community as a {assignment_type}", timeout=10, severity="information") return True @@ -487,6 +679,7 @@ def leave_community(self, community_guid: str) -> bool: client = self._community_client() # same GUID order as link_assignment_scope: scope (the community) first, actor second client.detach_assignment_scope(community_guid, self.user_GUID, {"class": "DeleteRelationshipRequestBody"}) + self._communities_changed() self.notify("You have left the community", timeout=10, severity="information") return True except PyegeriaException as e: diff --git a/my_egeria/my_egeria/my_profile.py b/my_egeria/my_egeria/my_profile.py index cb8d5957..0e03ac18 100644 --- a/my_egeria/my_egeria/my_profile.py +++ b/my_egeria/my_egeria/my_profile.py @@ -739,6 +739,8 @@ async def handle_option_selected(self, event: OptionList.OptionSelected) -> None self.show_my_network() elif selected_option == "Work with Communities": self.show_communities() + elif selected_option == "Find Community": + self.show_communities(find=True) elif selected_option == "Change User": await self.push_screen(ChangeUserScreen(), callback=self.change_user_callback) elif selected_option == "Subscriptions": @@ -762,6 +764,14 @@ def _on_roles_row_selected(self, event: DataTable.RowSelected) -> None: """ self.handle_roles_table_row_selection(event) + @on(DataTable.RowSelected, "#associations_table") + def _on_associations_row_selected(self, event: DataTable.RowSelected) -> None: + """Open the community in a User Associations row: its members, blog, resources and comments. + + Registered here rather than on CommunitiesMixin, for the same reason as the roles handler. + """ + self.open_community_row(event.data_table, event.row_key) + def show_main_screen(self) -> None: """Show or switch back to the main screen by unwinding the screen stack.""" # self.log("Returning to main screen") diff --git a/my_egeria/my_egeria/my_profile.tcss b/my_egeria/my_egeria/my_profile.tcss index 4e51e961..683c690b 100644 --- a/my_egeria/my_egeria/my_profile.tcss +++ b/my_egeria/my_egeria/my_profile.tcss @@ -13,6 +13,8 @@ Screen { layout: grid; grid-size: 3; grid-gutter: 1; + background: #1e1e2e; + color: #cdd6f4; } #main_screen { @@ -22,18 +24,23 @@ Screen { /*grid-rows: 3 1fr 1fr 1fr 1;*/ grid-rows: 1fr 1fr 1fr; grid-gutter: 1; + background: #1e1e2e; } #main_header { column-span: 3; height: 3; width: 100%; + background: #11111b; + color: #89b4fa; } #main_footer { column-span: 3; height: 1; width: 100%; + background: #11111b; + color: #cdd6f4; } ModalScreen { @@ -72,17 +79,26 @@ MyNetworkScreen { column-span: 2; width: 100%; height: 100%; + border: round #89b4fa; + background: #181825; + padding: 0 1; } #people_search_container { width: 100%; height: 100%; + border: round #89b4fa; + background: #181825; + padding: 0 1; } #peer_details_container { column-span: 3; width: 100%; height: 100%; + border: round #89b4fa; + background: #181825; + padding: 0 1; } #my_network_table { @@ -139,7 +155,7 @@ CommunitiesScreen { max-height: 12; } -/* Title row, then the blog + members side by side, then the comments */ +/* Title row, then the blog + members side by side, then the resources + comments */ CommunityScreen { grid-rows: auto 1fr 1fr; background: $background; @@ -156,37 +172,88 @@ CommunityScreen { height: 100%; } +#community_resources_container { + column-span: 2; + width: 100%; + height: 100%; +} + #community_comments_container { - column-span: 3; width: 100%; height: 100%; } #community_posts_table, #community_members_table, +#community_resources_table, #community_comments_table { height: auto; max-height: 12; } +#community_blog_buttons, +#community_join_controls { + height: auto; +} + +#community_join_assignment_type { + width: 20; +} + +/* Title row, the like/rating summary, then the ratings + comments side by side */ +ElementFeedbackScreen { + grid-rows: auto auto 1fr; + background: $background; +} + +#feedback_summary_container { + column-span: 3; + height: auto; +} + +#feedback_buttons { + height: auto; +} + +#element_ratings_container { + width: 100%; + height: 100%; +} + +#element_comments_container { + column-span: 2; + width: 100%; + height: 100%; +} + +#element_ratings_table, +#element_comments_table { + height: auto; + max-height: 16; +} + #community_post_text { padding: 1 0; } -NewBlogPostScreen, AddCommunityCommentScreen { +NewBlogPostScreen, AddCommunityCommentScreen, RateElementScreen { grid-rows: auto 1fr; background: $background; } -#new_blog_post_container, #add_community_comment_container, #create_community_container { +#new_blog_post_container, #add_community_comment_container, #create_community_container, #rate_element_container { column-span: 3; } -#new_blog_post_container Horizontal, #add_community_comment_container Horizontal, #create_community_container Horizontal { +#new_blog_post_container Horizontal, #add_community_comment_container Horizontal, #create_community_container Horizontal, #rate_element_container Horizontal { height: auto; } -#blog_post_text, #community_comment_text, #community_mission_text { +#star_rating_select { + width: 30; +} + +#blog_post_text, #community_comment_text, #community_mission_text, #rating_review_text { height: 10; } @@ -194,23 +261,45 @@ ModalScreen > Header, ModalScreen > Footer { column-span: 3; } -#communities_title, #community_title, #new_blog_post_title, #add_community_comment_title, #create_community_title, #confirm_leave_community_title, #my_network_title, #confirm_remove_peer_title, #edit_profile_title, #create_profile_title, #edit_teams_title, #edit_roles_title, #edit_communities_title, #edit_identities_title, #status_title, #generic_data_view_title, #feedback_title, #feedback_log_title, .span-3 { +#communities_title, #community_title, #element_feedback_title, #rate_element_title, #new_blog_post_title, #add_community_comment_title, #create_community_title, #confirm_leave_community_title, #my_network_title, #confirm_remove_peer_title, #edit_profile_title, #create_profile_title, #edit_teams_title, #edit_roles_title, #edit_communities_title, #edit_identities_title, #status_title, #generic_data_view_title, #feedback_title, #feedback_log_title, .span-3 { column-span: 3; text-align: center; - background: darkblue; - color: white; + background: #89b4fa; + color: #11111b; + text-style: bold; } #main_projects_container, #main_communities_container, #main_roles_container, #main_teams_container, -#main_activities_container, -#main_identities_container, #other_function_container, -#other_function_list { +#other_function_list, +#main_associations_container, +#my_collections_container { + width: 100%; + height: 100%; + border: round #89b4fa; + background: #181825; +} + +#main_activities_container { width: 100%; height: 100%; + background: #181825; +} + +#main_identities_container { + column-span: 2; + width: 100%; + height: 100%; + border: round #89b4fa; + background: #181825; +} + +#main_screen ScrollableContainer > Static { + color: #89b4fa; + text-style: bold; } #edit_roles_container, @@ -229,11 +318,12 @@ ModalScreen > Header, ModalScreen > Footer { #user_identities_container, #generic_data_view_container, #feedback_container, -#feedback_log_container, -#main_identities_container { +#feedback_log_container { column-span: 2; width: 100%; height: 100%; + border: round #89b4fa; + background: #181825; } #technology_types_table, @@ -267,14 +357,25 @@ ModalScreen > Header, ModalScreen > Footer { } Button { - /*background: greenyellow;*/ - color: black; - border: solid white; + background: #89b4fa; + color: #11111b; + border: none; + text-style: bold; +} + +Button:focus { + background: #a6e3a1; + color: #11111b; +} + +Button:hover { + background: #b4befe; + color: #11111b; } DataTable, TextArea, SelectionList, Tree, OptionList { - color: white; - background: black; + color: #cdd6f4; + background: #181825; } DataTable { @@ -282,18 +383,19 @@ DataTable { } DataTable:focus { - border: heavy green; - background: $surface-lighten-1; + border: heavy #a6e3a1; + background: #1e1e2e; } DataTable > .datatable--header { - background: darkblue; - color: white; + background: #313244; + color: #89b4fa; text-style: bold; } DataTable > .datatable--cursor, OptionList > .option-list--cursor { - background: greenyellow; - color: black; + background: #a6e3a1; + color: #11111b; + text-style: bold; } #edit_profile_form Horizontal { @@ -319,43 +421,55 @@ DataTable > .datatable--cursor, OptionList > .option-list--cursor { Input { width: 1fr; height: auto; - border: solid white; + border: round #89b4fa; + background: #11111b; + color: #cdd6f4; } -#show_comment_textarea_container { - width: 100%; - height: 1fr; - border: solid white; + +Input:focus { + border: round #a6e3a1; } + +#show_comment_textarea_container, #add_comment_textarea_container { width: 100%; height: 1fr; - border: solid white; + border: round #89b4fa; + background: #181825; } + #feedback_text { width: 100%; height: 10; - border: solid white; + border: round #89b4fa; + background: #11111b; + color: #cdd6f4; } + #feedback_container Horizontal { height: auto; } + .SplashScreen { - align: center middle; /* 1. Center the entire card on the screen */ - background: rgba(0, 0, 0, 0.6); /* Dim the 3x3 grid behind it */ - } + align: center middle; /* 1. Center the entire card on the screen */ + background: rgba(30, 30, 46, 0.7); /* Dim the 3x3 grid behind it */ +} + #splash-card { - background: $panel; - border: thick $accent; - padding: 2 4; - width: 80; - height: 24; - /* 2. Vertically pushes layout content down and horizontally centers items */ - align: center bottom; - } + background: #181825; + border: thick #89b4fa; + padding: 2 4; + width: 80; + height: 24; + /* 2. Vertically pushes layout content down and horizontally centers items */ + align: center bottom; +} + #splash-text { text-align: center; width: 100%; margin-bottom: 2; + color: #cdd6f4; } #splash_buttons { @@ -368,22 +482,30 @@ Input { #splash_buttons Button { margin: 0 1; /* Space out the two buttons side by side */ } + .empty { - background: transparent; /* Invisible structural spacer */ - } + background: transparent; /* Invisible structural spacer */ +} + ConnectionErrorScreen, ChangeUserScreen { align: center middle; - background: rgba(0, 0, 0, 0.6); + background: rgba(30, 30, 46, 0.7); } -#connection_error_card, #change_user_card { - background: $panel; - border: thick $error; + +#connection_error_card { + background: #181825; + border: thick #f38ba8; padding: 1 2; width: 90; height: auto; } + #change_user_card { - border: thick $accent; + background: #181825; + border: thick #89b4fa; + padding: 1 2; + width: 90; + height: auto; } #connection_error_reason { color: $error; diff --git a/my_egeria/my_egeria/recording/BUSINESS_VALUE_SERIES_SCRIPTS.md b/my_egeria/my_egeria/recording/BUSINESS_VALUE_SERIES_SCRIPTS.md new file mode 100644 index 00000000..2c3ab47e --- /dev/null +++ b/my_egeria/my_egeria/recording/BUSINESS_VALUE_SERIES_SCRIPTS.md @@ -0,0 +1,189 @@ +# Coco Pharmaceuticals Business Value Video Series — Production Scripts & Cue Sheets + +**Series Context:** *Coco Pharmaceuticals Data Governance Journey* +**Executive Sponsor:** Jules Keeper (Chief Data Officer) & Reggie Mint (Chief Financial Officer) +**Practitioner Anchor:** Companion to Mandy Chessell's Jules Keeper Video Series ([prd-associates.com](https://www.prd-associates.com)) and Egeria Technical Blog Series (Parts 6–10). +**Format:** 1080p Full HD video blog episodes combining presenter dialogue, Catppuccin Mocha title cards/slide decks, and automated terminal recordings (`my_profile.py`). +**Voice Synthesis:** Microsoft Azure Neural TTS (`edge-tts`) with distinct regional accents per persona, plus offline macOS `say` support. + +--- + +## Technical Blog Mappings (Egeria 6.3 Release & Architecture) + +Each episode connects directly to the architectural patterns and technical features detailed in Mandy Chessell's latest technical blog series ([Egeria Docs Blogs](https://github.com/odpi/egeria-docs/tree/main/blogs)): + +| Episode | Lead Persona | Topic | Primary Technical Blog References | +|:---|:---|:---|:---| +| **Episode 0** | **Callie Quartile** | Practitioner Cockpit & Shopping for Data | **Blog 6** (Autonomous Housekeeping & Karma), **Blog 8** (Bitol ODCS Contracts), **Blog 10** (DCAT v3 Exchange) | +| **Episode 1** | **Faith Broker** | Automating Data Privacy & GDPR Compliance | **Blog 7** (NIST CSF 2.0 / SP 800-53 OSCAL Catalogs, ISO 27001 SoA, Fail-Closed Security Zones) | +| **Episode 2** | **Peter Profile** | Zero-Touch Stewardship & Curation at Scale | **Blog 6** (Automated Connectors: `Simon`, `Pasteur`, `Mendel`; Reversible `Undo` / `Restore`) | +| **Episode 3** | **Dr. Tessa Tube** | Clinical Data Provenance & Trial Integrity | **Blog 8** (Bitol ODCS/ODPS, `Darwin` Dependency Validation) & **Blog 9** (5 Lineage Layers) | +| **Episode 4** | **Stew Faster** | Supply Chain Traceability & ESG Sustainability | **Blog 10** (W3C DCAT v3 JSON-LD Document Exchange & Connectors: Files, Events, Publishers) | +| **Episode 5** | **Erin Overview** | Business Glossaries & Technical Architecture | **Blog 9** (PostgreSQL-backed OpenLineage graph analysis, 5 Lineage Layers, Bounded Exploration) | +| **Episode 6** | **Sidney Seeker** | Forensic Auditing & Fraud Investigation | **Blog 6** (Reversible Operations) & **Blog 7** (STRIDE Threat Models, Tamper-Evident History) | +| **Episode 7** | **Jules Keeper & Reggie Mint** | Executive Cockpit: Measuring Data ROI | **Blog 6** (TCO Reduction & Autonomous Housekeeping), **Blog 7** (Security Posture), **Blog 8** (Data Mesh ROI) | + +--- + +## Episode 1: Automating Data Privacy & GDPR Compliance + +* **Lead Persona:** Faith Broker (`faithbroker`) — HR Director & Data Privacy Officer +* **Voice Profile:** `en-GB-LibbyNeural` (British English, Authoritative & Professional) +* **Business Driver:** Regulatory Compliance (GDPR, HIPAA), Privacy Governance & Risk Mitigation +* **Target Audience:** Data Privacy Officers, Compliance Directors, Legal Counsel, HR Executives + +### Scene Breakdown & Teleprompter Cue Sheet + +#### Scene 0: Cold Open — The Compliance Challenge +* **Speaker:** Faith Broker +* **Visual:** Slide Card — Title: *EPISODE 1: AUTOMATING DATA PRIVACY & GDPR* +* **Callouts:** + - Challenge: Manual Subject Access Requests (SARs) take weeks of manual investigation + - Risk: GDPR & HIPAA non-compliance penalties and clinical patient exposure + - Egeria Solution: Automated classification tagging and unified metadata lineage + - Security Frameworks: NIST CSF 2.0 & ISO 27001 Statement of Applicability +* **Dialogue:** + > "Hello, I'm Faith Broker, HR Director and Data Privacy Officer at Coco Pharmaceuticals. When GDPR, HIPAA, and patient privacy regulations require you to locate, classify, and protect every piece of personally identifiable information across hundreds of clinical and employee databases, traditional manual audits simply cannot keep up. Today, I'm going to show you how open metadata with Egeria turns privacy compliance from a terrifying manual bottleneck into an automated, continuous operational safeguard." + +#### Scene 1: The Governance Context — Privacy Classifications +* **Speaker:** Faith Broker +* **Visual:** Slide Card — Title: *GOVERNANCE CLASSIFICATIONS & PRIVACY CONTROLS* +* **Callouts:** + - Security Content Pack: Built-in NIST CSF 2.0 and NIST SP 800-53 OSCAL catalogs + - Standardized Classifications: Confidentiality, Impact, Criticality, Retention + - Fail-Closed Security Zones: Operational metadata restricted by default + - ISO 27001 Mapping: Statement of Applicability directly linked to data assets +* **Dialogue:** + > "In our corporate governance council, Chief Data Officer Jules Keeper and I established clear governance classifications. Egeria integrates pre-packaged NIST CSF 2.0 and SP 800-53 security controls, aligning directly with our ISO 27001 Statement of Applicability. Every asset entering our data estate is automatically evaluated for Confidentiality, Impact, and Retention within fail-closed security zones. In my workspace, I can instantly see which clinical trial cohorts contain restricted patient identifiers, and ensure that governance action engines enforce masking policies before research data is ever shared." + +#### Scene 2: Workspace Navigation — Privacy Role & Team Hierarchy +* **Speaker:** Faith Broker +* **Visual:** Terminal — `uv run my_profile --user faithbroker` (Roles & My Team drill-down) +* **Dialogue:** + > "Let's step into the My Profile workspace. Logging in as Faith Broker, the system immediately loads my assigned governance roles. Notice my association with the Corporate Privacy Council and HR Leadership team. I can drill down into my team hierarchy, verifying that Tanya Tidie in Clinical Records and our data stewards have completed their privacy review to-dos for the newest clinical hospital feeds." + +#### Scene 3: Subject Access Request (SAR) & PII Lineage +* **Speaker:** Faith Broker +* **Visual:** Terminal — Shop for Data & My Network graph filtering for Restricted PII +* **Dialogue:** + > "When an employee or clinical trial participant submits a Subject Access Request, we no longer need to email dozens of database administrators. Using the unified catalog search and My Network graph, I can trace the entire information supply chain for patient identifiers. From raw ingestion tables in PostgreSQL to S3 data lakes and analytics marts, Egeria maps the full lineage and confidential relationships in seconds." + +#### Scene 4: Real-Time Privacy Stewardship & Feedback +* **Speaker:** Faith Broker +* **Visual:** Terminal — To-Dos panel, `Ctrl+N` for audit comment, `Ctrl+F` for feedback modal +* **Dialogue:** + > "If a new dataset lacks appropriate retention rules, I can immediately tag a to-do, add an audit comment with Control-N, or submit real-time feedback with Control-F directly to the engineering team. Our data stewards earn Karma Points for completing privacy validations, making compliance a collaborative, rewarding process." + +#### Scene 5: Business Impact & Executive Takeaways +* **Speaker:** Faith Broker +* **Visual:** Slide Card — Title: *BUSINESS IMPACT & EXECUTIVE ROI* +* **Callouts:** + - 95% Reduction in SAR Response Time: From 3 weeks to under 2 hours + - Zero-Defect Audit Readiness: Continuous automated metadata classification + - Fail-Closed Protection: Operational secrets and PII strictly isolated + - Explore Egeria: egeria-project.org • Egeria Security Blog Series Part 7 +* **Dialogue:** + > "By anchoring privacy governance in Egeria's open metadata ecosystem, Coco Pharmaceuticals reduced Subject Access Request response times from three weeks down to under two hours, while eliminating the risk of regulatory fines. Data privacy is no longer a barrier to research—it's the trusted foundation that enables our scientists to innovate with confidence. Thank you for joining me!" + +--- + +## Episode 2: Zero-Touch Data Stewardship & Curation at Scale + +* **Lead Persona:** Peter Profile (`peterprofile`) — Information Analyst & Lead Data Steward +* **Voice Profile:** `en-GB-RyanNeural` (British English, Engaging & Methodical) +* **Business Driver:** Data Quality, Semantic Consistency & Curation Productivity +* **Target Audience:** Data Stewards, Information Analysts, Data Governance Teams, Analytics Engineers + +### Key Technical Mechanisms (Blog 6) +- **Housekeeping Connectors:** `Simon` (Karma calculation), `Pasteur` (task/process archival), `Mendel` (tag deduplication). +- **Native Reversibility:** Full `Undo` and cascading `Restore` for schemas, tables, and relationships. + +--- + +## Episode 3: Trusted Clinical Data Provenance & Trial Integrity + +* **Lead Persona:** Dr. Tessa Tube (`tessatube`) — Lead Clinical Researcher +* **Voice Profile:** `en-US-AriaNeural` (American English, Academic & Authoritative) +* **Business Driver:** Scientific Integrity, Regulatory Filings (FDA/EMA), Audit Provenance +* **Target Audience:** Clinical Researchers, R&D Directors, Biostatisticians, Regulatory Affairs + +### Key Technical Mechanisms (Blogs 8 & 9) +- **Data Mesh & Contracts:** Bitol Open Data Contract Standard (ODCS) and Open Data Product Standard (ODPS). +- **Darwin Governance Engine:** Validates runtime pipeline lineage events against declared contracts, alerting on unmapped dependencies. +- **Five Lineage Layers:** Business Flow, Data Mesh, System Fabric, Data Fabric, Data Mapping. +- **Differential Delivery:** `Baudot` & `Wedgwood` connectors provision delta updates without event storming. + +--- + +## Episode 4: Supply Chain Traceability & ESG Sustainability + +* **Lead Persona:** Stew Faster (`stewfaster`) — Manufacturing General Manager +* **Voice Profile:** `en-AU-WilliamMultilingualNeural` (Australian English, Direct & Pragmatic) +* **Business Driver:** ESG Compliance, Operational Efficiency, Carbon Accounting, IoT Telemetry +* **Target Audience:** COOs, Supply Chain Executives, Sustainability Officers, IoT Engineers + +### Key Technical Mechanisms (Blog 10) +- **W3C DCAT v3 Standards:** Bidirectional exchange using JSON-LD across Catalogs, Datasets, Distributions, and DataServices. +- **Connectors:** `DCAT Files Receiver`, `DCAT Event Receiver`, `DCAT Cataloguer`, and `DCAT Publisher`. +- **Industrial IoT:** Connecting MQTT/Kafka sensor telemetry and SCADA gateways to enterprise ESG ledgers. + +--- + +## Episode 5: Connecting Business Glossaries to Technical Architecture + +* **Lead Persona:** Erin Overview (`erinoverview`) — Enterprise Information Architect +* **Voice Profile:** `en-US-JennyNeural` (American English, Clear & Visionary) +* **Business Driver:** Semantic Interoperability, Architecture Governance, Vendor Neutrality +* **Target Audience:** Enterprise Architects, Chief Architects, Data Modelers, Integration Engineers + +### Key Technical Mechanisms (Blog 9) +- **High-Speed OpenLineage in PostgreSQL:** Analyzing 864,000 events in ~3 seconds (vs. 19 minutes previously). +- **Bounded Graph Exploration:** Default 8-hop radius queries preventing client/terminal freezes. +- **Semantic Federation:** Mapping `ISA` and `HASA` relationships across heterogeneous storage stacks. + +--- + +## Episode 6: Forensic Data Auditing & Fraud Investigation + +* **Lead Persona:** Sidney Seeker (`sidneyseeker`) — Lead Fraud Investigator & Security Auditor +* **Voice Profile:** `en-IE-ConnorNeural` (Irish English, Sharp & Methodical) +* **Business Driver:** Fraud Detection, Security Compliance, Shadow IT Discovery, Audit Defense +* **Target Audience:** Internal Auditors, CISOs, Fraud Investigators, Security Analysts, Risk Officers + +### Key Technical Mechanisms (Blogs 6 & 7) +- **STRIDE Threat Models:** Built-in threat definitions (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege). +- **Tamper-Evident Lineage:** Timestamped audit trails with author attribution. +- **Reversible Forensics:** Instant `Restore` for maliciously deleted or corrupted schemas and relationships. + +--- + +## Episode 7: The Executive Cockpit: Measuring Data ROI & Strategy + +* **Lead Personas:** Jules Keeper (`juleskeeper` - CDO) & Reggie Mint (`reggiemint` - CFO) +* **Voice Profiles:** `en-GB-ThomasNeural` (Jules Keeper) & `en-US-GuyNeural` (Reggie Mint) +* **Business Driver:** Data Valuation, Executive ROI, Strategic Governance, Board Reporting +* **Target Audience:** CDOs, CFOs, CIOs, CEOs, Board Directors, Enterprise Leaders + +### Key Technical Mechanisms (Blogs 6, 7, 8) +- **Autonomous Housekeeping ROI:** Operational cost reduction via automated metadata cleanup. +- **Contract Adherence:** Enterprise-wide metrics on Bitol data product subscriptions and SLA compliance. +- **Bottom-Line Financial Impact:** Eliminating duplicate catalog tools and saving 75% discovery time across hundreds of analysts. + +--- + +## Video Distribution & Archival (Shared `pdr-vault`) + +All certified 1080p MP4 masters, neural audio tracks, slide PNGs, and manifests are exported to the shared `pdr-vault` directory: +- **Master Videos:** `videos/` +- **Audio Tracks:** `audio/` +- **Slide Graphics:** `slides/` +- **Manifest:** `vault_manifest.json` & `README.md` + +### CLI Commands +```bash +# Generate all episodes with automated vault export +./my_egeria/my_egeria/recording/run_series.sh --all --export-vault + +# Sync standalone +uv run python my_egeria/my_egeria/recording/export_vault.py --all-assets +``` diff --git a/my_egeria/my_egeria/recording/README.md b/my_egeria/my_egeria/recording/README.md index c217483b..3fdb1b89 100644 --- a/my_egeria/my_egeria/recording/README.md +++ b/my_egeria/my_egeria/recording/README.md @@ -1,153 +1,171 @@ -# My Profile — Automated Terminal Recording Suite +# Egeria Video Production & Automated Recording Suite -**Series:** *Coco Pharmaceuticals Data Governance Journey* -**Presenter Persona:** Callie Quartile (Data Scientist) -**Companion Script:** [`../VIDEO_SCRIPT_DRAFT.md`](../VIDEO_SCRIPT_DRAFT.md) - -This directory contains production-ready automated recording scripts that capture high-definition terminal screencasts and vector SVG frames of `my_profile.py` for video blogs, tutorials, presentations, and documentation. +Automated video generation, slide rendering, terminal interaction recording, neural speech synthesis, and video assembly framework for the **Coco Pharmaceuticals Video Series** in `pyegeria` / `my_profile`. --- -## 📁 Directory Structure +## Series Architecture -```text -recording/ -├── my_profile_full_walkthrough.tape # Full continuous end-to-end VHS recording (Scenes 2-6) -├── scenes/ -│ ├── scene2_launch_and_startup.tape # Scene 2: Splash screen, async loading, continue -│ ├── scene3_main_dashboard.tape # Scene 3: Dashboard grid, roles & team drill-down -│ ├── scene4_curation_todos_comments.tape # Scene 4: To-Dos (Ctrl+T), Comments (Ctrl+O/N), Bookmarks (Ctrl+K/B) -│ ├── scene5_shop_for_data.tape # Scene 5: Catalogs, sample data (Ctrl+L), subscribe (Ctrl+U) -│ └── scene6_network_communities_feedback.tape # Scene 6: My Network, communities, feedback (Ctrl+F), tech types -├── generate_voiceover.py # AI Text-to-Speech voiceover generator & video assembly pipeline -├── record_demo.py # Python Textual Pilot automated driver (generates vector SVG captures) -├── run_recordings.sh # Master runner script for VHS, Pilot captures, and narration -├── output/ # Generated recording artifacts (ignored by git) -│ ├── audio/ # Synthesized MP3 narration audio tracks per scene & full episode -│ ├── videos/ # Rendered raw 1080p MP4 videos -│ ├── videos_narrated/ # 1080p MP4 videos with synchronized voiceover narration -│ ├── gifs/ # Animated GIF screencasts -│ ├── slides/ # 1080p PNG slide title cards -│ └── captures/ # Timestamped SVG/PNG screenshots per scene -└── README.md # This documentation guide -``` +The repository contains two complementary video suites: +1. **Practitioner Walkthrough (Callie Quartile - Episode 0)**: In-depth interactive walkthrough of the `my_profile.py` textual cockpit (launch, dashboard, to-dos/curation, shop for data, subscriptions, and networks). +2. **Business Value Series (Episodes 1 through 7)**: 7 modular video blog episodes covering enterprise ROI and governance value propositions presented by authentic Coco Pharmaceuticals personas with unique regional accents and voices. --- -## 🛠️ Prerequisites & Installation +## Business Value Episodes & Persona Voice Roster -### Option A: VHS Recording (Recommended for MP4/GIF Video Capture) -Charm's [VHS](https://github.com/charmbracelet/vhs) runs headless terminal scripts with exact millisecond timings and renders 1080p, 60fps MP4 and GIF videos. +| Ep # | Title & Value Driver | Lead Persona & Role | Assigned Voice | Accent & Character | +|:---:|:---|:---|:---|:---| +| **0** | **Shop for Data & Governed Research** | Callie Quartile (Data Scientist) | `en-GB-SoniaNeural` | British English (Engaging & Clear) | +| **1** | **Automating Data Privacy & GDPR** | Faith Broker (Privacy Officer & HR) | `en-GB-LibbyNeural` | British English (Authoritative & Rigorous) | +| **2** | **Zero-Touch Data Stewardship** | Peter Profile (Lead Data Steward) | `en-GB-RyanNeural` | British English (Methodical & Enthusiastic) | +| **3** | **Clinical Data Provenance & Integrity** | Dr. Tessa Tube (Lead Clinical Researcher) | `en-US-AriaNeural` | American English (Academic & Authoritative) | +| **4** | **Supply Chain & ESG Sustainability** | Stew Faster (Manufacturing GM) | `en-AU-WilliamMultilingualNeural` | Australian English (Direct & Pragmatic) | +| **5** | **Business Glossaries & Architecture** | Erin Overview (Information Architect) | `en-US-JennyNeural` | American English (Strategic & Visionary) | +| **6** | **Forensic Auditing & Fraud Investigation** | Sidney Seeker (Lead Fraud Auditor) | `en-IE-ConnorNeural` | Irish English (Inquisitive & Sharp) | +| **7** | **The Executive Cockpit & Measuring ROI** | Jules Keeper (CDO) & Reggie Mint (CFO) | `en-GB-ThomasNeural` & `en-US-GuyNeural` | British Executive & American CFO | -- **macOS (Homebrew):** - ```bash - brew install vhs ffmpeg ttyd - ``` -- **Linux (Debian / Ubuntu):** - ```bash - sudo apt-get install ffmpeg ttyd - go install github.com/charmbracelet/vhs@latest - ``` +--- -### Option B: Textual Pilot Recorder (Python Native, No External Binary Needed) -Built on Textual's async `Pilot` framework, included out of the box with the project dependencies (`uv sync`). +## Directory Layout ---- +``` +recording/ +├── episode_specs.py # Narrative scripts, scene breakdowns & persona voice metadata +├── generate_series_videos.py # Multi-episode video & slide generator (Episodes 1 to 7) +├── generate_voiceover.py # Callie Quartile foundational walkthrough generator (Episode 0) +├── run_series.sh # Master CLI runner for the Business Value Video Series +├── run_recordings.sh # Terminal recording runner (VHS tapes + headless pilot) +├── my_profile_full_walkthrough.tape # Full 1080p VHS terminal recording script +├── record_demo.py # Headless Textual Pilot automated SVG/PNG frame capture +├── README.md # This documentation guide +└── output/ + ├── episode1/ ... episode7/ # Per-episode audio, 1080p slide PNGs, scene clips & master MP4s + │ ├── audio/ # Scene MP3 voiceovers & full stitched episode audio + │ ├── slides/ # 1080p Catppuccin Mocha slide cards + │ └── videos_narrated/ # Assembled master MP4 episodes + ├── audio/ # Callie Quartile voiceover tracks + ├── slides/ # Callie Quartile intro/outro slides + ├── videos/ # Raw raster terminal recordings from VHS + └── videos_narrated/ # Callie Quartile assembled video blog master +``` -## 🚀 Running Recordings +--- -### 1. Master Runner Script (`run_recordings.sh`) +## Generating & Rebuilding Video Episodes +### 1. Generate the Entire Business Value Series (Episodes 1–7) ```bash -# Make sure the script is executable -chmod +x my_egeria/my_egeria/recording/run_recordings.sh +./my_egeria/my_egeria/recording/run_series.sh --all +``` -# Record the full continuous walkthrough (Scenes 2 to 6) -./my_egeria/my_egeria/recording/run_recordings.sh --walkthrough +### 2. Generate and Automatically Export to `pdr-vault` +```bash +# Generate all episodes and auto-sync master videos to the shared vault +./my_egeria/my_egeria/recording/run_series.sh --all --export-vault -# Record all individual scene clips -./my_egeria/my_egeria/recording/run_recordings.sh --scenes +# Generate a single episode and auto-sync with all audio/slide assets +./my_egeria/my_egeria/recording/run_series.sh --episode 1 --export-vault --all-assets +``` -# Record a specific scene clip (e.g. Scene 5: Shop for Data) -./my_egeria/my_egeria/recording/run_recordings.sh --scene 5 +### 3. Generate an Individual Episode +```bash +# Generate Episode 1 (Faith Broker - Data Privacy) +./my_egeria/my_egeria/recording/run_series.sh --episode 1 -# Generate all vector SVG screenshots via Textual Pilot -./my_egeria/my_egeria/recording/run_recordings.sh --pilot +# Generate Episode 2 (Peter Profile - Stewardship) +./my_egeria/my_egeria/recording/run_series.sh --episode 2 -# Generate AI Voiceover narration (Callie Quartile) and merge with recorded video clips -./my_egeria/my_egeria/recording/run_recordings.sh --narrate +# Generate Episode 4 (Stew Faster - Supply Chain) +./my_egeria/my_egeria/recording/run_series.sh --episode 4 +``` -# Assemble the complete 5-minute master video blog episode (Scenes 0-7) -./my_egeria/my_egeria/recording/run_recordings.sh --assemble +### 4. Generate Audio Tracks or Slide Cards Only +```bash +# Audio tracks only +./my_egeria/my_egeria/recording/run_series.sh --all --audio-only -# Run everything (walkthrough + all individual scenes + SVG captures + narration + master episode) -./my_egeria/my_egeria/recording/run_recordings.sh --all +# Slide cards only +./my_egeria/my_egeria/recording/run_series.sh --all --slides-only ``` -### 2. Running Voiceover Generator Directly (`generate_voiceover.py`) +--- + +## Centralized Video Vault Synchronization (`pdr-vault`) +To prevent repository bloat while providing a certified, single source of truth for distribution, master video exports and media assets are synchronized to the shared `pdr-vault` repository. + +### 1. On-Demand Vault Synchronization +Run the standalone vault exporter anytime to sync master videos to the vault: ```bash -# Generate narration and merge videos using Edge Neural TTS (default: en-GB-SoniaNeural) -uv run python my_egeria/my_egeria/recording/generate_voiceover.py --assemble +# Default destination ($PDR_VAULT_PATH or /Users/Shared/pdr-vault) +uv run python my_egeria/my_egeria/recording/export_vault.py -# Use native macOS speech synthesizer (offline fallback) -uv run python my_egeria/my_egeria/recording/generate_voiceover.py --tts say --voice "Flo (English (UK))" --assemble +# Custom vault location +uv run python my_egeria/my_egeria/recording/export_vault.py --vault-path /Volumes/pdr-vault -# Generate narration audio files only -uv run python my_egeria/my_egeria/recording/generate_voiceover.py --audio-only +# Export all media assets (master MP4s, per-scene audio MP3s, slide PNGs) +uv run python my_egeria/my_egeria/recording/export_vault.py --all-assets -# Target a specific scene (e.g. Scene 5) -uv run python my_egeria/my_egeria/recording/generate_voiceover.py --scene 5 +# Dry-run preview +uv run python my_egeria/my_egeria/recording/export_vault.py --dry-run ``` -### 3. Running Individual VHS Tapes Directly +### 2. Destination Resolution Hierarchy +The exporter automatically resolves the vault target in the following order: +1. Explicit `--vault-path` or `--export-vault ` CLI argument +2. `PDR_VAULT_PATH` or `EGERIA_VAULT_PATH` environment variable +3. `/Volumes/pdr-vault` (if mounted) +4. `/Users/Shared/pdr-vault` (default local shared directory) +5. `~/pdr-vault` -```bash -cd my_egeria/my_egeria/recording +### 3. Vault Structure & Metadata Index +Upon synchronization, `export_vault.py` writes a `README.md` and `vault_manifest.json` inside the vault directory with file checksums, sizes, durations, and episode personas: +``` +pdr-vault/ +├── README.md # Vault index catalog and playback guidance +├── vault_manifest.json # Machine-readable sync manifest with checksums +├── videos/ # 1080p Master MP4 video episodes +├── audio/ # (Optional) Synthesized neural voiceover MP3 tracks +└── slides/ # (Optional) 1080p Catppuccin Mocha slide PNGs +``` -# Run full walkthrough tape -vhs my_profile_full_walkthrough.tape +--- -# Run a specific scene tape -vhs scenes/scene5_shop_for_data.tape -``` +## How to Play and View the Rendered Videos -### 4. Running the Python Pilot Driver Directly +On macOS, you can launch the completed MP4 video files directly in QuickTime Player or open their folders in Finder: ```bash -# Record all scenes headlessly to SVG -uv run python my_egeria/my_egeria/recording/record_demo.py +# Episode 1: Faith Broker (Data Privacy & GDPR) +open my_egeria/my_egeria/recording/output/episode1/videos_narrated/coco_pharma_episode_1_faith_broker.mp4 -# Record a specific scene -uv run python my_egeria/my_egeria/recording/record_demo.py --scene 4 +# Episode 2: Peter Profile (Zero-Touch Stewardship) +open my_egeria/my_egeria/recording/output/episode2/videos_narrated/coco_pharma_episode_2_peter_profile.mp4 -# Specify custom output directory -uv run python my_egeria/my_egeria/recording/record_demo.py --output-dir ./my_captures -``` +# Episode 3: Dr. Tessa Tube (Clinical Data Provenance) +open my_egeria/my_egeria/recording/output/episode3/videos_narrated/coco_pharma_episode_3_dr_tessa_tube.mp4 ---- +# Episode 4: Stew Faster (Supply Chain & ESG Sustainability) +open my_egeria/my_egeria/recording/output/episode4/videos_narrated/coco_pharma_episode_4_stew_faster.mp4 -## ⚙️ Environment Configuration +# Episode 5: Erin Overview (Business Glossaries & Architecture) +open my_egeria/my_egeria/recording/output/episode5/videos_narrated/coco_pharma_episode_5_erin_overview.mp4 -You can customize the target persona and Egeria platform environment variables before executing recordings: +# Episode 6: Sidney Seeker (Forensic Auditing & Fraud Investigation) +open my_egeria/my_egeria/recording/output/episode6/videos_narrated/coco_pharma_episode_6_sidney_seeker.mp4 -```bash -# Set persona to Callie Quartile (default) -export EGERIA_USER_NAME="calliequartile" +# Episode 7: Jules Keeper & Reggie Mint (The Executive Cockpit & ROI) +open my_egeria/my_egeria/recording/output/episode7/videos_narrated/coco_pharma_episode_7_jules_keeper_reggie_mint.mp4 -# Set custom view server or platform URL if recording against live or demo servers -export EGERIA_VIEW_SERVER="qs-view-server" -export EGERIA_PLATFORM_URL="https://127.0.0.1:9443" +# Open the Master Episodes Folder in macOS Finder +open my_egeria/my_egeria/recording/output ``` --- -## 🎬 Video Production Workflow +## Required Toolchain & Dependencies -1. **Audio Narration:** Record spoken voiceover according to [`VIDEO_SCRIPT_DRAFT.md`](../VIDEO_SCRIPT_DRAFT.md). -2. **Video Capture:** Execute `./run_recordings.sh --walkthrough` to generate `output/videos/my_profile_full_walkthrough.mp4` (or use individual scene `.mp4` files from `output/videos/`). -3. **Assembly & Editing:** In your video editor (e.g. DaVinci Resolve, Premiere Pro, Camtasia, CapCut, or via `ffmpeg`): - - Place the presenter intro / title card (Scene 0 & Scene 1). - - Sync the recorded `.mp4` terminal clips with Callie's narration for Scenes 2 through 6. - - Insert lower-thirds, hotkey overlays (`Ctrl+T`, `Ctrl+O`, `Ctrl+L`, `Ctrl+U`), and zoom spotlights onto active dialogs. - - Attach the closing call to action and link end cards (Scene 7). +- **Python Environment**: `uv pip install edge-tts Pillow textual rich` +- **System CLI Utilities (Homebrew)**: `brew install vhs ffmpeg ttyd` +- **Container / Video Standards**: 1080p H.264 video (`yuv420p`), 48kHz stereo AAC audio, QuickTime `-bf 0` stream safety, and `faststart` metadata for instant seeking. diff --git a/my_egeria/my_egeria/recording/episode_specs.py b/my_egeria/my_egeria/recording/episode_specs.py new file mode 100644 index 00000000..65eb6f34 --- /dev/null +++ b/my_egeria/my_egeria/recording/episode_specs.py @@ -0,0 +1,907 @@ +""" +Egeria Coco Pharmaceuticals Business Value Video Series - Episode Specifications. + +Defines the complete narrative scripts, scene breakdowns, UI cues, and persona +voice/accent profiles for the 7 business value video episodes. +""" + +from dataclasses import dataclass, field +from typing import List, Optional, Dict + + +@dataclass +class SceneSpec: + number: int + title: str + speaker: str + voice: str # Edge TTS voice ID + dialogue: str + slide_title: Optional[str] = None + slide_subtitle: Optional[str] = None + callouts: Optional[List[str]] = None + terminal_cue: Optional[str] = None + duration_estimate: float = 25.0 + + +@dataclass +class EpisodeSpec: + id: str + episode_number: int + title: str + subtitle: str + lead_persona: str + persona_username: str + persona_role: str + persona_team: str + business_driver: str + target_audience: str + voice_name: str + voice_locale: str + voice_gender: str + voice_accent: str + voice_description: str + offline_say_voice: str + scenes: List[SceneSpec] = field(default_factory=list) + + +# ============================================================================== +# EPISODE ROSTER & SPECIFICATIONS +# ============================================================================== + +EPISODES: Dict[str, EpisodeSpec] = { + # -------------------------------------------------------------------------- + # Episode 1: Faith Broker - Data Privacy & GDPR Compliance + # -------------------------------------------------------------------------- + "episode1": EpisodeSpec( + id="episode1", + episode_number=1, + title="Automating Data Privacy & GDPR Compliance", + subtitle="Zero-Friction PII Governance & Subject Access Requests at Coco Pharmaceuticals", + lead_persona="Faith Broker", + persona_username="faithbroker", + persona_role="HR Director & Data Privacy Officer", + persona_team="Corporate Governance & Regulatory Compliance", + business_driver="Regulatory Compliance, Privacy Governance & Risk Mitigation", + target_audience="Data Privacy Officers, Compliance Directors, Legal Counsel, HR Executives", + voice_name="en-GB-LibbyNeural", + voice_locale="en-GB", + voice_gender="Female", + voice_accent="British English (Authoritative & Professional)", + voice_description="Refined, clear British executive tone emphasizing accountability and regulatory rigor.", + offline_say_voice="Samantha", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Compliance Challenge", + speaker="Faith Broker", + voice="en-GB-LibbyNeural", + dialogue=( + "Hello, I'm Faith Broker, HR Director and Data Privacy Officer at Coco Pharmaceuticals. " + "When GDPR, HIPAA, and patient privacy regulations require you to locate, classify, and protect " + "every piece of personally identifiable information across hundreds of clinical and employee databases, " + "traditional manual audits simply cannot keep up. Today, I'm going to show you how open metadata with " + "Egeria turns privacy compliance from a terrifying manual bottleneck into an automated, continuous operational safeguard." + ), + slide_title="EPISODE 1: AUTOMATING DATA PRIVACY & GDPR", + slide_subtitle="Operationalizing Privacy by Design • Faith Broker (Privacy Officer)", + callouts=[ + "Challenge: Manual Subject Access Requests (SARs) take weeks of manual investigation", + "Risk: GDPR & HIPAA non-compliance penalties and clinical patient exposure", + "Egeria Solution: Automated classification tagging and unified metadata lineage", + "Security Frameworks: NIST CSF 2.0 & ISO 27001 Statement of Applicability", + ], + ), + SceneSpec( + number=1, + title="The Governance Context - Privacy Classifications", + speaker="Faith Broker", + voice="en-GB-LibbyNeural", + dialogue=( + "In our corporate governance council, Chief Data Officer Jules Keeper and I established clear governance classifications. " + "Egeria integrates pre-packaged NIST CSF 2.0 and SP 800-53 security controls, aligning directly with our ISO 27001 Statement of Applicability. " + "Every asset entering our data estate is automatically evaluated for Confidentiality, Impact, and Retention within fail-closed security zones. " + "In my workspace, I can instantly see which clinical trial cohorts contain restricted patient identifiers, and ensure that " + "governance action engines enforce masking policies before research data is ever shared." + ), + slide_title="GOVERNANCE CLASSIFICATIONS & PRIVACY CONTROLS", + slide_subtitle="Coco Pharmaceuticals Governance Framework • Policy to Execution", + callouts=[ + "Security Content Pack: Built-in NIST CSF 2.0 and NIST SP 800-53 OSCAL catalogs", + "Standardized Classifications: Confidentiality, Impact, Criticality, Retention", + "Fail-Closed Security Zones: Operational metadata restricted by default", + "ISO 27001 Mapping: Statement of Applicability directly linked to data assets", + ], + ), + SceneSpec( + number=2, + title="Workspace Navigation - Privacy Role & Team Hierarchy", + speaker="Faith Broker", + voice="en-GB-LibbyNeural", + dialogue=( + "Let's step into the My Profile workspace. Logging in as Faith Broker, the system immediately loads my assigned governance roles. " + "Notice my association with the Corporate Privacy Council and HR Leadership team. " + "I can drill down into my team hierarchy, verifying that Tanya Tidie in Clinical Records and our data stewards have completed " + "their privacy review to-dos for the newest clinical hospital feeds." + ), + terminal_cue="Launch uv run my_profile --user faithbroker and inspect Roles & My Team screen", + ), + SceneSpec( + number=3, + title="Subject Access Request (SAR) & PII Lineage", + speaker="Faith Broker", + voice="en-GB-LibbyNeural", + dialogue=( + "When an employee or clinical trial participant submits a Subject Access Request, we no longer need to email dozens of database administrators. " + "Using the unified catalog search and My Network graph, I can trace the entire information supply chain for patient identifiers. " + "From raw ingestion tables in PostgreSQL to S3 data lakes and analytics marts, Egeria maps the full lineage and confidential relationships in seconds." + ), + terminal_cue="Navigate to Shop for Data and My Network screen, filtering for Restricted PII terms", + ), + SceneSpec( + number=4, + title="Real-Time Privacy Stewardship & Feedback", + speaker="Faith Broker", + voice="en-GB-LibbyNeural", + dialogue=( + "If a new dataset lacks appropriate retention rules, I can immediately tag a to-do, add an audit comment with Control-N, " + "or submit real-time feedback with Control-F directly to the engineering team. " + "Our data stewards earn Karma Points for completing privacy validations, making compliance a collaborative, rewarding process." + ), + terminal_cue="Open To-Dos table, strike Ctrl+N to add privacy comment, open Ctrl+F feedback modal", + ), + SceneSpec( + number=5, + title="Business Impact & Executive Takeaways", + speaker="Faith Broker", + voice="en-GB-LibbyNeural", + dialogue=( + "By anchoring privacy governance in Egeria's open metadata ecosystem, Coco Pharmaceuticals reduced Subject Access Request response times " + "from three weeks down to under two hours, while eliminating the risk of regulatory fines. " + "Data privacy is no longer a barrier to research—it's the trusted foundation that enables our scientists to innovate with confidence. " + "Thank you for joining me!" + ), + slide_title="BUSINESS IMPACT & EXECUTIVE ROI", + slide_subtitle="Measurable Privacy Outcomes • Coco Pharmaceuticals Case Study", + callouts=[ + "95% Reduction in SAR Response Time: From 3 weeks to under 2 hours", + "Zero-Defect Audit Readiness: Continuous automated metadata classification", + "Fail-Closed Protection: Operational secrets and PII strictly isolated", + "Explore Egeria: egeria-project.org • Egeria Security Blog Series Part 7", + ], + ), + ], + ), + + # -------------------------------------------------------------------------- + # Episode 2: Peter Profile - Zero-Touch Data Stewardship + # -------------------------------------------------------------------------- + "episode2": EpisodeSpec( + id="episode2", + episode_number=2, + title="Zero-Touch Data Stewardship & Curation at Scale", + subtitle="Automating Cataloging, Glossary Mapping & Stewardship Karma at Coco Pharmaceuticals", + lead_persona="Peter Profile", + persona_username="peterprofile", + persona_role="Information Analyst & Lead Data Steward", + persona_team="Clinical Trials & Enterprise Data Stewardship", + business_driver="Data Quality, Semantic Consistency & Curation Productivity", + target_audience="Data Stewards, Information Analysts, Data Governance Teams, Analytics Engineers", + voice_name="en-GB-RyanNeural", + voice_locale="en-GB", + voice_gender="Male", + voice_accent="British English (Engaging & Methodical)", + voice_description="Articulate, energetic British analyst tone showcasing hands-on productivity and stewardship precision.", + offline_say_voice="Daniel", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Stewardship Dilemma", + speaker="Peter Profile", + voice="en-GB-RyanNeural", + dialogue=( + "Hi everyone! I'm Peter Profile, Lead Information Analyst and Data Steward at Coco Pharmaceuticals. " + "As data stewards, we've all faced the impossible backlog: thousands of database columns arriving every week, " + "cryptic abbreviations, missing descriptions, and manual spreadsheets that go out of date the minute you save them. " + "Today, I'll demonstrate how Egeria transforms data curation into an automated, streamlined experience right from your terminal." + ), + slide_title="EPISODE 2: ZERO-TOUCH DATA STEWARDSHIP", + slide_subtitle="Curating at Machine Speed • Peter Profile (Information Analyst)", + callouts=[ + "Problem: Massive schema volumes and manual glossary mapping backlogs", + "Friction: Cryptic physical column names disconnected from business definitions", + "Egeria Solution: Automated housekeeping connectors & gamified curation", + "Lead Persona: Peter Profile (Information Analyst & Lead Data Steward)", + ], + ), + SceneSpec( + number=1, + title="Semantic Glossary Alignment & Housekeeping Connectors", + speaker="Peter Profile", + voice="en-GB-RyanNeural", + dialogue=( + "At Coco Pharma, we connect technical schemas directly to our enterprise business glossaries using ISA and HASA relationships. " + "What makes Egeria 6.3 revolutionary are its automated housekeeping connectors running behind the scenes. " + "Pasteur archives completed to-dos while permanently preserving our audit lineage, Mendel deduplicates redundant keyword tags, " + "and native reversibility gives us Undo and Restore for any accidental metadata modification." + ), + slide_title="SEMANTIC KNOWLEDGE GRAPH & HOUSEKEEPING", + slide_subtitle="Automated Connectors: Simon, Pasteur & Mendel", + callouts=[ + "Automated Housekeeping: Pasteur archives finished tasks, Mendel deduplicates tags", + "Standardized Relationships: ISA (specialization) and HASA (composition)", + "Native Reversibility: Full Undo and cascading Restore on metadata changes", + "Cross-System Consistency: One term definition across Snowflake, Kafka, and PostgreSQL", + ], + ), + SceneSpec( + number=2, + title="The Steward Cockpit - Managing Curation To-Dos", + speaker="Peter Profile", + voice="en-GB-RyanNeural", + dialogue=( + "When I launch the My Profile cockpit, my active curation assignments are already loaded in parallel. " + "In the To-Dos panel, I see pending validation requests from clinical researchers like Dr. Tessa Tube and Callie Quartile. " + "With Control-T, I can edit my to-do table directly in the terminal, updating progress notes and verifying schema certifications." + ), + terminal_cue="Launch uv run my_profile --user peterprofile, navigate to To-Dos, hit Ctrl+T to edit", + ), + SceneSpec( + number=3, + title="Real-Time Annotation & Discussion Threads", + speaker="Peter Profile", + voice="en-GB-RyanNeural", + dialogue=( + "Collaboration is built directly into the metadata stream. " + "Selecting a newly registered oncology assay dataset, I can press Control-O to view comments left by our lab technicians. " + "With Control-N, I add an authoritative steward note confirming the confidence score of 0.98 validated against laboratory assay standards. " + "Everyone in the enterprise sees this annotation immediately." + ), + terminal_cue="Press Ctrl+O to inspect comments, strike Ctrl+N to add verification comment, Ctrl+K to bookmark", + ), + SceneSpec( + number=4, + title="Earning Karma Points & Community Leadership", + speaker="Peter Profile", + voice="en-GB-RyanNeural", + dialogue=( + "Every time I complete a glossary review, resolve a data quality defect, or assist a colleague, Egeria awards Karma Points. " + "Our Simon connector calculates these scores periodically in the background, keeping our event bus completely free of noise. " + "Governance isn't a chore anymore—it's a transparent, recognized contribution to Coco Pharma's life-saving cancer research." + ), + terminal_cue="Navigate to Karma panel and Communities screen, review stewardship contributions", + ), + SceneSpec( + number=5, + title="Business Impact & Value Realization", + speaker="Peter Profile", + voice="en-GB-RyanNeural", + dialogue=( + "With Egeria's automated stewardship toolset, we've increased our cataloged asset coverage to over 98% " + "while cutting curation cycle times by seventy percent. " + "High-quality, curated metadata is the fuel that powers Coco Pharmaceuticals' AI and research breakthroughs. " + "Thanks for watching, and happy cataloging!" + ), + slide_title="STEWARDSHIP IMPACT & PRODUCTIVITY GAINS", + slide_subtitle="Quantifiable Metadata Quality Metrics • Coco Pharmaceuticals", + callouts=[ + "98%+ Metadata Coverage: Automated ingestion across all primary data stores", + "70% Faster Curation Cycles: Interactive terminal editing and semantic linking", + "Zero Maintenance Debt: Pasteur and Mendel eliminate backlog clutter automatically", + "Learn More: egeria-project.org • Egeria Release 6.3 Blog Series Part 6", + ], + ), + ], + ), + + # -------------------------------------------------------------------------- + # Episode 3: Dr. Tessa Tube - Clinical Trial Data Provenance + # -------------------------------------------------------------------------- + "episode3": EpisodeSpec( + id="episode3", + episode_number=3, + title="Trusted Clinical Data Provenance & Trial Integrity", + subtitle="End-to-End Scientific Audit Trails & Governed Research Subscriptions", + lead_persona="Dr. Tessa Tube", + persona_username="tessatube", + persona_role="Lead Clinical Researcher", + persona_team="Oncology Research & Clinical Trials", + business_driver="Scientific Integrity, Regulatory Filings (FDA/EMA) & Audit Provenance", + target_audience="Clinical Researchers, R&D Directors, Biostatisticians, Regulatory Affairs", + voice_name="en-US-AriaNeural", + voice_locale="en-US", + voice_gender="Female", + voice_accent="American English (Academic & Authoritative)", + voice_description="Crisp, mature authoritative academic tone delivering scientific credibility and audit precision.", + offline_say_voice="Moira", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Scientific Imperative", + speaker="Dr. Tessa Tube", + voice="en-US-AriaNeural", + dialogue=( + "Welcome! I'm Dr. Tessa Tube, Lead Clinical Researcher directing oncology trials at Coco Pharmaceuticals. " + "When developing novel cancer therapies, scientific integrity is paramount. " + "Regulatory agencies like the FDA and EMA require complete, uncompromised provenance proving that our clinical conclusions " + "originate from certified, unmodified laboratory data. " + "Let's explore how Egeria provides the unbroken chain of custody our scientific research demands." + ), + slide_title="EPISODE 3: CLINICAL DATA PROVENANCE & INTEGRITY", + slide_subtitle="Unbroken Lineage for Cancer Research • Dr. Tessa Tube (Lead Researcher)", + callouts=[ + "Requirement: Bulletproof audit trails for FDA and EMA regulatory filings", + "Risk: Data drift, untracked transformations, and unverified assay data", + "Egeria Solution: Automated end-to-end provenance graphs and certified data products", + "Standards: Bitol Open Data Contract Standard (ODCS) & OpenLineage", + ], + ), + SceneSpec( + number=1, + title="The Research Lifecycle & Governance Guarantees", + speaker="Dr. Tessa Tube", + voice="en-US-AriaNeural", + dialogue=( + "In our oncology trials, raw patient data flows from hospital ingress feeds into certified analytical data products. " + "Under the governance framework established by Jules Keeper, Egeria extracts five distinct lineage layers: " + "from business flows down to column mappings. " + "Our Darwin governance engine reconciles runtime OpenLineage pipeline events against declared Bitol data contracts, " + "flagging any discrepancy between architectural intent and actual data movement." + ), + slide_title="SCIENTIFIC PROVENANCE & DATA GOVERNANCE", + slide_subtitle="Tracking Data Supply Chains from Clinical Assay to Statistical Model", + callouts=[ + "Five Lineage Layers: Business Flow, Data Mesh, System Fabric, Data Fabric, Data Mapping", + "Darwin Engine: Reconciles pipeline execution against Bitol ODCS contracts", + "Quality Scoring: Laboratory validation metrics and confidence scores (>=0.95)", + "High-Speed OpenLineage: Analyzing 800k+ events in seconds via PostgreSQL backend", + ], + ), + SceneSpec( + number=2, + title="Shopping for Certified Clinical Data Products", + speaker="Dr. Tessa Tube", + voice="en-US-AriaNeural", + dialogue=( + "In the My Profile workspace, navigating to 'Shop for Data' gives me access to our certified Digital Data Products catalog. " + "External hospital trial datasets are imported seamlessly using W3C DCAT version 3 standards. " + "I can search for oncology cohort assays, filter by quality score, and inspect the underlying schema. " + "With Control-L, I can instantly sample the dataset in a secure sandbox, verifying row counts and distributions " + "before ever committing to a full research pipeline." + ), + terminal_cue="Navigate to Shop for Data, search for 'Oncology Cohort', press Ctrl+L for data sampling modal", + ), + SceneSpec( + number=3, + title="Governed Subscription Requests & Usage Terms", + speaker="Dr. Tessa Tube", + voice="en-US-AriaNeural", + dialogue=( + "Once I verify the sample, pressing Control-U opens the Subscription Request modal. " + "I specify my clinical trial protocol number, research purpose, and duration under our Bitol ODCS agreement. " + "Egeria automatically routes the request through Faith Broker for privacy clearance and Peter Profile for data stewardship, " + "while the Baudot connector provisions differential updates without notification spam." + ), + terminal_cue="Press Ctrl+U to open Create Subscription Request modal, fill trial protocol, confirm request", + ), + SceneSpec( + number=4, + title="Cross-Functional Research Collaboration", + speaker="Dr. Tessa Tube", + voice="en-US-AriaNeural", + dialogue=( + "Using the My Network visualization, I can see my collaborative links with data scientist Callie Quartile " + "and integration architect Bob Nitter. " + "When Callie updates her patient survival prediction model, our research team receives automated notifications, " + "ensuring everyone works on the same certified version of clinical truth." + ), + terminal_cue="Open My Network screen, examine research relationships and collaborative links", + ), + SceneSpec( + number=5, + title="Scientific Impact & Accelerated Discovery", + speaker="Dr. Tessa Tube", + voice="en-US-AriaNeural", + dialogue=( + "Egeria has revolutionized our clinical trial lifecycle. " + "We have eliminated months of audit preparation for regulatory filings and reduced research onboarding times by eighty percent. " + "Most importantly, trusted open metadata accelerates our ability to deliver life-saving therapies to patients faster. " + "Thank you for exploring clinical provenance with me today!" + ), + slide_title="RESEARCH IMPACT & REGULATORY READINESS", + slide_subtitle="Accelerating Clinical Outcomes • Coco Pharmaceuticals", + callouts=[ + "80% Reduction in Audit Preparation: Instant export of complete provenance graphs", + "100% Traceability: Every regulatory claim backed by immutable metadata lineage", + "Bitol ODCS Compliance: Automated contract enforcement across all trial pipelines", + "Explore Egeria: egeria-project.org • Egeria Data Mesh Blog Series Part 8 & 9", + ], + ), + ], + ), + + # -------------------------------------------------------------------------- + # Episode 4: Stew Faster - Supply Chain & ESG Sustainability + # -------------------------------------------------------------------------- + "episode4": EpisodeSpec( + id="episode4", + episode_number=4, + title="Supply Chain Traceability & ESG Sustainability", + subtitle="Unifying IoT Telemetry, Carbon Accounting & Factory Operations with Open Metadata", + lead_persona="Stew Faster", + persona_username="stewfaster", + persona_role="Manufacturing General Manager", + persona_team="Plant Operations & Sustainability Leadership", + business_driver="ESG Compliance, Operational Efficiency, Carbon Accounting & IoT Telemetry", + target_audience="COOs, Supply Chain Executives, Sustainability Officers, IoT Engineers", + voice_name="en-AU-WilliamMultilingualNeural", + voice_locale="en-AU", + voice_gender="Male", + voice_accent="Australian English (Pragmatic & Energetic)", + voice_description="Direct, pragmatic Australian operational leader tone delivering actionable insights on plant floor efficiency.", + offline_say_voice="Daniel", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Manufacturing Reality", + speaker="Stew Faster", + voice="en-AU-WilliamMultilingualNeural", + dialogue=( + "G'day! I'm Stew Faster, Manufacturing General Manager at Coco Pharmaceuticals. " + "Running high-precision pharmaceutical plants means juggling thousands of IoT sensors, chemical reactors, " + "and cold-chain logistics, while meeting strict corporate ESG and carbon reduction mandates. " + "If your operational tech and corporate data lakes don't talk to each other, you're flying blind. " + "Today, I'll show you how Egeria connects our factory floor directly into our enterprise metadata catalog." + ), + slide_title="EPISODE 4: SUPPLY CHAIN & ESG SUSTAINABILITY", + slide_subtitle="From Factory Floor to Carbon Ledger • Stew Faster (Manufacturing GM)", + callouts=[ + "Challenge: Fragmented operational technology (OT) and corporate IT silos", + "ESG Mandate: Real-time carbon tracking, solvent recovery & cold-chain compliance", + "Egeria Solution: Unified metadata catalog for IoT telemetry and physical assets", + "Open Standards: W3C DCAT v3 JSON-LD Document Exchange", + ], + ), + SceneSpec( + number=1, + title="Connecting Physical Assets to Digital Metadata & DCAT v3", + speaker="Stew Faster", + voice="en-AU-WilliamMultilingualNeural", + dialogue=( + "In our advanced manufacturing plants, every chemical batch reactor, autoclave, and temperature logger " + "is cataloged as an Egeria asset under our Technology Types taxonomy. " + "With Egeria 6.3's W3C DCAT version 3 connectors—including DCAT Files Receiver and DCAT Publisher—we exchange " + "operational datasets with partner suppliers and public registries seamlessly in standard JSON-LD, " + "linking physical equipment to real-time carbon and solvent recovery metrics." + ), + slide_title="OPERATIONAL TECHNOLOGY & DCAT V3 CATALOGING", + slide_subtitle="W3C DCAT v3 Connectors: Files, Events, Cataloguers & Publishers", + callouts=[ + "W3C DCAT v3 Standards: Interoperable JSON-LD catalog exchange across supply chains", + "Technology Types: Standardized catalog for IoT sensors, PLC controllers, and reactors", + "Real-Time Telemetry: MQTT and Kafka topic metadata indexed alongside batch records", + "ESG Carbon Tracking: Kilowatt-hour and solvent consumption bound to batch IDs", + ], + ), + SceneSpec( + number=2, + title="The Plant Manager Dashboard & Roles", + speaker="Stew Faster", + voice="en-AU-WilliamMultilingualNeural", + dialogue=( + "Launching My Profile, my operational workspace loads our Sustainability & Manufacturing council roles. " + "I can see my team links with CFO Reggie Mint and Chief Data Officer Jules Keeper. " + "We monitor shared KPIs for energy efficiency and batch yield right from our team activity stream." + ), + terminal_cue="Launch uv run my_profile --user stewfaster, inspect Sustainability roles and team panel", + ), + SceneSpec( + number=3, + title="Exploring Technology Types & Factory Catalogs", + speaker="Stew Faster", + voice="en-AU-WilliamMultilingualNeural", + dialogue=( + "Navigating to the Technology Types screen, I can inspect the hardware and software connectors powering our plants. " + "From SCADA database gateways to Apache NiFi pipelines, Egeria details the operational status and deployment configurations. " + "When an autoclave temperature sensor flags an anomaly, we can trace the exact batch and affected ampoules in seconds." + ), + terminal_cue="Navigate to Technology Types screen, inspect SCADA and IoT sensor connectors", + ), + SceneSpec( + number=4, + title="Collaborative Operational Feedback & Actions", + speaker="Stew Faster", + voice="en-AU-WilliamMultilingualNeural", + dialogue=( + "When our sustainability audit identifies an opportunity for solvent recycling, " + "I can open the Feedback dialog with Control-F or log an immediate to-do for our engineering team. " + "Using Dr. Egeria markdown reports, our carbon footprint metrics are automatically generated and published to executive leadership." + ), + terminal_cue="Strike Ctrl+F for operational feedback, review automated Dr. Egeria sustainability report", + ), + SceneSpec( + number=5, + title="Operational Outcomes & Sustainability ROI", + speaker="Stew Faster", + voice="en-AU-WilliamMultilingualNeural", + dialogue=( + "By bridging the gap between factory floor operations and open metadata governance, " + "Coco Pharmaceuticals reduced manufacturing energy waste by eighteen percent and achieved automated, real-time ESG compliance. " + "Open metadata isn't just for software developers—it's the backbone of modern, sustainable manufacturing. " + "Cheers, and thanks for watching!" + ), + slide_title="MANUFACTURING ROI & SUSTAINABILITY IMPACT", + slide_subtitle="Tangible Operational Gains • Coco Pharmaceuticals", + callouts=[ + "18% Reduction in Energy Waste: Real-time sensor telemetry optimization", + "Inter-Company Interoperability: Seamless DCAT v3 partner catalog exchange", + "Automated ESG Reporting: Dr. Egeria markdown reports generated in minutes", + "Learn More: egeria-project.org • Egeria DCAT Exchange Blog Series Part 10", + ], + ), + ], + ), + + # -------------------------------------------------------------------------- + # Episode 5: Erin Overview - Business Glossaries & Architecture + # -------------------------------------------------------------------------- + "episode5": EpisodeSpec( + id="episode5", + episode_number=5, + title="Connecting Business Glossaries to Technical Architecture", + subtitle="Eliminating Semantic Ambiguity with Enterprise Knowledge Graphs & Open Metadata", + lead_persona="Erin Overview", + persona_username="erinoverview", + persona_role="Enterprise Information Architect", + persona_team="Information Architecture & Enterprise Strategy", + business_driver="Semantic Interoperability, Architecture Governance & Vendor Neutrality", + target_audience="Enterprise Architects, Chief Architects, Data Modelers, Integration Engineers", + voice_name="en-US-JennyNeural", + voice_locale="en-US", + voice_gender="Female", + voice_accent="American English (Clear, Strategic & Visionary)", + voice_description="Articulate, visionary American architectural voice connecting enterprise strategy to technical implementation.", + offline_say_voice="Samantha", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Tower of Babel", + speaker="Erin Overview", + voice="en-US-JennyNeural", + dialogue=( + "Hello! I'm Erin Overview, Enterprise Information Architect at Coco Pharmaceuticals. " + "In any large organization, the biggest hurdle to digital transformation is the language barrier. " + "Business executives say 'Customer Account', marketing says 'Subscriber Profile', and the database column is named 'CUST_MSTR_01'. " + "When terms diverge, data integration fails and AI models produce hallucinations. " + "Today, I'll show you how Egeria builds an enterprise knowledge graph that unifies business language and technical architecture." + ), + slide_title="EPISODE 5: BUSINESS GLOSSARIES & ARCHITECTURE", + slide_subtitle="Unified Semantic Architecture • Erin Overview (Information Architect)", + callouts=[ + "Problem: Semantic ambiguity and disconnected naming across business and IT", + "Cost: Broken ETL pipelines, slow data integration, and untrusted analytics", + "Egeria Solution: Open metadata knowledge graph connecting glossaries to schemas", + "High-Speed Graph: Five supply-chain lineage layers & PostgreSQL OpenLineage", + ], + ), + SceneSpec( + number=1, + title="The Enterprise Taxonomy & Semantic Knowledge Graph", + speaker="Erin Overview", + voice="en-US-JennyNeural", + dialogue=( + "At Coco Pharma, we structure our business concepts into Subject Areas, Glossaries, and Controlled Vocabularies. " + "Egeria's semantic relationship model allows us to define relationships like 'Is A', 'Has A', and 'Synonym' across terms. " + "In Egeria 6.3, our OpenLineage engine processes 30 days of pipeline history—over 800,000 events—in just 3 seconds, " + "extracting five clear views from business flows down to physical column transformations with bounded graph queries." + ), + slide_title="ENTERPRISE TAXONOMY & RELATIONSHIP ARCHITECTURE", + slide_subtitle="Federated Open Metadata Across Heterogeneous Technology Platforms", + callouts=[ + "Five Lineage Layers: Business Flow, Data Mesh, System Fabric, Data Fabric, Data Mapping", + "High Performance: 800k+ OpenLineage events analyzed in ~3 seconds in PostgreSQL", + "Bounded Exploration: 8-hop navigation limits prevent browser and terminal freezes", + "Vendor Independence: Open standards eliminate proprietary catalog lock-in", + ], + ), + SceneSpec( + number=2, + title="Architecture Cockpit & Communities Navigation", + speaker="Erin Overview", + voice="en-US-JennyNeural", + dialogue=( + "In my My Profile workspace, I monitor cross-functional architecture initiatives. " + "Under the Communities tab, our Architecture Review Board collaborates with API developers like Bob Nitter " + "and business leaders like Harry Hopeful. " + "We can review newly proposed data product models and verify semantic compliance before code is ever deployed." + ), + terminal_cue="Launch uv run my_profile --user erinoverview, open Communities screen and view Architecture Board", + ), + SceneSpec( + number=3, + title="Browsing the Unified Glossary & Term Dictionaries", + speaker="Erin Overview", + voice="en-US-JennyNeural", + dialogue=( + "Let's switch into the Shop for Data and Glossary browser. " + "Here, any developer or business analyst can search for business terms like 'Bioequivalence Threshold' " + "and immediately see its formal mathematical definition, governance classification, and linked physical datasets. " + "With Control-K, I bookmark active architecture standards for rapid recall." + ), + terminal_cue="Open Shop for Data, search for 'Bioequivalence', inspect term details, press Ctrl+K to bookmark", + ), + SceneSpec( + number=4, + title="Graph Visualization of Architectural Dependencies", + speaker="Erin Overview", + voice="en-US-JennyNeural", + dialogue=( + "The My Network screen visualizes the interconnected topology of our information supply chain. " + "We can see how changes to an upstream hospital schema cascade downstream to clinical research models and executive reports. " + "This level of impact analysis makes enterprise architecture proactive rather than reactive." + ), + terminal_cue="Open My Network screen, navigate relationship graph and architectural connections", + ), + SceneSpec( + number=5, + title="Architectural ROI & Strategic Agility", + speaker="Erin Overview", + voice="en-US-JennyNeural", + dialogue=( + "By adopting Egeria's open metadata architecture, Coco Pharmaceuticals reduced new data integration project timelines by sixty percent " + "and eliminated costly misunderstandings between business teams and engineering. " + "A unified semantic foundation is what makes agile, reliable AI possible. " + "Thank you for exploring enterprise information architecture with me today!" + ), + slide_title="STRATEGIC IMPACT & ARCHITECTURE ROI", + slide_subtitle="Enabling Enterprise Agility • Coco Pharmaceuticals", + callouts=[ + "60% Faster Data Integration: Immediate reuse of pre-mapped semantic models", + "Zero Hallucination AI: Grounding LLMs in authoritative open metadata definitions", + "Future-Proof Architecture: Open standards ensure complete platform portability", + "Explore More: egeria-project.org • Egeria Lineage Layers Blog Series Part 9", + ], + ), + ], + ), + + # -------------------------------------------------------------------------- + # Episode 6: Sidney Seeker - Forensic Auditing & Fraud Investigation + # -------------------------------------------------------------------------- + "episode6": EpisodeSpec( + id="episode6", + episode_number=6, + title="Forensic Data Auditing & Fraud Investigation", + subtitle="Uncovering Shadow Pipelines, Unauthorized Changes & Compliance Anomalies with Open Metadata", + lead_persona="Sidney Seeker", + persona_username="sidneyseeker", + persona_role="Lead Fraud Investigator & Security Auditor", + persona_team="Internal Audit & Security Investigation", + business_driver="Fraud Detection, Security Compliance, Shadow IT Discovery & Audit Defense", + target_audience="Internal Auditors, CISOs, Fraud Investigators, Security Analysts, Risk Officers", + voice_name="en-IE-ConnorNeural", + voice_locale="en-IE", + voice_gender="Male", + voice_accent="Irish English (Sharp, Inquisitive & Methodical)", + voice_description="Sharp, inquisitive Irish forensic auditor tone highlighting vigilance, evidence integrity, and anomaly detection.", + offline_say_voice="Daniel", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Hidden Anomaly", + speaker="Sidney Seeker", + voice="en-IE-ConnorNeural", + dialogue=( + "Hello there! I'm Sidney Seeker, Lead Fraud Investigator and Security Auditor at Coco Pharmaceuticals. " + "In modern enterprises, fraud and security breaches rarely announce themselves. " + "They hide in unapproved database schema copies, unauthorized pipeline modifications, and subtle changes to financial payment tables. " + "When investigating anomalies, you need an immutable, timestamped record of every metadata change. " + "Today, I'll show you how Egeria provides the forensic visibility needed to catch discrepancies before they become disasters." + ), + slide_title="EPISODE 6: FORENSIC DATA AUDITING & INVESTIGATION", + slide_subtitle="Time-Travel Lineage & Anomaly Detection • Sidney Seeker (Auditor)", + callouts=[ + "Threat: Shadow data pipelines, unapproved replication copies, and invoice tampering", + "Audit Challenge: Reconstructing who altered an asset definition and when", + "Egeria Solution: STRIDE threat modeling, reversible metadata & audit lineage", + "Lead Persona: Sidney Seeker (Lead Fraud Investigator & Auditor)", + ], + ), + SceneSpec( + number=1, + title="The Audit Trail - Metadata Versioning & Classifications", + speaker="Sidney Seeker", + voice="en-IE-ConnorNeural", + dialogue=( + "Under Coco Pharma's zero-trust security architecture, Egeria records every metadata event. " + "Egeria 6.3 incorporates standardized STRIDE threat models and fail-closed security zones. " + "If an unauthorized change occurs, native reversibility allows us to execute Undo or cascading Restore on affected schemas. " + "Working alongside CISO Simon Burr and Payments Clerk Sally Counter, we continuously audit sensitive transactional streams." + ), + slide_title="IMMUTABLE AUDITING & THREAT MODELING", + slide_subtitle="STRIDE Threats, Reversible Metadata & Fail-Closed Zones", + callouts=[ + "Built-in STRIDE Threat Models: Spoofing, Tampering, Repudiation, Info Disclosure, DoS, Elevation", + "Native Reversible Operations: Full Undo and cascading Restore for metadata elements", + "Fail-Closed Security Zones: Operational credentials and configs restricted by default", + "Cross-System Verification: Matching ERP ledger entries against banking egress feeds", + ], + ), + SceneSpec( + number=2, + title="Investigator Workspace & Security Team Drill-Down", + speaker="Sidney Seeker", + voice="en-IE-ConnorNeural", + dialogue=( + "Launching the My Profile workspace as Sidney Seeker, my active security roles and investigation to-dos are right at my fingertips. " + "I can drill down into our Security Investigation Team, checking notes from DevOps specialist Lemmie Stage " + "and reviewing open audit alerts across our payment infrastructure." + ), + terminal_cue="Launch uv run my_profile --user sidneyseeker, review Audit roles and Security Team drill-down", + ), + SceneSpec( + number=3, + title="Tracing Payment Anomalies with Lineage & Comments", + speaker="Sidney Seeker", + voice="en-IE-ConnorNeural", + dialogue=( + "When an anomalous invoice payment is flagged, I navigate to the Accounts Payable data product in Shop for Data. " + "Pressing Control-O, I view discussion threads between Sally Counter and our auditors. " + "With Control-N, I attach a forensic audit tag to the dataset, flagging it for immediate review and locking down unverified exports." + ), + terminal_cue="Open Shop for Data, filter Accounts Payable dataset, hit Ctrl+O for comments, Ctrl+N to log audit tag", + ), + SceneSpec( + number=4, + title="Uncovering Shadow IT in the Network Graph", + speaker="Sidney Seeker", + voice="en-IE-ConnorNeural", + dialogue=( + "The My Network graph allows us to spot unapproved data replication instantly. " + "If an uncataloged server is pulling sensitive patient or payment records from our core databases, " + "Egeria's relationship visualizer highlights the unauthorized relationship, allowing us to isolate the system immediately." + ), + terminal_cue="Open My Network screen, examine anomalous relationship clusters and unauthorized links", + ), + SceneSpec( + number=5, + title="Investigation Outcomes & Risk Protection", + speaker="Sidney Seeker", + voice="en-IE-ConnorNeural", + dialogue=( + "With Egeria's comprehensive metadata forensics, Coco Pharmaceuticals has eliminated shadow data copies " + "and reduced internal investigation cycle times by eighty-five percent. " + "Proactive metadata governance is the ultimate shield protecting enterprise reputation and financial integrity. " + "Stay vigilant, and thank you for watching!" + ), + slide_title="AUDIT IMPACT & SECURITY ROI", + slide_subtitle="Protecting Enterprise Integrity • Coco Pharmaceuticals", + callouts=[ + "85% Faster Investigation Cycles: Instant reconstruction of data history and access logs", + "Zero Unmonitored Shadow IT: Continuous discovery of rogue data copies", + "Reversible Recovery: Instant Restore on deleted or compromised catalog assets", + "Explore Egeria: egeria-project.org • Egeria Security Blog Series Part 7", + ], + ), + ], + ), + + # -------------------------------------------------------------------------- + # Episode 7: Jules Keeper & Reggie Mint - The Executive Cockpit + # -------------------------------------------------------------------------- + "episode7": EpisodeSpec( + id="episode7", + episode_number=7, + title="The Executive Cockpit: Measuring Data ROI & Strategy", + subtitle="Transforming Data Governance from a Cost Center into a Value Creation Engine", + lead_persona="Jules Keeper & Reggie Mint", + persona_username="juleskeeper", + persona_role="Chief Data Officer & Chief Financial Officer", + persona_team="Executive Board & Strategic Leadership", + business_driver="Data Valuation, Executive ROI, Strategic Governance & Board Reporting", + target_audience="CDOs, CFOs, CIOs, CEOs, Board Directors, Enterprise Leaders", + voice_name="en-GB-ThomasNeural", + voice_locale="en-GB", + voice_gender="Male", + voice_accent="British English (Executive, Warm & Authoritative)", + voice_description="Distinguished British executive tone paired with American CFO dialogue illustrating strategic leadership.", + offline_say_voice="Daniel", + scenes=[ + SceneSpec( + number=0, + title="Cold Open - The Boardroom Challenge", + speaker="Jules Keeper", + voice="en-GB-ThomasNeural", + dialogue=( + "Welcome back to Coco Pharmaceuticals. I'm Jules Keeper, Chief Data Officer, joined by our Chief Financial Officer, Reggie Mint. " + "Every executive team asks the fundamental question: Is our investment in data governance actually moving the needle? " + "Historically, data governance was viewed as a bureaucratic cost center that slowed business velocity. " + "Today, Reggie and I will show you how Egeria transforms open metadata into a measurable engine of enterprise value, innovation, and return on investment." + ), + slide_title="EPISODE 7: THE EXECUTIVE COCKPIT", + slide_subtitle="Measuring Data ROI & Strategic Value • Jules Keeper (CDO) & Reggie Mint (CFO)", + callouts=[ + "Executive Challenge: Quantifying the business ROI of data governance initiatives", + "Legacy Trap: Bureaucratic policy documents disconnected from daily operations", + "Egeria Solution: Actionable metadata cockpits, product subscriptions & measurable KPI tracking", + "Standards & Automation: Bitol ODCS contracts, autonomous housekeeping & fail-closed security", + ], + ), + SceneSpec( + number=1, + title="Executive Alignment - Governance Linked to Business Imperatives", + speaker="Jules Keeper", + voice="en-GB-ThomasNeural", + dialogue=( + "When we launched our 'Better Data for Everyone' initiative, we anchored every governance policy to concrete business imperatives: " + "accelerating clinical cancer trials with Dr. Tessa Tube, ensuring zero-breach privacy with Faith Broker, " + "and optimizing manufacturing energy efficiency with Stew Faster. " + "Attaching governance responsibilities to appointed operational roles turned abstract policy into measurable daily accountability." + ), + slide_title="STRATEGIC GOVERNANCE IMPERATIVES", + slide_subtitle="Aligning Open Metadata Directly with Strategic Business Goals", + callouts=[ + "Clinical Innovation: Accelerating trial filings from years to months", + "Regulatory Protection: Zero-defect compliance across GDPR, HIPAA, and financial audits", + "Operational Sustainability: 18% energy waste reduction across manufacturing plants", + "Role-Based Execution: Policies bound to appointed team members across the enterprise", + ], + ), + SceneSpec( + number=2, + title="Executive Cockpit Walkthrough & Portfolio Health", + speaker="Jules Keeper", + voice="en-GB-ThomasNeural", + dialogue=( + "Logging into the executive cockpit, Reggie and I have real-time visibility across Coco Pharma's entire data estate. " + "In our executive associations, we oversee active Digital Data Product subscriptions adhering to Bitol standards, " + "monitor autonomous housekeeping efficiency, and review strategic cross-functional initiatives across all business units." + ), + terminal_cue="Launch uv run my_profile --user juleskeeper, inspect Executive associations and data product portfolios", + ), + SceneSpec( + number=3, + title="CFO Perspective - Quantifying Data ROI & Risk Reduction", + speaker="Reggie Mint", + voice="en-US-GuyNeural", + dialogue=( + "From a financial standpoint, Egeria delivers measurable bottom-line value. " + "We've reduced data discovery overhead by seventy-five percent across our three hundred analysts, saving thousands of productive hours each month. " + "By eliminating shadow data infrastructure and retiring duplicate legacy tools, we cut enterprise licensing costs by over two million dollars annually." + ), + terminal_cue="Navigate to Shop for Data and Active Subscriptions, demonstrating enterprise usage metrics", + ), + SceneSpec( + number=4, + title="Community Engagement & Collaborative Governance", + speaker="Jules Keeper", + voice="en-GB-ThomasNeural", + dialogue=( + "Governance succeeds when it's embraced by people, not mandated from an ivory tower. " + "Our Karma Points leaderboard and Community forums empower practitioners like Callie Quartile and Peter Profile " + "to take pride in data excellence. Culture and technology working in unison is what makes Coco Pharmaceuticals an industry leader." + ), + terminal_cue="Open Communities and Karma panel, reviewing enterprise participation leaderboard", + ), + SceneSpec( + number=5, + title="The Future of Governed Enterprise Intelligence", + speaker="Jules Keeper", + voice="en-GB-ThomasNeural", + dialogue=( + "Open metadata is the essential foundation for reliable AI, regulatory trust, and sustainable business growth. " + "We invite you to explore Egeria's open source ecosystem, try out the Coco Pharmaceuticals personas, " + "and bring the power of open metadata to your own organization. " + "Thank you for watching the Coco Pharmaceuticals Business Value Series!" + ), + slide_title="THE EXECUTIVE SUMMARY & CALL TO ACTION", + slide_subtitle="Delivering 'Better Data for Everyone' • Coco Pharmaceuticals", + callouts=[ + "$2M+ Annual Infrastructure Savings: Consolidated catalog and reduced tool redundancy", + "75% Faster Analyst Discovery: Immediate access to governed digital data products", + "Autonomous Housekeeping: Simon, Pasteur, and Baudot maintain clean metadata 24/7", + "Join the Movement: egeria-project.org • github.com/odpi/egeria", + ], + ), + ], + ), +} diff --git a/my_egeria/my_egeria/recording/export_vault.py b/my_egeria/my_egeria/recording/export_vault.py new file mode 100644 index 00000000..2bcdf799 --- /dev/null +++ b/my_egeria/my_egeria/recording/export_vault.py @@ -0,0 +1,391 @@ +#!/usr/bin/env python3 +""" +Egeria Coco Pharmaceuticals Video Vault Exporter. + +Automates the synchronization and export of rendered master videos, audio tracks, +slide graphics, and manifests from the recording output directory into the shared +pdr-vault archival and distribution repository. + +Usage: + uv run python export_vault.py + uv run python export_vault.py --vault-path /Volumes/pdr-vault + uv run python export_vault.py --all-assets + uv run python export_vault.py --dry-run +""" + +import argparse +import hashlib +import json +import os +import re +import shutil +import sys +from datetime import datetime +from pathlib import Path +from typing import List, Dict, Optional, Tuple + +SCRIPT_DIR = Path(__file__).resolve().parent +OUTPUT_DIR = SCRIPT_DIR / "output" + +DEFAULT_VAULT_ENV_VARS = ["PDR_VAULT_PATH", "EGERIA_VAULT_PATH", "EGERIA_RECORDING_VAULT"] +DEFAULT_CANDIDATE_PATHS = [ + Path.home() / "Library/Mobile Documents/com~apple~CloudDocs/PDR 2/pdr-vault/Peter Videos", + Path.home() / "Library/Mobile Documents/com~apple~CloudDocs/PDR 2/pdr-vault", + Path("/Volumes/pdr-vault"), + Path("/Users/Shared/pdr-vault"), + Path.home() / "pdr-vault", + Path("/Volumes/Shared/pdr-vault"), +] + + +def resolve_vault_path(custom_path: Optional[str] = None) -> Path: + """Resolve the destination vault path from args, environment variables, or standard candidate locations.""" + if custom_path: + return Path(custom_path).expanduser().resolve() + + for env_var in DEFAULT_VAULT_ENV_VARS: + val = os.getenv(env_var) + if val: + return Path(val).expanduser().resolve() + + for cand in DEFAULT_CANDIDATE_PATHS: + if cand.exists() and cand.is_dir(): + return cand.resolve() + + # Default fallback: /Users/Shared/pdr-vault + return Path("/Users/Shared/pdr-vault").resolve() + + +def file_checksum(path: Path) -> str: + """Compute sha256 checksum of a file.""" + hasher = hashlib.sha256() + with open(path, "rb") as f: + while chunk := f.read(65536): + hasher.update(chunk) + return hasher.hexdigest() + + +class VaultExporter: + """Manages synchronization of recording artifacts to the pdr-vault repository.""" + + def __init__(self, vault_path: Path, dry_run: bool = False, all_assets: bool = False): + self.vault_path = vault_path + self.dry_run = dry_run + self.all_assets = all_assets + self.vault_videos_dir = self.vault_path / "videos" + self.vault_audio_dir = self.vault_path / "audio" + self.vault_slides_dir = self.vault_path / "slides" + self.vault_episodes_dir = self.vault_path / "episodes" + + def ensure_directories(self): + """Create target vault directories if they do not exist.""" + if not self.dry_run: + self.vault_path.mkdir(parents=True, exist_ok=True) + self.vault_videos_dir.mkdir(parents=True, exist_ok=True) + if self.all_assets: + self.vault_audio_dir.mkdir(parents=True, exist_ok=True) + self.vault_slides_dir.mkdir(parents=True, exist_ok=True) + self.vault_episodes_dir.mkdir(parents=True, exist_ok=True) + + def collect_master_videos(self) -> List[Path]: + """Collect all master narrated MP4 videos across series and walkthrough outputs.""" + videos = [] + if not OUTPUT_DIR.exists(): + return videos + + # 1. Walkthrough master videos + walkthrough_narrated = OUTPUT_DIR / "videos_narrated" + if walkthrough_narrated.exists(): + for mp4 in walkthrough_narrated.glob("*.mp4"): + if mp4.is_file(): + videos.append(mp4) + + # 2. Business Value Series episode master videos + for ep_dir in OUTPUT_DIR.glob("episode*"): + if ep_dir.is_dir(): + ep_narrated = ep_dir / "videos_narrated" + if ep_narrated.exists(): + for mp4 in ep_narrated.glob("*.mp4"): + if mp4.is_file() and not mp4.name.startswith("scene_"): + videos.append(mp4) + + return sorted(list(set(videos)), key=lambda p: p.name) + + def collect_scene_clips(self) -> List[Path]: + """Collect individual scene video clips.""" + clips = [] + for ep_dir in OUTPUT_DIR.glob("episode*"): + if ep_dir.is_dir(): + ep_narrated = ep_dir / "videos_narrated" + if ep_narrated.exists(): + for mp4 in ep_narrated.glob("scene_*_narrated.mp4"): + clips.append(mp4) + return sorted(clips, key=lambda p: p.name) + + def collect_audio_tracks(self) -> List[Path]: + """Collect all MP3 audio tracks.""" + audios = [] + # Walkthrough audio + walkthrough_audio = OUTPUT_DIR / "audio" + if walkthrough_audio.exists(): + for mp3 in walkthrough_audio.glob("*.mp3"): + audios.append(mp3) + + # Series episode audio + for ep_dir in OUTPUT_DIR.glob("episode*"): + if ep_dir.is_dir(): + ep_audio = ep_dir / "audio" + if ep_audio.exists(): + for mp3 in ep_audio.glob("*.mp3"): + audios.append(mp3) + return sorted(audios, key=lambda p: p.name) + + def collect_slides(self) -> List[Path]: + """Collect all 1080p slide PNGs.""" + slides = [] + walkthrough_slides = OUTPUT_DIR / "slides" + if walkthrough_slides.exists(): + for png in walkthrough_slides.glob("*.png"): + slides.append(png) + + for ep_dir in OUTPUT_DIR.glob("episode*"): + if ep_dir.is_dir(): + ep_slides = ep_dir / "slides" + if ep_slides.exists(): + for png in ep_slides.glob("*.png"): + slides.append(png) + return sorted(slides, key=lambda p: p.name) + + def copy_file(self, src: Path, dest: Path) -> Tuple[bool, str]: + """Copy file if destination doesn't exist or content differs. Returns (copied, reason).""" + if not src.exists(): + return False, "source missing" + + if dest.exists(): + if src.stat().st_size == dest.stat().st_size: + # Fast check passed, verify checksum + if file_checksum(src) == file_checksum(dest): + return False, "up to date" + + if not self.dry_run: + dest.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(src, dest) + return True, "copied" + + def sync(self) -> Dict[str, any]: + """Execute full vault synchronization.""" + self.ensure_directories() + master_videos = self.collect_master_videos() + audio_tracks = self.collect_audio_tracks() if self.all_assets else [] + slides = self.collect_slides() if self.all_assets else [] + + print("\n" + "=" * 80) + print("EGERIA VIDEO RECORDING VAULT SYNCHRONIZATION (pdr-vault)") + print("=" * 80) + print(f"Source Directory: {OUTPUT_DIR}") + print(f"Vault Location: {self.vault_path}") + print(f"Mode: {'DRY-RUN' if self.dry_run else 'ACTIVE SYNC'}") + print(f"Asset Scope: {'ALL ASSETS (Videos, Audio, Slides)' if self.all_assets else 'MASTER VIDEOS ONLY'}") + print("-" * 80) + + results = { + "timestamp": datetime.now().isoformat(), + "vault_path": str(self.vault_path), + "master_videos": [], + "audio_tracks": [], + "slides": [], + "copied_count": 0, + "skipped_count": 0, + } + + # Sync master videos + print(f"\n[1/3] Syncing Master Episode & Walkthrough Videos ({len(master_videos)} files)...") + has_episode_folders = any(self.vault_path.glob("Episode *")) or self.vault_path.name == "Peter Videos" + + for video_src in master_videos: + dest = self.vault_videos_dir / video_src.name + copied, status = self.copy_file(video_src, dest) + size_mb = video_src.stat().st_size / (1024 * 1024) + icon = "✓ [COPY]" if copied else "• [SKIP]" + print(f" {icon} {video_src.name} ({size_mb:.1f} MB) -> {status}") + if copied: + results["copied_count"] += 1 + else: + results["skipped_count"] += 1 + results["master_videos"].append({ + "filename": video_src.name, + "size_bytes": video_src.stat().st_size, + "size_mb": round(size_mb, 2), + "status": status, + }) + + # Also sync into specific episode folder if present (e.g. Peter Videos / Episode 1 - ...) + if has_episode_folders: + m = re.search(r"episode_(\d+)", video_src.name) + if m: + ep_num = m.group(1) + matching_folders = list(self.vault_path.glob(f"Episode {ep_num}*")) + for ep_folder in matching_folders: + ep_dest = ep_folder / video_src.name + ep_copied, ep_status = self.copy_file(video_src, ep_dest) + if ep_copied: + print(f" ✓ [VAULT FOLDER SYNC] -> {ep_folder.name}/{video_src.name}") + + if self.all_assets: + local_ep_dir = OUTPUT_DIR / f"episode{ep_num}" + if local_ep_dir.exists(): + for sub in ["audio", "captures", "slides", "videos_narrated"]: + src_sub = local_ep_dir / sub + if src_sub.exists(): + dest_sub = ep_folder / "output" / sub + dest_sub.mkdir(parents=True, exist_ok=True) + for f in src_sub.glob("*"): + if f.is_file(): + self.copy_file(f, dest_sub / f.name) + + # Sync audio tracks if requested + if self.all_assets and audio_tracks: + print(f"\n[2/3] Syncing Audio Narration Tracks ({len(audio_tracks)} files)...") + for audio_src in audio_tracks: + dest = self.vault_audio_dir / audio_src.name + copied, status = self.copy_file(audio_src, dest) + if copied: + results["copied_count"] += 1 + else: + results["skipped_count"] += 1 + results["audio_tracks"].append({ + "filename": audio_src.name, + "size_bytes": audio_src.stat().st_size, + "status": status, + }) + print(f" ✓ Processed {len(audio_tracks)} audio files.") + else: + print("\n[2/3] Audio Narration Tracks: Skipped (pass --all-assets to include)") + + # Sync slide graphics if requested + if self.all_assets and slides: + print(f"\n[3/3] Syncing 1080p Slide Graphics ({len(slides)} files)...") + for slide_src in slides: + dest = self.vault_slides_dir / slide_src.name + copied, status = self.copy_file(slide_src, dest) + if copied: + results["copied_count"] += 1 + else: + results["skipped_count"] += 1 + results["slides"].append({ + "filename": slide_src.name, + "size_bytes": slide_src.stat().st_size, + "status": status, + }) + print(f" ✓ Processed {len(slides)} slide images.") + else: + print("\n[3/3] Slide Graphics: Skipped (pass --all-assets to include)") + + # Generate Index Manifest in Vault + self.generate_manifest(results) + + print("\n" + "=" * 80) + print("VAULT SYNCHRONIZATION COMPLETE!") + print(f" • Files Copied / Updated: {results['copied_count']}") + print(f" • Files Up-to-Date: {results['skipped_count']}") + print(f" • Vault Directory: {self.vault_path}") + print("=" * 80 + "\n") + return results + + def generate_manifest(self, sync_results: Dict[str, any]): + """Generate manifest JSON and Markdown index inside the vault directory.""" + if self.dry_run: + return + + manifest_path = self.vault_path / "vault_manifest.json" + with open(manifest_path, "w", encoding="utf-8") as f: + json.dump(sync_results, f, indent=2) + + # Markdown Index + index_path = self.vault_path / "README.md" + content = [ + "# Coco Pharmaceuticals Video Vault (`pdr-vault`)", + "", + "Central repository and distribution archive for certified master video blog episodes,", + "practitioner walkthroughs, neural voiceover tracks, and slide decks.", + "", + f"**Last Synchronized:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')} ", + f"**Total Master Videos:** {len(sync_results['master_videos'])}", + "", + "---", + "", + "## Master Video Episodes (`videos/`)", + "", + "| Video File | Size | Description | Lead Persona / Topic | Technical Blog Reference |", + "|:---|:---:|:---|:---|:---|", + "| `coco_pharma_my_profile_complete_video_blog.mp4` | Master | Full Episode 0 Video Blog | Callie Quartile (Data Scientist) | [Blog 6](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-6-egeria-6-3-release), [Blog 8](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-8-building-the-data-mesh), [Blog 10](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-10-exchanging-dcat-documents) |", + "| `my_profile_full_walkthrough_narrated.mp4` | Master | Continuous Terminal Walkthrough | Callie Quartile (Data Scientist) | [Blog 6](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-6-egeria-6-3-release) (Autonomous Housekeeping) |", + "| `coco_pharma_episode_1_faith_broker.mp4` | Master | Episode 1: Data Privacy & GDPR | Faith Broker (Privacy Officer) | [Blog 7](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-7-supporting-security-professionals) (NIST CSF 2.0 & ISO 27001) |", + "| `coco_pharma_episode_2_peter_profile.mp4` | Master | Episode 2: Zero-Touch Stewardship | Peter Profile (Lead Steward) | [Blog 6](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-6-egeria-6-3-release) (Connectors: Simon, Pasteur, Mendel) |", + "| `coco_pharma_episode_3_dr_tessa_tube.mp4` | Master | Episode 3: Clinical Data Provenance | Dr. Tessa Tube (Lead Researcher) | [Blog 8](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-8-building-the-data-mesh) (Bitol ODCS) & [Blog 9](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-9-business-flows-from-lineage) (5 Lineage Layers) |", + "| `coco_pharma_episode_4_stew_faster.mp4` | Master | Episode 4: Supply Chain & ESG | Stew Faster (Manufacturing GM) | [Blog 10](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-10-exchanging-dcat-documents) (W3C DCAT v3 JSON-LD) |", + "| `coco_pharma_episode_5_erin_overview.mp4` | Master | Episode 5: Business Glossaries | Erin Overview (Info Architect) | [Blog 9](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-9-business-flows-from-lineage) (OpenLineage in PostgreSQL) |", + "| `coco_pharma_episode_6_sidney_seeker.mp4` | Master | Episode 6: Fraud & Audit Lineage | Sidney Seeker (Fraud Auditor) | [Blog 6](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-6-egeria-6-3-release) & [Blog 7](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-7-supporting-security-professionals) (STRIDE & Reversibility) |", + "| `coco_pharma_episode_7_jules_keeper_reggie_mint.mp4` | Master | Episode 7: Executive Cockpit & ROI | Jules Keeper (CDO) & Reggie Mint (CFO) | [Blog 6](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-6-egeria-6-3-release), [Blog 7](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-7-supporting-security-professionals), [Blog 8](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-8-building-the-data-mesh) |", + "", + "---", + "", + "## Technical Reference Links (Mandy Chessell Blog Series)", + "", + "- **Blog 6 (Release 6.3):** [Egeria 6.3: A Metadata Platform Built for the Long Run](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-6-egeria-6-3-release) — Housekeeping connectors (`Simon`, `Pasteur`, `Mendel`, `Baudot`, `Wedgwood`), native `Undo`/`Restore`, and fail-closed zones.", + "- **Blog 7 (Security):** [Supporting Security Professionals: Frameworks, Threat Models & Findings](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-7-supporting-security-professionals) — NIST CSF 2.0, NIST SP 800-53 OSCAL catalogs, ISO 27001 Statement of Applicability, and STRIDE threat modeling.", + "- **Blog 8 (Data Mesh):** [Building the Data Mesh: Products, Subscriptions, Contracts & Dependencies](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-8-building-the-data-mesh) — Bitol Open Data Contract Standard (ODCS), Open Data Product Standard (ODPS), and `Darwin` runtime dependency validation.", + "- **Blog 9 (Lineage):** [From Pipeline Runs to Business Flows: Extracting Lineage Layers](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-9-business-flows-from-lineage) — PostgreSQL OpenLineage analysis (800k+ events in ~3s), five supply chain views, and bounded exploration.", + "- **Blog 10 (DCAT):** [Speaking the Language of Open Data: Exchanging DCAT Documents](https://github.com/odpi/egeria-docs/tree/main/blogs/blog-10-exchanging-dcat-documents) — W3C DCAT v3 JSON-LD exchange, DCAT Files/Event receivers, and DCAT Publishers.", + "", + "---", + "", + "## Storage & Playback Notes", + "", + "- **Video Format:** 1080p Full HD H.264 (`yuv420p`), 25 fps, `-bf 0` QuickTime compatible.", + "- **Audio Format:** 48kHz Stereo AAC (192 kbps).", + "- **Metadata:** Faststart `moov` container atom enabled for instant streaming and playback.", + "", + ] + with open(index_path, "w", encoding="utf-8") as f: + f.write("\n".join(content)) + + +def export_to_vault(vault_path: Optional[str] = None, all_assets: bool = False, dry_run: bool = False) -> Dict[str, any]: + """Helper function to run export directly from other Python scripts.""" + target_path = resolve_vault_path(vault_path) + exporter = VaultExporter(target_path, dry_run=dry_run, all_assets=all_assets) + return exporter.sync() + + +def parse_args(): + parser = argparse.ArgumentParser(description="Export Egeria Coco Pharmaceuticals video media to pdr-vault") + parser.add_argument( + "--vault-path", + "-p", + type=str, + default=None, + help="Custom destination path for pdr-vault (default: $PDR_VAULT_PATH or /Users/Shared/pdr-vault)", + ) + parser.add_argument( + "--all-assets", + "-a", + action="store_true", + help="Export all media assets including per-scene MP3s, slide PNGs, and captures (default: master MP4s only)", + ) + parser.add_argument( + "--dry-run", + "-n", + action="store_true", + help="Preview files that would be synchronized without copying", + ) + return parser.parse_args() + + +def main(): + args = parse_args() + export_to_vault(vault_path=args.vault_path, all_assets=args.all_assets, dry_run=args.dry_run) + + +if __name__ == "__main__": + main() diff --git a/my_egeria/my_egeria/recording/generate_episode_tapes.py b/my_egeria/my_egeria/recording/generate_episode_tapes.py new file mode 100644 index 00000000..4db92607 --- /dev/null +++ b/my_egeria/my_egeria/recording/generate_episode_tapes.py @@ -0,0 +1,384 @@ +""" +Generate VHS tape scripts for all 7 Coco Pharmaceuticals Business Value episodes +and execute them using Charm's `vhs` to create pixel-perfect, 1080p full-color +terminal video walkthroughs. +""" + +import subprocess +import shutil +from pathlib import Path +from episode_specs import EPISODES + +TAPES_DIR = Path(__file__).parent / "tapes" +TAPES_DIR.mkdir(parents=True, exist_ok=True) +OUTPUT_BASE_DIR = Path(__file__).parent / "output" + + +def build_tape_content(ep_id: str, episode) -> str: + ep_num = episode.episode_number + username = episode.persona_username + output_mp4 = OUTPUT_BASE_DIR / f"episode{ep_num}" / "videos" / f"episode_{ep_num}_walkthrough.mp4" + output_mp4.parent.mkdir(parents=True, exist_ok=True) + + header = f"""# VHS Tape: Episode {ep_num} - {episode.title} +# Lead Persona: {episode.lead_persona} ({username}) +# Output: 1080p TrueColor MP4 Terminal Walkthrough + +Output "{output_mp4}" + +Set Shell "bash" +Set FontSize 18 +Set FontFamily "Menlo" +Set Width 1920 +Set Height 1080 +Set Padding 20 +Set Theme "Catppuccin Mocha" +Set WindowBar "Colorful" +Set WindowBarSize 40 +Set MarginFill "#1e1e2e" +Set BorderRadius 10 +Set TypingSpeed 50ms + +Type "unset NO_COLOR && export COLORTERM=truecolor TERM=xterm-256color FORCE_COLOR=1 EGERIA_USER={username} && clear" +Enter +Sleep 500ms + +# Launch My Profile as {username} +Type "uv run my_profile" +Enter +Sleep 3.5s +""" + + if ep_id == "episode1": + body = """ +# Welcome Splash Screen (Faith Broker - HR & Data Privacy Officer) +Sleep 7s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Privacy Role & Team Hierarchy +# Focus on Governance Council and HR Leadership +Tab 1 +Sleep 2s +Down 2 +Sleep 2s +Enter +Sleep 4s +Escape +Sleep 2s + +# Scene 3: Subject Access Request (SAR) & PII Lineage +# Select Catalogs / Shop for Data from Other Functions +Tab 3 +Down 6 +Sleep 1s +Enter +Sleep 4s +Type "Clinical" +Sleep 1s +Enter +Sleep 3s +Escape +Sleep 2s + +# Navigate to My Network Graph +Tab 3 +Down 1 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 2s + +# Scene 4: Real-Time Privacy Stewardship & Feedback +# Open Leave Feedback modal +Tab 3 +Down 3 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 2s + +# Focus on To-Dos table +Tab 4 +Ctrl+T +Sleep 3s +Escape +Sleep 4s +""" + elif ep_id == "episode2": + body = """ +# Welcome Splash Screen (Peter Profile - Lead Data Steward) +Sleep 6s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Managing Curation To-Dos +Tab 4 +Sleep 1.5s +Ctrl+T +Sleep 4s +Down 2 +Sleep 2s +Escape +Sleep 2s + +# Scene 3: Real-Time Annotation & Discussion Threads +Ctrl+O +Sleep 4s +Ctrl+N +Sleep 3s +Type "Verified ISO 27001 mapping for clinical trial schemas" +Sleep 2s +Escape +Sleep 2s +Escape +Sleep 2s + +# Scene 4: Earning Karma Points & Community Leadership +Ctrl+B +Sleep 4s +Escape +Sleep 2s +Tab 1 +Sleep 2s +Down 1 +Sleep 4s +""" + elif ep_id == "episode3": + body = """ +# Welcome Splash Screen (Dr. Tessa Tube - Lead Clinical Researcher) +Sleep 7s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Shopping for Certified Clinical Data Products +Tab 3 +Down 6 +Sleep 1s +Enter +Sleep 4s +Type "Cohort" +Sleep 1s +Enter +Sleep 4s + +# Scene 3: Governed Subscription Requests & Usage Terms +Ctrl+L +Sleep 4s +Escape +Sleep 2s +Ctrl+U +Sleep 4s +Escape +Sleep 2s +Escape +Sleep 2s + +# Scene 4: Cross-Functional Research Collaboration +Tab 3 +Down 2 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 4s +""" + elif ep_id == "episode4": + body = """ +# Welcome Splash Screen (Stew Faster - Manufacturing General Manager) +Sleep 5s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Plant Manager Dashboard & Roles +Tab 1 +Sleep 2s +Down 2 +Sleep 3s + +# Scene 3: Exploring Technology Types & Factory Catalogs +Tab 3 +Down 7 +Sleep 1s +Enter +Sleep 5s +Down 2 +Sleep 2s +Up 1 +Sleep 2s +Escape +Sleep 2s + +# Scene 4: Collaborative Operational Feedback & Actions +Tab 3 +Down 3 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 3s +""" + elif ep_id == "episode5": + body = """ +# Welcome Splash Screen (Erin Overview - Enterprise Information Architect) +Sleep 6s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Architecture Cockpit & Communities Navigation +Tab 3 +Down 2 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 2s + +# Scene 3: Browsing Unified Glossary & Term Dictionaries +Tab 3 +Down 6 +Sleep 1s +Enter +Sleep 4s +Type "Architecture" +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 2s + +# Scene 4: Graph Visualization of Architectural Dependencies +Tab 3 +Down 1 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 3s +""" + elif ep_id == "episode6": + body = """ +# Welcome Splash Screen (Sidney Seeker - Lead Fraud Investigator) +Sleep 6s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Investigator Workspace & Security Team Drill-Down +Tab 1 +Sleep 2s +Down 2 +Sleep 3s + +# Scene 3: Tracing Payment Anomalies with Lineage & Comments +Ctrl+O +Sleep 4s +Escape +Sleep 2s +Tab 4 +Ctrl+T +Sleep 3s +Escape +Sleep 2s + +# Scene 4: Uncovering Shadow IT in the Network Graph +Tab 3 +Down 1 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 3s +""" + else: # episode7 + body = """ +# Welcome Splash Screen (Jules Keeper - Chief Data Officer) +Sleep 6s + +# Dismiss Splash Screen into Main Dashboard +Enter +Sleep 3.5s + +# Scene 2: Executive Cockpit Walkthrough & Portfolio Health +Tab 1 +Sleep 2s +Down 2 +Sleep 3s + +# Scene 3: CFO Perspective - Quantifying Data ROI & Risk Reduction +Tab 3 +Down 9 +Sleep 1s +Enter +Sleep 3s +Escape +Sleep 2s + +# Scene 4: Community Engagement & Collaborative Governance +Tab 3 +Down 2 +Sleep 1s +Enter +Sleep 4s +Escape +Sleep 2s +Tab 3 +Down 3 +Sleep 1s +Enter +Sleep 3s +Escape +Sleep 3s +""" + + return header + body + + +def generate_all_tapes(): + print("Generating VHS Tape scripts for Episodes 1 through 7...") + for ep_id, ep in EPISODES.items(): + tape_path = TAPES_DIR / f"{ep_id}_walkthrough.tape" + content = build_tape_content(ep_id, ep) + tape_path.write_text(content) + print(f" ✓ Created {tape_path.name}") + + +def run_vhs(ep_id: str = "all"): + if not shutil.which("vhs"): + print("[ERROR] 'vhs' CLI is not installed. Please install with 'brew install vhs'.") + return False + + targets = [ep_id] if ep_id != "all" else list(EPISODES.keys()) + for target in targets: + tape_path = TAPES_DIR / f"{target}_walkthrough.tape" + if not tape_path.exists(): + print(f"[WARN] Tape {tape_path.name} not found. Generating...") + tape_path.write_text(build_tape_content(target, EPISODES[target])) + + print(f"\n▶ Running VHS recording for {target}: {tape_path.name}...") + res = subprocess.run(["vhs", str(tape_path)], capture_output=True, text=True) + if res.returncode == 0: + print(f" ✓ Successfully recorded {target} terminal walkthrough MP4!") + else: + print(f" ✗ VHS failed for {target}:\n{res.stderr}") + return True + + +if __name__ == "__main__": + import sys + generate_all_tapes() + target_ep = sys.argv[1] if len(sys.argv) > 1 else "all" + run_vhs(target_ep) diff --git a/my_egeria/my_egeria/recording/generate_series_videos.py b/my_egeria/my_egeria/recording/generate_series_videos.py new file mode 100644 index 00000000..bf91ae92 --- /dev/null +++ b/my_egeria/my_egeria/recording/generate_series_videos.py @@ -0,0 +1,1134 @@ +#!/usr/bin/env python3 +""" +Egeria Coco Pharmaceuticals Business Value Video Series Generator. + +Generates 1080p slide cards, synthesized neural voiceovers with distinct persona +accents and voices, VHS terminal interaction tapes, and assembled narrated MP4 videos +for Episodes 1 through 7 of the Coco Pharmaceuticals Business Value Series. + +Prerequisites: + - uv pip install edge-tts Pillow textual + - brew install vhs ffmpeg ttyd (optional for VHS raster captures) +""" + +import argparse +import asyncio +import io +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path +from typing import List, Optional, Dict + +import edge_tts +from PIL import Image, ImageDraw, ImageFont + +try: + import resvg_py +except ImportError: + resvg_py = None + +from episode_specs import EPISODES, EpisodeSpec, SceneSpec +from export_vault import export_to_vault + +SCRIPT_DIR = Path(__file__).resolve().parent +OUTPUT_DIR = SCRIPT_DIR / "output" +SLIDES_DIR = OUTPUT_DIR / "slides" +AUDIO_DIR = OUTPUT_DIR / "audio" +VIDEOS_DIR = OUTPUT_DIR / "videos" +NARRATED_DIR = OUTPUT_DIR / "videos_narrated" +CAPTURES_DIR = OUTPUT_DIR / "captures" +TAPES_DIR = SCRIPT_DIR / "tapes" + + +def slugify(text: str) -> str: + """Convert text into a filesystem-safe lowercase slug.""" + text = re.sub(r"[^\w\s-]", "", text).strip().lower() + return re.sub(r"[-\s]+", "_", text) + + +class AudioSynthesizer: + """Generates speech audio using Microsoft Azure Neural TTS (via edge-tts) or macOS say.""" + + def __init__(self, audio_dir: Path): + self.audio_dir = audio_dir + self.audio_dir.mkdir(parents=True, exist_ok=True) + + async def _synth_edge_tts(self, text: str, voice: str, output_path: Path): + communicate = edge_tts.Communicate(text, voice) + await communicate.save(str(output_path)) + + def synthesize_scene(self, scene: SceneSpec, episode: EpisodeSpec, output_path: Path): + """Synthesize dialogue for a single scene with the specified persona voice.""" + voice = scene.voice or episode.voice_name + print(f" [TTS] {scene.speaker} ({episode.voice_accent}) -> {output_path.name}") + try: + asyncio.run(self._synth_edge_tts(scene.dialogue, voice, output_path)) + except Exception as e: + print(f" [WARN] Edge-TTS error ({e}), falling back to macOS 'say'...") + self._synth_macos_say(scene.dialogue, episode.offline_say_voice, output_path) + + def _synth_macos_say(self, text: str, voice: str, output_path: Path): + aiff_path = output_path.with_suffix(".aiff") + subprocess.run(["say", "-v", voice, "-o", str(aiff_path), text], check=True) + subprocess.run( + [ + "ffmpeg", + "-y", + "-i", + str(aiff_path), + "-codec:a", + "libmp3lame", + "-qscale:a", + "2", + str(output_path), + ], + capture_output=True, + check=True, + ) + if aiff_path.exists(): + aiff_path.unlink() + + def stitch_full_audio(self, scene_mp3s: List[Path], output_path: Path): + """Stitch individual scene MP3 files into a continuous episode audio track.""" + filter_inputs = "".join(f"[{i}:0]" for i in range(len(scene_mp3s))) + filter_complex = f"{filter_inputs}concat=n={len(scene_mp3s)}:v=0:a=1[a]" + cmd = ["ffmpeg", "-y"] + for p in scene_mp3s: + cmd.extend(["-i", str(p)]) + cmd.extend( + [ + "-filter_complex", + filter_complex, + "-map", + "[a]", + "-c:a", + "libmp3lame", + "-b:a", + "192k", + str(output_path), + ] + ) + subprocess.run(cmd, capture_output=True, check=True) + + +BOX_CHARS = set("╭╮╯╰│─┌┐└┘├┤┬┴┼═║╔╗╚╝╠╣╦╩╬━┃┏┓┗┛┣┫┳┻╋") + +EXTRA_SVG_STYLES = """ + .term-border { fill: #89b4fa !important; font-weight: normal; } + .term-header-title { fill: #89b4fa !important; font-weight: bold; } + .term-section-title { fill: #cba6f7 !important; font-weight: bold; } + .term-table-header { fill: #74c7ec !important; font-weight: bold; } + .term-gold { fill: #f9e2af !important; font-weight: bold; } + .term-green { fill: #a6e3a1 !important; font-weight: bold; } + .term-peach { fill: #fab387 !important; font-weight: bold; } + .term-red { fill: #f38ba8 !important; font-weight: bold; } + .term-sky { fill: #74c7ec !important; font-weight: bold; } + .term-white { fill: #cdd6f4 !important; } +""" + +CATPPUCCIN_MOCHA_MAP = { + r"#0178d4\b": "#89b4fa", # Sapphire / Blue + r"#004578\b": "#74c7ec", # Sky + r"#004854\b": "#cba6f7", # Mauve + r"#00008b\b": "#89b4fa", # Dark Blue -> Blue + r"#28c840\b": "#a6e3a1", # Green + r"#febc2e\b": "#f9e2af", # Yellow + r"#ff5f57\b": "#f38ba8", # Red +} + +KNOWN_SECTIONS = [ + "User Associations", "My Collections", "Other Functions", "Roles", "Teams", + "Blogs", "Journal", "To-Dos", "User Identity", "Projects", "Select a function", + "Shop for Data", "My Network", "Work with Communities", "User Bookmarks", + "Leave Feedback", "Feedback Log", "Subscriptions", "Catalogs/Shop for Data", + "Technology Types", "Edit Profile", "Change User", "Peer Details", "People Search", + "Community Details", "Active Communities", "Joined Communities", "Available Roles", + "Catalog Search", "Digital Data Products", "Data Dictionaries", "Business Domains" +] + + +def sanitize_svg(svg_content: str) -> str: + """Sanitize Textual/Rich SVG export to fix typography, character spacing, and full TrueColor colors.""" + # 1. Remove font-variant-east-asian: full-width (which forces double spacing) + svg = re.sub(r"font-variant-east-asian:\s*full-width;?", "", svg_content) + + # 2. Embed JetBrains Mono webfont & vibrant TrueColor styling + jb_font = f""" + @font-face {{ + font-family: "JetBrains Mono"; + src: local("JetBrainsMono-Regular"), local("JetBrains Mono"), + url("https://cdn.jsdelivr.net/gh/JetBrains/JetBrainsMono/web/woff2/JetBrainsMono-Regular.woff2") format("woff2"); + font-style: normal; + font-weight: 400; + }} + @font-face {{ + font-family: "JetBrains Mono"; + src: local("JetBrainsMono-Bold"), local("JetBrains Mono-Bold"), + url("https://cdn.jsdelivr.net/gh/JetBrains/JetBrainsMono/web/woff2/JetBrainsMono-Bold.woff2") format("woff2"); + font-style: bold; + font-weight: 700; + }} +{EXTRA_SVG_STYLES} +""" + if "