Repository navigation
Conversation
…QL, MariaDB and SQLite Add the contrib module google-adk-database-session-service: a BaseSessionService that keeps sessions, events and app/user state in a relational database through a caller-supplied javax.sql.DataSource, in the ADK Python v1 table layout. - DatabaseSessionService(DataSource) and a builder (dialect, InstantSource, UuidProvider, opt-in auto-migration, statement timeout); the module's only public type. - The schema is created by an explicit migrate() with embedded Flyway, using its own adk_schema_history table and migration location. - One transaction per operation. An append locks app and user state only when the event changes them and rejects a stale writer; the caller's session is updated only after the commit. - One contract suite runs against InMemorySessionService, SQLite, PostgreSQL, MySQL and MariaDB; a workflow runs it on real databases.
|
Hi @voborl00, Thank you for your contribution and for taking the time to submit this pull request. Our team is currently reviewing your changes, and we will reach out if we need any further information. |
damianmomotgoogle
left a comment
There was a problem hiding this comment.
Thanks for the rework. It addresses the points from #700 and the #647 schema, and documents each difference. Below are answers to your questions and a few requests (details inline).
Event format: if possible - use the Python format. Check if we could reuse SessionJsonConverter here
- Differences 1-4:
event_sequence: OK. Java event timestamps use milliseconds and often tie.- PostgreSQL
JSON: OK. ADK Pythonmainmade the same switch for the same reasons. - Stale-writer check by last event ID: OK for now. The Java Runner always loads the full history. Please document in the README that a session loaded with a filter must be reloaded before appending.
adk_java_storage_format: keep it, but after the format change it should mean "written by ADK Java in the Python format". Please read it once per service instance rather than at the start of every transaction.
- Python index: ok to keep it.
- Separate workflow: ok. Please add
core/**to itspathsso that changes toEvent/Sessionserialization also run the real-database suite. - Filtered session: acceptable for v1 if documented (see 1).
- Docs: keeping the setup instructions in the module README is fine. Please remove the
sql/scripts; creating databases and accounts is a deployment concern, andmigrate()creates the tables.
| * @throws SessionException if the event has no ID, without which it could not be read back, or | ||
| * holds a value that cannot be written as JSON | ||
| */ | ||
| String writeEvent(Event event) { |
There was a problem hiding this comment.
Please write the event in ADK Python's model_dump(exclude_none=True, mode="json") form: snake_case keys (leave keys inside args, response, and state values as written), and timestamp plus actions.compaction.start_timestamp/end_timestamp in float seconds. Do the reverse on read. Core's SessionJsonConverter already handles the read direction for the Vertex service and may be reusable. Please add a test that reads fixtures produced by ADK Python.
| value VARCHAR(256) NOT NULL | ||
| ); | ||
| INSERT INTO adk_internal_metadata (key, value) VALUES | ||
| ('schema_version', '1'), ('adk_java_storage_format', 'jdbc-v1'); |
There was a problem hiding this comment.
Keep this row, but after the event format change it should mean "written by ADK Java in the Python format" (using a new value, e.g. python-v1). Because Python-created schemas lack event_sequence, the check still guards against them.
| phase = Phase.SCHEMA; | ||
| try (PreparedStatement query = prepare(connection, DatabaseDialect.metadataQuery(dialect)); | ||
| ResultSet metadata = query.executeQuery()) { | ||
| DatabaseSchemaMigrator.requireMetadata(metadata); |
There was a problem hiding this comment.
This reads adk_internal_metadata at the start of every transaction, adding a query to every operation to guard against a schema swapped mid-flight. Checking it once per service instance (e.g. when the store is created) is enough.
| String actual = hasTail ? rows.getString("id") : null; | ||
| // Both empty, or the same last event. Anything else means the caller's copy is behind | ||
| // the stored history, or was loaded with a filter that cut its end off. | ||
| if (!Objects.equals(actual, expectedTailId)) { |
There was a problem hiding this comment.
Fine for v1, since the Java Runner always loads the full history. Please document in the README that a session loaded with numRecentEvents or afterTimestamp must be reloaded before appendEvent. If Java later adds Python's RunConfig.get_session_config, this check will need to accept filtered sessions, like Python's _storage_update_marker.
| if (existing.isEmpty()) { | ||
| return false; | ||
| } | ||
| if (!existing.contains("adk_internal_metadata")) { |
There was a problem hiding this comment.
If the first migrate() call is interrupted after Flyway creates adk_schema_history but before V1 commits (e.g. on a connection drop or timeout), only adk_schema_history exists. This check then throws "not created by ADK Java" on every retry, leaving the schema unrecoverable without manually dropping that table. Please treat a history table with no applied V1 as an empty schema, and add a test for this case.
| } | ||
| ``` | ||
|
|
||
| - The module's migration and the application's own Flyway migrations do not |
There was a problem hiding this comment.
Not quite: they share the schema, and execution order matters (see the comment above). Please reword.
| </dependency> | ||
| <dependency> | ||
| <groupId>org.flywaydb</groupId> | ||
| <artifactId>flyway-core</artifactId> |
There was a problem hiding this comment.
Please mark flyway-core as <optional>true</optional>, matching the per-database modules. The code already handles its absence via the LinkageError path in migrate(). As a compile dependency, it puts Flyway on every consumer's classpath, which on Spring Boot 3.x triggers Boot's Flyway auto-configuration for the application's primary DataSource.
| } | ||
| // One append at a time per Session object: the second of two concurrent calls has to | ||
| // see the event the first one added, or it would be rejected as stale. | ||
| synchronized (session) { |
There was a problem hiding this comment.
This synchronizes on a caller-visible object (Session) across the entire database round trip. Consider using a private per-session lock, or relying on the session row lock and the stale check like the other services.
| pull_request: | ||
| branches: [main, 'release/**'] | ||
| paths: | ||
| - 'contrib/database-session-service/**' |
There was a problem hiding this comment.
Please add core/** here so that changes to Event/Session serialization in core also trigger the real-database suite.
| -- ANY KIND, either express or implied. See the License for the specific language | ||
| -- governing permissions and limitations under the License. | ||
|
|
||
| -- Prepares PostgreSQL for the ADK Java database session service: a database of its own and |
There was a problem hiding this comment.
Please remove both sql/ scripts. Creating databases and accounts is a deployment concern, and migrate() creates the tables. ADK Python and Go don't ship setup scripts either.
Adds
contrib/database-session-service(artifactgoogle-adk-database-session-service): SessionService that keeps sessions, events and app/user state in a relational database through a caller-suppliedjavax.sql.DataSource. It supports PostgreSQL, MySQL, MariaDB and SQLite and uses the ADK Python table layout.This is the new PR agreed in #647. It follows the review in #700 and the schema listed in #647. Fixes #665.
Contains:
DatabaseSessionService(DataSource)and a builder (dialect,InstantSource,UuidProvider, opt-in auto-migration, statement timeout).Schema creation through an explicit
migrate(), with Flyway embedded: it runs over the caller'sDataSource, keeps its history inadk_schema_history, and reads its scripts fromcom/google/adk/sessions/migration/<dialect>/.One contract suite that runs against
InMemorySessionService(the part both share), SQLite, PostgreSQL, MySQL and MariaDB.README with setup steps, and
sql/scripts that create a dedicated database and accounts.Constructor and builder as requested. There are no URL or properties constructors and no connection pool of its own.
Migration runs only through
migrate()orautoMigrate(true). The module reads no system properties or environment variables.DatabaseSessionServiceis the only public type; everything else is package-private.There is no
getAppState,setAppState,getUserStateorsetUserState.Only
SessionExceptionandSessionNotFoundExceptionare thrown, without driver or Flyway exceptions and without caller data in messages. A duplicate ID givesSESSION_ALREADY_EXISTS.The dialect can be set on the builder; otherwise it is taken from the connection's metadata.
Differences from the Python v1 schema
The five tables, their columns and keys, and the index
idx_events_app_user_session_ts_idare as listed in #647, withJSONandTIMESTAMP(6)mapped as described there. The exceptions are below. Each can be changed if you prefer strict parity:eventshas an extraevent_sequence BIGINT, unique per session, and history is read in that order because Java event timestamps are caller-set milliseconds that can tie or go backwards, andInMemorySessionServicereturns history in append order.(timestamp, id)cannot reproduce that.For Python parity: drop the column and order by
(timestamp, id). Append order is then lost when timestamps tie.PostgreSQL uses
JSON, notJSONB. becausJSONBrejects strings with an escaped NUL and rewrites numbers, so an event would not always be stored, or read back, as it was appended. The service never queries inside the documents.For Python parity: switch to
JSONBand reject such payloads on append.A stale writer is detected by the last event ID, not by
update_time. The caller's last event ID is compared with the stored tail while the session row is locked.sessions.update_timeis still the last event's timestamp. The reason is that Python compares a private marker kept on itsSession; Java'sSessionhas no such field, and a millisecondupdate_timedoes not change when two events share a timestamp.For Python parity: add a revision marker to core
Sessionin a separate change and compare that.adk_internal_metadatahas a second row,adk_java_storage_format = jdbc-v1, and the service refuses a schema without it because the layout matches Python but the payloads do not. For example, Java stores event time in milliseconds and Python in seconds. Sharing a database across SDKs was not a goal (feat: Implementation of MySQL-backed session service for ADK #647).For Python parity: remove the row and the check.
Questions:
event_sequence. Should the index stay for parity?.github/workflows/database-session-service.yml, which runs the contract suite against real PostgreSQL, MySQL and MariaDB through Testcontainers when the module or the rootpom.xmlchanges. The regularvalidationbuild covers the unit tests and SQLite. Is a separate workflow fine?sql/scripts, or should they go to adk-docs?Smaller additions:
Con PostgreSQL,utf8mb4_0900_binon MySQL,utf8mb4_nopad_binon MariaDB,BINARYon SQLite), so identifiers compare the way Java strings do. Python uses the database default.CHECKconstraints require the JSON columns to hold a valid JSON object.events.event_datastays nullable as in Python, but Java always writes it.invocation_idisNOT NULL, so an event without one is stored with an empty string.adk_schema_historyis a sixth table.Outside the module the PR changes the root
pom.xml(module entry and Flyway entries),release-please-config.json(the module README, so its version snippet is updated) and adds the workflow above.