Skip to content

sqlite: add sqlite.query tracing channel - #66581

Open
logaretm wants to merge 1 commit into
nodejs:mainfrom
logaretm:awad/sqlite-tracing-channel
Open

logaretm wants to merge 1 commit into
nodejs:mainfrom
logaretm:awad/sqlite-tracing-channel

Conversation

@logaretm

@logaretm logaretm commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

sqlite in Node only has the sqlite.db.query "profiling" channel, which can't give APMs real query spans, using a tracing channels allows for:

  • Adds clear start event.
  • Clearer span timings based on the actual operation runtime.
  • Makes use of error channel, which allows APMs to mark failing query spans.
  • Queries can be more easily parsed and sanitized.

This PR publishes run(), get(), all(), exec() and the tag store equivalents to a sqlite.query TracingChannel.

The tests check the context for statement, tag store and exec queries, error events for SQL that fails to compile in exec() and for runtime failures, that a store bound to start stays active during the query, that iterate() isn't traced, and that every query method publishes exactly once.

The end result is APM's and users can build spans/metrics/logs telemetry out of these channels easily with a simple subscriber. This is an example of a subscriber that creates spans out of it (using disk driver):

CleanShot 2026-10-13 at 10 39 50 AM

Regarding tag stores

For tracing tag store queries, we needed to cache the SQLTagStore template, since createTagStore() built a new one per store, so every store had its own prototype. Stores now share one prototype, which is visible from JS.

Happy to split this and tag store tracing into a follow-up PR if that's easier to review.

Benchmarks

ns/call, no subscribers native this PR
get() 336.7 330.4
run() 627.1 625.5
get(), cold 384.4 397.7
run() that throws 1817.0 1910.6
run() that throws, cold 1690.0 1875.8

Median of 30 interleaved runs on an M3 Pro, in-memory db, 10k calls, with a 300k-call warm-up except for the cold rows. "native" is the binding's native method, the one main puts on the prototype. Like #66575, calls that throw pay for the extra JS frame, about 90 ns once optimized.

'use strict';
// Usage: node --expose-internals --no-warnings sqlite-query-bench.js <native|pr> <get|run|throw> <cold|warm>
// Prints ns per call. "native" calls the binding's native method directly,
// which is what main installs on Statement.prototype.
const { internalBinding } = require('internal/test/binding');
const { Database, Statement } = require('node:sqlite');

const [variant, kind, temp] = process.argv.slice(2);
const method = kind === 'get' ? 'get' : 'run';

const db = new Database(':memory:');
db.exec('CREATE TABLE t (x INTEGER PRIMARY KEY); INSERT INTO t VALUES (1)');
const stmt = db.prepare({
  get: 'SELECT x FROM t WHERE x = ?',
  run: 'INSERT OR REPLACE INTO t VALUES (?)',
  throw: 'INSERT INTO t VALUES (?)', // Always violates the primary key.
}[kind]);

// Both variants are installed as own properties so lookups are identical.
stmt[method] = variant === 'native' ?
  internalBinding('sqlite').statementQueryMethods[method] :
  Statement.prototype[method];

function call() {
  try {
    stmt[method](1);
  } catch {
    // Expected for 'throw'.
  }
}

if (temp === 'warm') for (let i = 0; i < 3e5; i++) call();
const n = 1e4;
const t0 = process.hrtime.bigint();
for (let i = 0; i < n; i++) call();
console.log(Number(process.hrtime.bigint() - t0) / n);

Refs: #62241
Refs: #66575

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance
  • @nodejs/sqlite

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Oct 7, 2026
@logaretm logaretm changed the title sqlite: add sqlite.query tracing channel sqlite: add sqlite.query tracing channel Oct 7, 2026
@logaretm
logaretm force-pushed the awad/sqlite-tracing-channel branch from 8bb848d to 62d59d8 Compare October 7, 2026 22:33
@logaretm
logaretm marked this pull request as ready for review October 7, 2026 22:34
@codecov

codecov Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.78531% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.65%. Comparing base (d630b5d) to head (1d8998c).
⚠️ Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
src/node_sqlite.cc 78.84% 4 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66581      +/-   ##
==========================================
+ Coverage   90.59%   90.65%   +0.06%     
==========================================
  Files         797      797              
  Lines      278230   278439     +209     
  Branches    53482    53549      +67     
==========================================
+ Hits       252066   252424     +358     
+ Misses      16558    16395     -163     
- Partials     9606     9620      +14     
Files with missing lines Coverage Δ
lib/sqlite.js 100.00% <100.00%> (ø)
src/node_sqlite.h 87.27% <ø> (ø)
src/node_sqlite.cc 82.25% <78.84%> (+0.04%) ⬆️

... and 39 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@logaretm
logaretm force-pushed the awad/sqlite-tracing-channel branch 2 times, most recently from d316b62 to 4d08cae Compare October 8, 2026 01:48
Comment thread doc/api/diagnostics_channel.md
logaretm added a commit to getsentry/js-tracing-channels-proposals that referenced this pull request Oct 9, 2026

@Qard Qard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The hasSubscribers checks could be simplified, but also fine to leave as-is. Using the shared one would check the other async channels too unnecessarily.

Comment thread lib/sqlite.js Outdated
@logaretm
logaretm force-pushed the awad/sqlite-tracing-channel branch from 4d08cae to a24fe85 Compare October 26, 2026 15:46
Publish run(), get(), all(), exec() and the matching SQLTagStore methods
to a 'sqlite.query' TracingChannel, so APMs can produce query spans with
correct parent linkage, errors, and the source SQL with its parameters.

The query methods are now defined in lib/sqlite.js around the native
implementations, which the binding exports. Without subscribers they
only check the channel's hasSubscribers. iterate() is not traced.

Signed-off-by: Abdelrahman Awad <abdelrahman.awad@sentry.io>
@logaretm
logaretm force-pushed the awad/sqlite-tracing-channel branch from a24fe85 to 1d8998c Compare October 26, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants