feat(cloud): sync a local folder with Deepnote Cloud by tkislan · Pull Request #601 · deepnote/vscode-deepnote

tkislan · 2026-11-04T07:03:32Z

Summary

Adds two commands that sync local folders with Deepnote Cloud: "Deepnote: Sync with Deepnote Cloud", run from the Command Palette or the sync button in the Deepnote explorer title bar, and "Deepnote: Sync a New Folder with Deepnote Cloud…". Both call syncWorkspace from @deepnote/cloud-sync once: projects changed in Deepnote are pulled, projects changed locally are pushed, and each conflict gets an Override or Skip prompt. A summary of the counts follows.

Two commands: they follow VS Code's Git commands, which use the only repository directly, ask when there are several, and keep "Initialize Repository" separate.
- Sync with Deepnote Cloud syncs folders that are already synced, found by their .deepnote-sync.json. With none, it offers "Sync a New Folder…". With one, it syncs it without asking. With several, it asks, listing first the folder that contains the file in the active editor.
- Sync a New Folder with Deepnote Cloud… opens a folder dialog. It is in the Command Palette and the Deepnote explorer's "…" menu. A folder chosen inside a sync root syncs that root.
Refused folders: a folder outside the open workspace folders is refused, and so is one that contains or is inside another synced folder. These checks run before any token is requested. The nested-folder search passes exclude: null to workspace.findFiles, because any other value keeps files.exclude applied and can hide a nested manifest.
Before syncing: a first sync asks before downloading every project. Unsaved Deepnote notebooks, and .deepnote files with unsaved changes in a text editor, get a "Save and Sync" modal. Nothing syncs if one stays unsaved.
Conflicts: cloud-changed-after-local-edit, changed-on-both-sides and untracked-local-directory each get a modal with one override button and "Skip". Dismissing the modal skips.
Summary: pulled, pushed, unchanged, skipped and failed counts, plus projects kept locally after they left Deepnote and local files that were not uploaded. It has a "Show Log" button and is a warning when any project failed. Every project outcome is logged.
Working files: sync leaves each project's .files directory alone, as deepnote sync does without --all-files. The issue keeps allFiles out of scope.
One sync at a time: both commands share one guard. A second run while a sync is in progress shows a message and does nothing else. The summary and the "Sync a New Folder…" offer are not awaited, so leaving them on screen does not block the next sync.
Token lookup: IDeepnoteApiTokenService now reads DEEPNOTE_TOKEN from the .env files between the resource's folder and the nearest folder holding .deepnote-sync.json. With no such folder, it reads only the resource's own folder.
- Before, it went up to the workspace folder. A folder's first sync, before its manifest existed, could then read a .env above the folder that later syncs ignored.
- Now every sync of a folder, the first included, reads only that folder's .env, as deepnote sync does. This is the rule Get deepnote sync and publish ready for the editor and agents deepnote#543 proposes for every CLI command.
- Callers pass only the resource.
- Trade-off: a .env at the workspace root no longer applies to files in subfolders that are not synced.
deepnote.apiUrl setting: the base URL of the Deepnote API, with the same resource scope as deepnote.domain.
- When it is empty (the default), the API URL stays https://api.<deepnote.domain>.
- Sync, the token check and "Open in Deepnote" use it. Browser links still use deepnote.domain.
- Saved tokens are keyed by the API URL, so a saved token is never sent to a different one.
E2E: a new cloud group runs against a mock Deepnote Cloud in test/e2e/helpers/mockDeepnoteCloud.ts.
- The mock is an express server that serves, from a temp directory, the Public API v2 routes that the token check and sync call. Its types are generated from the published OpenAPI spec (npm run generate:e2e:deepnote-cloud-api), and npm run mock:deepnote-cloud runs it for manual testing.
- Its import route answers 422 and writes nothing for the archives Deepnote Cloud rejects: a body that is not a ZIP, a document of another project, a document without exactly one notebook, and documents that disagree on the project name or on their integration ids. mockDeepnoteCloud.e2e.test.ts checks this over HTTP (9 tests).
- syncWithDeepnoteCloud.e2e.test.ts (10 tests) opens one workspace. Each test creates its own cloud projects and local folders, runs the commands through the UI, and checks the summary and the files on both sides. Each folder's .env holds the token, except in the test that saves one with Set API Token and syncs with it. The tests cover:
  - the "Sync a New Folder…" offer, and a first sync that downloads a project with two notebook files;
  - an edited and a new notebook pushed as one project, with no folder pick;
  - notebooks edited, added and deleted in Deepnote, mirrored locally;
  - a conflict resolved to the Deepnote version, and one skipped while another project still pushes;
  - the folder pick listing the active editor's folder first;
  - local notebooks outside every project, and projects deleted in Deepnote, kept and reported;
  - a project's .files directory, neither uploaded nor removed by a pull.
- After a failed test, the suite closes the prompts it left open, so the sync behind them cannot block the next test.
- fflate is a new devDependency, used for the project ZIPs.
Availability: both commands are disabled, and their explorer entries hidden, in untrusted workspaces and in the web extension host, the same as Set and Clear API Token.

Closes #597

Builds on #563 (BLU-6518, the token service).

Deviations from the issue

Two commands instead of one pick. The issue asked for one command that lists the synced folders followed by "Choose another folder…" on every run, even with a single synced folder. The command is now split as described above.
No syncRoot argument. The issue asked for an optional syncRoot argument on getToken and getOrPromptToken. The token service is shared with future Deepnote Cloud features, so the lookup rule changed instead (see "Token lookup").
Beyond the issue's scope: the deepnote.apiUrl setting and the cloud E2E group with its mock.
When the workspace folder is a filesystem or drive root (/, D:\), the overlap check skips the parent lookup. Without the guard, such a folder's own manifest would make every sync after the first fail as "overlaps itself". The issue doesn't mention this case.
The unit tests cover more than the issue's list: a dismissed folder dialog, a failed sync-root lookup for a folder chosen in the dialog, a token lookup that throws (the in-progress flag still clears), the quick-pick and dialog options, and logging of outcomes without a detail and of warnings.

Verification

npm run typecheck, npm run lint (no findings in the changed files), npm run format and npm run spell-check all pass.
npm test: 2985 passing, 0 failing.
The cloud E2E group passed locally, 19 tests, on an extension packaged from the branch's current source.
The new unit tests were checked against single-line bugs applied to the compiled code one at a time, and each bug failed at least one test. There were 19 for the first version and 8 for the command split. Examples: exclude: undefined, swapped override labels, awaiting the summary, awaiting the "Sync a New Folder…" offer, a separate guard for the new command, and not stripping the notebook URI's query before matching the active editor.
E2E tests were checked the same way, with a bug applied to the packaged extension per run:
- swapped conflict answers failed the two conflict tests;
- a summary without the untracked-files sentence failed that test;
- a pull that clears the project folder failed only the .files pull test.
- With the active-editor ordering removed, the previous version of the pick-order test failed (['alpha', 'beta'] instead of ['beta', 'alpha']). The rewritten test checks the same order.
A sync test changed to leave its conflict prompts unanswered failed alone; the other 9 sync tests passed.
The token lookup tests fail when the old fallback to the workspace folder is restored.

Not verified

No real Deepnote workspace was used. The unit tests stub syncWorkspace, and the E2E suite runs it against the mock. The issue's "Manual follow-up" is not done: the round trip against real Deepnote, the Cursor check and the public docs PR.
Windows was not run. The drive-root guard was confirmed for / only, and the new unit tests compare raw paths, so they may fail on Windows. CI runs on Ubuntu.
Known from review and not addressed here: in a trusted virtual (vscode-vfs) workspace, the first-sync confirmation appears before the token service refuses the non-local folder.

linear-code · 2026-11-04T07:03:39Z

BLU-6519

codecov · 2026-11-04T07:06:51Z

Codecov Report

❌ Patch coverage is 96.44970% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 38%. Comparing base (dfe6f85) to head (e83a1c6).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
.../cloud/syncWithDeepnoteCloudCommandHandler.node.ts	96%	2 Missing and 3 partials ⚠️
src/notebooks/serviceRegistry.node.ts	0%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##            main    #601    +/-   ##
======================================
  Coverage     38%     38%            
======================================
  Files        824     825     +1     
  Lines      41251   41418   +167     
  Branches    9079    9111    +32     
======================================
+ Hits       15816   15944   +128     
- Misses     23350   23397    +47     
+ Partials    2085    2077     -8

Files with missing lines	Coverage Δ
...oks/deepnote/cloud/deepnoteApiTokenService.node.ts	`93% <100%> (-1%)`	⬇️
src/notebooks/deepnote/cloud/types.ts	`100% <ø> (+100%)`	⬆️
src/notebooks/deepnote/importClient.node.ts	`50% <100%> (+3%)`	⬆️
src/platform/common/constants.ts	`100% <100%> (ø)`
src/platform/common/utils/localize.ts	`93% <100%> (+<1%)`	⬆️
src/notebooks/serviceRegistry.node.ts	`0% <0%> (ø)`
.../cloud/syncWithDeepnoteCloudCommandHandler.node.ts	`96% <96%> (ø)`

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Adds "Deepnote: Sync with Deepnote Cloud", run from the Command Palette or the Deepnote explorer title bar. It calls `syncWorkspace` from `@deepnote/cloud-sync` once to pull projects changed in Deepnote, push projects changed locally, and ask Override or Skip for each conflict, then reports the counts. - The command offers the workspace's synced folders, or a folder from a dialog, which syncs the nearest sync root at or above it. - It refuses a folder outside the open workspace folders, or one that contains or is inside another synced folder. The nested check passes `exclude: null` to `findFiles`, because any other value keeps `files.exclude` applied and can hide a nested manifest. - A first sync asks before downloading every project, and unsaved Deepnote notebooks must be saved first. - One sync runs at a time. The summary is not awaited, so leaving it on screen does not block the next sync. - `getToken` and `getOrPromptToken` take an optional `syncRoot`, which stops the `.env` lookup there. Every sync of a folder, the first included, then reads `DEEPNOTE_TOKEN` only from that folder's `.env`, as `deepnote sync` does, instead of from a `.env` above it until the manifest exists. Closes #597

The "Save and Sync" check only looked at notebook documents. A .deepnote file reopened with Reopen Editor With > Text Editor is a text document, so its unsaved edits were skipped. A pull then wrote the cloud version under the dirty buffer, and choosing Overwrite in the "content of the file is newer" dialog made the next sync push the pre-pull text back, undoing the cloud change. Dirty text documents with the file scheme and a .deepnote path now go through the same save-and-recheck flow. The scheme check leaves out notebook cells, which carry the notebook's .deepnote path under their own scheme.

Uri.fsPath lower-cases a Windows drive letter, but the temp directory from os.tmpdir() keeps an upper-case one. Paths the test built from it then differed from the handler's fsPath-derived paths, so six tests would fail on Windows. The temp directory now goes through Uri.file(...).fsPath once, as the API token service test already does.

The API URL was always https://api.<deepnote.domain>, so the extension could not talk to a local or mock Deepnote API. When deepnote.apiUrl is set, getApiEndpoint returns it without trailing slashes; when it is empty (the default) nothing changes. Browser links still use deepnote.domain. The setting has the same resource scope as deepnote.domain. Saved tokens are keyed by the API URL, so a saved token is never sent to a different one.

Adds a cloud E2E group. The suite starts test/e2e/helpers/mockDeepnoteCloud.ts on a free port and sets deepnote.apiUrl in its temp workspace's .vscode/settings.json. It then saves an API token, which the mock checks with GET /v2/me, and syncs three times: - first sync: downloads the cloud project - second sync: pushes a local edit - third sync: keeps the cloud version in a conflict The mock serves the Public API v2 routes that the token check and sync call (me, list projects, export, import) from a directory with one subdirectory per project. Its response types are generated from the published OpenAPI spec (npm run generate:e2e:deepnote-cloud-api). It also runs on its own for manual testing (npm run mock:deepnote-cloud). fflate is a new devDependency; the mock uses it to build and read project zip archives. clickDialogOkButton now takes the button label, because the sync folder dialog sets openLabel to "Sync Here".

getToken and getOrPromptToken took a sync-only syncRoot argument, so every future Deepnote Cloud caller would have had to pick where the .env lookup stops. The argument existed because the lookup fell back to the workspace folder when no sync manifest was found. A folder's first sync could then read a .env above the folder, and later syncs could not. The lookup now runs from the resource's folder up to the nearest folder holding .deepnote-sync.json, and reads only that first folder when there is none. A folder reads the same .env before and after its first sync, as deepnote/deepnote#543 proposes for every CLI command. A .env at the workspace root no longer applies to files in unsynced subfolders.

The mock wrote any import archive whose entries were .deepnote files that parse. Deepnote Cloud answers 422 when a document belongs to another project, has other than one notebook, or disagrees with the others on the project name or on the integration ids it declares. The mock now does the same, before any write. A body that is not a ZIP gets 422 instead of the 500 from the error middleware. The new mockDeepnoteCloud.e2e.test.ts in the cloud group posts archives to the import route over HTTP and checks the status, the message and that rejected imports leave the project's files unchanged.

…sync-a-local-folder-with-deepnote-cloud-from-the-extension

… folder The sync command showed a pick of the synced folders plus "Choose another folder…" on every run, even when only one folder was synced. It now works like VS Code's Git commands, which use the only repository directly and keep "Initialize Repository" separate: - "Sync with Deepnote Cloud" syncs folders that are already synced. With none, it offers "Sync a New Folder…". With one, it syncs it without asking. With several, it asks, listing the folder of the active editor first. - "Sync a New Folder with Deepnote Cloud…" opens the folder dialog and runs the existing checks and first-sync confirmation. It is in the Command Palette and the Deepnote explorer's "…" menu. Both commands share the one-sync-at-a-time guard. The offer is not awaited, so the guard is clear by the time its button runs the other command. The cloud E2E suite now syncs a first folder through the offer, syncs it again without a pick, adds a second folder with the new command, and checks the pick order with a file from the second folder open.

…sync-a-local-folder-with-deepnote-cloud-from-the-extension

Each command method now calls its folder chooser directly and passes only the chosen folder to syncFolder, instead of forwarding a chooser callback through three methods. The one-sync-at-a-time guard moves into runOneSyncAtATime, which still wraps the folder choice, so no dialog or pick opens while a sync runs. The commands are registered with registerCommand's thisArg. The unit test stub now binds the captured handler to it as VS Code does, and a new test covers a failed search for synced folders, whose error handling is now separate from the new-folder chooser's.

Every sync used to run in one `before` hook, with each test only asserting on values recorded there. Now `before` only starts the mock and opens the workspace. Each test creates its own cloud projects and local folders, runs the commands, and asserts. `afterEach` empties both sides, and after a failed attempt it first closes the prompts the failure left open, so the sync behind them cannot block later tests. Each local folder gets the token from its own `.env`, so only the Set API Token test relies on a saved token, and that test clears it. New coverage: - a project with several notebook files, pulled, pushed with an edited and a new notebook, and mirrored after cloud edits, adds and deletes; - a conflict in one of several projects resolved with Skip; - local notebooks outside every project and projects deleted in Deepnote, reported and kept; - a project's `.files` directory, which sync neither uploads nor removes on a pull because the extension does not pass `allFiles`.

tkislan force-pushed the tomaskislan/blu-6519-sync-a-local-folder-with-deepnote-cloud-from-the-extension branch from 980d76e to 948b57d Compare November 4, 2026 11:23

Base automatically changed from tomaskislan/blu-6518-store-the-deepnote-api-token-in-env-and-resolve-it-for to main November 5, 2026 14:49

tkislan added 2 commits November 5, 2026 14:50

tkislan force-pushed the tomaskislan/blu-6519-sync-a-local-folder-with-deepnote-cloud-from-the-extension branch from 948b57d to 544fb1f Compare November 5, 2026 14:55

tkislan added 4 commits November 5, 2026 16:37

tkislan mentioned this pull request Nov 7, 2026

feat(cloud-mock): add a local Deepnote Cloud API mock and test sync against it deepnote/deepnote#631

Draft

This was referenced Nov 7, 2026

Publish a folder as a Deepnote app #608

Open

feat(cloud): publish a folder as a Deepnote app #609

Draft

Merge remote-tracking branch 'origin/main' into tomaskislan/blu-6519-…

fa8e800

…sync-a-local-folder-with-deepnote-cloud-from-the-extension

tkislan mentioned this pull request Nov 7, 2026

chore(deps): update @deepnote packages to their zod 4 releases #610

Merged

tkislan added 5 commits November 9, 2026 09:24

Merge remote-tracking branch 'origin/main' into tomaskislan/blu-6519-…

5e9b23c

…sync-a-local-folder-with-deepnote-cloud-from-the-extension

refactor(cloud): compare the chosen sync folder with null explicitly

ea8e99b

Navigation Menu

Repository navigation

feat(cloud): sync a local folder with Deepnote Cloud - #601

feat(cloud): sync a local folder with Deepnote Cloud#601
tkislan wants to merge 13 commits into
mainfrom
tomaskislan/blu-6519-sync-a-local-folder-with-deepnote-cloud-from-the-extension

tkislan commented Nov 4, 2026 •

edited

Loading

Uh oh!

linear-code Bot commented Nov 4, 2026

Uh oh!

codecov Bot commented Nov 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

tkislan commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Deviations from the issue

Verification

Not verified

Uh oh!

linear-code Bot commented Nov 4, 2026

Uh oh!

codecov Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tkislan commented Nov 4, 2026 •

edited

Loading

codecov Bot commented Nov 4, 2026 •

edited

Loading