Repository navigation
Document scripted SQL in fly mpg connect and use it for imports - #2647
Conversation
Covers -c, -f, the -- pass-through and exit status from flyctl#5436, and replaces the fly mpg proxy + psql import recipes with fly mpg connect. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Replace the whole-dump grep filter with a header-only edit, make the file restore single-transaction so a stopped run can be retried, limit the piped form to small databases, and match page conventions. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
# Conflicts: # postgres/index.mdx
Say when a failed dump or dropped connection leaves partial data and how to recover, spell out the piped sed, read SQL from stdin without cat, and tidy step punctuation and headings. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
kcmartin
left a comment
There was a problem hiding this comment.
Checked against flyctl v0.5.9 and it all matches. Thanks for the thorough testing. A few small things:
- Holding until #2649 merges so the reference page lists
-c/-f. --filetakes-for stdin with the same defaults, so the piped examples could be... | fly mpg connect <CLUSTER_ID> -f - -- -q --single-transactionand skip-X -v ON_ERROR_STOP=1. Any reason not to?- The Heroku page says to create a new cluster after a dropped connection; the import page says a new empty database. Could both use the database option?
- troubleshooting.mdx:
<cluster-id>→<CLUSTER_ID>to match.
From review: read piped SQL with -f - so it gets the file defaults, recover a dropped Heroku restore into a new database instead of a new cluster, and match the cluster placeholder on the troubleshooting page. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Thanks @kcmartin for the quick feedback! There was actually no reason not to use
Both pages use it now, and the troubleshooting placeholder is The Heroku page uses the database option too. I can confirm Let me know if anything else should be changed. |
kcmartin
left a comment
There was a problem hiding this comment.
Thanks for addressing my comments! LGTM!
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Summary of changes
Merge #2649 first, or together with this PR. The new section links to the generated
fly mpg connectreference page, which only lists-cand-fonce #2649 lands. This PR doesn't edit that page.flyctl v0.5.9 adds
-c,-fand--pass-through tofly mpg connect(superfly/flyctl#5436). You can now run SQL against a Managed Postgres cluster without startingfly mpg proxyor copying the cluster password into psql. The old import recipes needed both. This PR documents the new flags and moves the import recipes to them.postgres/create-and-connect.mdx: new "Running SQL without a session" section. It covers-c,-f, passing flags to psql after--, SQL on stdin, exit statuses, and the fixed local port 16380 that it shares withfly mpg proxy. Also updates the page description and changes one<clusterID>placeholder to<CLUSTER_ID>.postgres/import.mdx: new recipe. Dump to a file. Ifpg_dumpis 17+ and the cluster runs Postgres 16, remove theSET transaction_timeoutline from the dump header. Restore withfly mpg connect <CLUSTER_ID> -f dump.sql -- -q --single-transaction, which stops at the first error and leaves nothing behind. Adds "Restore errors", "Piping a small database" and "Using another tool" (fly mpg proxyforpg_restoreand similar tools). Removes the old one-linepg_dump | psqlexample and the WireGuard paragraph.getting-started/migrate-from-heroku.mdx: both recipes now dump to a file, then restore withfly mpg connect … -- -q -v ON_ERROR_STOP=0. Unlike the import page, they still carry on past errors, as before, because this page already lists Heroku extension errors to ignore. New text says which errors are expected, and that after a dropped connection you should restore into a new, empty database instead of re-running.getting-started/troubleshooting.mdx: adds the one-shot form to the list of ways to connect.postgres/index.mdx: two mentions of "pg_dumpandpg_restore" now say "pg_dumpand psql", to match the import page.Two problems found during testing are now documented on the import page.
pg_dump17+ writesSET transaction_timeout = 0;, which Postgres 16 rejects. And a dump from another MPG cluster fails onCOMMENT ON EXTENSIONfor the preinstalledpgauditandpg_stat_monitor.Worth a close read: the "Restore errors" list on the import page, and the Heroku page carrying on past errors on purpose.
Preview
Related Fly.io community and GitHub links
fly mpg connectwithout a terminal flyctl#5436: adds the flags, released in flyctl v0.5.9fly mpg connectreference page for v0.5.9Notes
The Vale check passes.
Tested on Linux with flyctl built at the v0.5.9 commit and psql/pg_dump 18.6, against a throwaway MPG Basic cluster on Postgres 16. The large runs used a 2.36 GB plain-SQL dump (10M rows, 8 indexes, a second schema, an enum type). Each was checked afterwards by row counts, id sums and index count.
-f … -- -q --single-transactionpg_dumpof an MPG cluster taken with--exclude-extension=pgaudit --exclude-extension=pg_stat_monitortransaction_timeout), nothing left behind-f … -- -q -v ON_ERROR_STOP=0transaction_timeouterror-f -), small databasepg_dump, exits 0 withoutpipefailand 1 with itfly mpg attach --databaseDATABASE_URLpoints at the new databasepg_stat_statements(2), Heroku's event triggers (5), andtransaction_timeouton Postgres 16Every
-c,-f,--, stdin and exit-status statement in the new section was also run.Not tested:
out of shared memoryentry under "Restore errors" comes from a local reproduction with 3,000 tables on stock Postgres. The threshold on MPG is unknown.🤖 Generated with Claude Code