Skip to content

Document scripted SQL in fly mpg connect and use it for imports - #2647

Merged
tripledoublev merged 9 commits into
mainfrom
mpg-connect-scripted-sql
Nov 13, 2026
Merged

tripledoublev merged 9 commits into
mainfrom
mpg-connect-scripted-sql

Conversation

@tripledoublev

@tripledoublev tripledoublev commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary of changes

Merge #2649 first, or together with this PR. The new section links to the generated fly mpg connect reference page, which only lists -c and -f once #2649 lands. This PR doesn't edit that page.

flyctl v0.5.9 adds -c, -f and -- pass-through to fly mpg connect (superfly/flyctl#5436). You can now run SQL against a Managed Postgres cluster without starting fly mpg proxy or copying the cluster password into psql. The old import recipes needed both. This PR documents the new flags and moves the import recipes to them.

  • postgres/create-and-connect.mdx: new "Running SQL without a session" section. It covers -c, -f, passing flags to psql after --, SQL on stdin, exit statuses, and the fixed local port 16380 that it shares with fly mpg proxy. Also updates the page description and changes one <clusterID> placeholder to <CLUSTER_ID>.
  • postgres/import.mdx: new recipe. Dump to a file. If pg_dump is 17+ and the cluster runs Postgres 16, remove the SET transaction_timeout line from the dump header. Restore with fly mpg connect <CLUSTER_ID> -f dump.sql -- -q --single-transaction, which stops at the first error and leaves nothing behind. Adds "Restore errors", "Piping a small database" and "Using another tool" (fly mpg proxy for pg_restore and similar tools). Removes the old one-line pg_dump | psql example and the WireGuard paragraph.
  • getting-started/migrate-from-heroku.mdx: both recipes now dump to a file, then restore with fly mpg connect … -- -q -v ON_ERROR_STOP=0. Unlike the import page, they still carry on past errors, as before, because this page already lists Heroku extension errors to ignore. New text says which errors are expected, and that after a dropped connection you should restore into a new, empty database instead of re-running.
  • getting-started/troubleshooting.mdx: adds the one-shot form to the list of ways to connect.
  • postgres/index.mdx: two mentions of "pg_dump and pg_restore" now say "pg_dump and psql", to match the import page.

Two problems found during testing are now documented on the import page. pg_dump 17+ writes SET transaction_timeout = 0;, which Postgres 16 rejects. And a dump from another MPG cluster fails on COMMENT ON EXTENSION for the preinstalled pgaudit and pg_stat_monitor.

Worth a close read: the "Restore errors" list on the import page, and the Heroku page carrying on past errors on purpose.

Preview

Related Fly.io community and GitHub links

Notes

The Vale check passes.

Tested on Linux with flyctl built at the v0.5.9 commit and psql/pg_dump 18.6, against a throwaway MPG Basic cluster on Postgres 16. The large runs used a 2.36 GB plain-SQL dump (10M rows, 8 indexes, a second schema, an enum type). Each was checked afterwards by row counts, id sums and index count.

Run Result
Import recipe: header edit, then -f … -- -q --single-transaction Completed, 19 min. A data row with the same text as the removed header line was kept
Same, on a pg_dump of an MPG cluster taken with --exclude-extension=pgaudit --exclude-extension=pg_stat_monitor Completed, 39 min
Import recipe on an unedited pg_dump 18 file Stopped at line 13 (transaction_timeout), nothing left behind
Heroku page form: -f … -- -q -v ON_ERROR_STOP=0 Completed, 20 min, with the one expected transaction_timeout error
Piped form (-f -), small database Completed. With a failed pg_dump, exits 0 without pipefail and 1 with it
Restore into a new database, then fly mpg attach --database DATABASE_URL points at the new database
Heroku page commands on a real Heroku Postgres 18 database (5 extensions, 25k rows), into MPG on Postgres 16 and 17 Completed, all rows, sequences, indexes and views present. Only the errors the page lists: pg_stat_statements (2), Heroku's event triggers (5), and transaction_timeout on Postgres 16

Every -c, -f, --, stdin and exit-status statement in the new section was also run.

Not tested:

  • macOS and Windows. The recipes are bash, like the neighboring pages.
  • The piped form on a large database. The page limits it to small ones.
  • The out of shared memory entry under "Restore errors" comes from a local reproduction with 3,000 tables on stock Postgres. The threshold on MPG is unknown.

🤖 Generated with Claude Code

Covers -c, -f, the -- pass-through and exit status from flyctl#5436, and
replaces the fly mpg proxy + psql import recipes with fly mpg connect.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
fly-io 🟢 Ready View Preview Nov 12, 2026, 11:54 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

Replace the whole-dump grep filter with a header-only edit, make the
file restore single-transaction so a stopped run can be retried, limit
the piped form to small databases, and match page conventions.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Say when a failed dump or dropped connection leaves partial data and how
to recover, spell out the piped sed, read SQL from stdin without cat,
and tidy step punctuation and headings.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@tripledoublev
tripledoublev marked this pull request as ready for review November 12, 2026 22:49
@tripledoublev
tripledoublev requested a review from a team as a code owner November 12, 2026 22:49

@kcmartin kcmartin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against flyctl v0.5.9 and it all matches. Thanks for the thorough testing. A few small things:

  • Holding until #2649 merges so the reference page lists -c/-f.
  • --file takes - for stdin with the same defaults, so the piped examples could be ... | fly mpg connect <CLUSTER_ID> -f - -- -q --single-transaction and skip -X -v ON_ERROR_STOP=1. Any reason not to?
  • The Heroku page says to create a new cluster after a dropped connection; the import page says a new empty database. Could both use the database option?
  • troubleshooting.mdx: <cluster-id> → <CLUSTER_ID> to match.

From review: read piped SQL with -f - so it gets the file defaults,
recover a dropped Heroku restore into a new database instead of a new
cluster, and match the cluster placeholder on the troubleshooting page.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@tripledoublev

tripledoublev commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks @kcmartin for the quick feedback! There was actually no reason not to use -f -.
I just checked it on v0.5.9:

  • stops on first error
  • skips psqlrc
  • works piped with --single-transaction

Both pages use it now, and the troubleshooting placeholder is <CLUSTER_ID>.

The Heroku page uses the database option too. I can confirm fly mpg attach --database points DATABASE_URL at the new database, and the import page now mentions that flag as well. I also tightened a couple of sentences under "Restore errors".

Let me know if anything else should be changed.

@kcmartin kcmartin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my comments! LGTM!

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@tripledoublev
tripledoublev merged commit 05850bc into main Nov 13, 2026
1 check passed
@tripledoublev
tripledoublev deleted the mpg-connect-scripted-sql branch November 13, 2026 00:15

This branch was successfully deployed

1 active deployment
staging — d36b0d2d Deployed Nov 12, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants