Repository navigation
Speed up urllib.parse.unquote() for ASCII input #160690
Description
Feature or enhancement
unquote() runs every string containing % through a regex that splits it into ASCII and non-ASCII chunks. For pure ASCII input, the a common case for URLs and form data, we can skip the regex and generator and decode directly:
if string.isascii():
return _unquote_impl(string).decode(encoding, errors)This also speeds up unquote_plus(), parse_qs() and parse_qsl(). Non-ASCII input is unchanged.
Pyperf ooutput on a PGO+LTO build:
unquote short ascii: Mean +- std dev: [main] 1.95 us +- 0.14 us -> [patched] 956 ns +- 85 ns: 2.04x faster
unquote long ascii: Mean +- std dev: [main] 92.4 us +- 5.2 us -> [patched] 88.5 us +- 7.0 us: 1.04x faster
unquote_plus form value: Mean +- std dev: [main] 2.05 us +- 0.13 us -> [patched] 1.09 us +- 0.09 us: 1.87x faster
parse_qsl 5 fields: Mean +- std dev: [main] 7.76 us +- 0.51 us -> [patched] 5.90 us +- 0.44 us: 1.32x faster
parse_qs 5 fields: Mean +- std dev: [main] 8.52 us +- 0.55 us -> [patched] 6.30 us +- 0.35 us: 1.35x faster
parse_qsl 30-field form: Mean +- std dev: [main] 73.4 us +- 4.8 us -> [patched] 44.2 us +- 3.4 us: 1.66x faster
Benchmark hidden because not significant (2): unquote no escapes, unquote non-ascii
Geometric mean: 1.36x faster
Benchmark script
import pyperf
from urllib.parse import unquote, unquote_plus, parse_qsl, parse_qs
runner = pyperf.Runner()
runner.bench_func("unquote short ascii", unquote, "/users/John%20Smith/items")
runner.bench_func("unquote no escapes", unquote, "/users/john/items")
runner.bench_func("unquote long ascii", unquote, "a%20b%2Fc" * 200)
runner.bench_func("unquote non-ascii", unquote, "/café/%E2%82%AC")
runner.bench_func("unquote_plus form value", unquote_plus, "hello+world%21+how+are+you")
QS = "q=python+web+framework&page=2&sort=-created_at&filter%5Btag%5D=a%2Cb&limit=50"
runner.bench_func("parse_qsl 5 fields", parse_qsl, QS)
runner.bench_func("parse_qs 5 fields", parse_qs, QS)
FORM = "&".join(f"field{i}=value+number+{i}%21" for i in range(30))
runner.bench_func("parse_qsl 30-field form", parse_qsl, FORM, True)Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
This is a very small change, I did not find a previous discussion in my search
Activity