URL Encoding Explained: percent-encoding, query strings, and the gotchas

URL encoding is one of those topics that feels trivial until production starts returning 400 Bad Request on every search that contains a French accent. The rules are simple, but they are scattered across four specs that do not entirely agree, and the JavaScript helpers have names that actively mislead. This guide pulls the rules into one place and marks the places where the standards quietly disagree.

What “URL encoding” actually means

URLs are restricted to a small subset of ASCII. Any character outside that set — or any character that has a structural meaning the parser would otherwise interpret — has to be represented as one or more bytes prefixed by %. That is percent-encoding, and the rules for what to encode come from RFC 3986. You can try any of the examples below with our URL encoder.

RFC 3986 splits characters into three groups. The unreserved set is A-Z a-z 0-9 - . _ ~; these are always safe and never need encoding. The reserved set (: / ? # [ ] @ ! $ & ' ( ) * + , ; =) carries structural meaning in some part of the URL; encode them when you mean the literal character, leave them alone when you mean the delimiter. Everything else — including space, ", <, >,{, and all non-ASCII — must be percent-encoded.

Where each character needs to be encoded

The same byte may or may not need encoding depending on which component of the URL it appears in. A path segment allows : and @ unencoded; a query value does not need to encode / or ? (because the parser stops at #); a fragment is the most permissive component of all. In practice, almost everyone ignores those distinctions and encodes anything outside the unreserved set whenever they are interpolating a value into a URL. That is conservative and correct.

The one place this matters: do not encode the / characters that separate your path segments. Encode each segment, then join with /. If you encode the slash too, you get %2F in the path, which most servers will refuse for security reasons (it would otherwise let attackers smuggle .. sequences past directory guards).

encodeURI vs encodeURIComponent in JavaScript

JavaScript ships two encoders, and the difference between them is the source of an enormous amount of wasted debugging time.

• encodeURI(value) assumes value is already a complete URL. It leaves the reserved characters (; / ? : @ & = + $ , #) alone so the URL stays parseable. It encodes spaces, accents, and the small set of characters that are never allowed anywhere in a URL.
• encodeURIComponent(value) assumes value is a single component being inserted into a URL — usually a path segment or a query value. It encodes everything except the unreserved set plus ! ' ( ) *.

The rule of thumb: if you are building a URL by concatenation, use encodeURIComponent on every interpolated value. If you are normalising an already-built URL, encodeURI is the right tool. There is almost no case where you want to apply both.

Form data: the +/space exception

HTML form submissions encode using the media type application/x-www-form-urlencoded, defined in the WHATWG URL standard. That format diverges from RFC 3986 in two ways. First, spaces become + instead of %20. Second, a literal + in the value is encoded as %2B so the decoder can tell them apart.

Both %20 and + decode to a space in practice — every mainstream server-side parser accepts both in a query string. But emitters should be consistent. Use + in form bodies and in query strings that imitate form bodies; use %20 elsewhere, especially in path segments where + is a literal plus sign.

Nested arrays and objects: there is no standard

Query strings were designed for flat key=value pairs. Nested data structures have to be flattened, and the flattening rule is a per-framework decision. The four conventions in widespread use:

1. Repeated keys. ?tag=js&tag=ts becomes an array on the server. The simplest convention; the default in Express, Go's net/url, and most Node parsers.
2. PHP-style brackets. ?tag[]=js&tag[]=ts for arrays; ?user[name]=alex for nested objects. The brackets must themselves be percent-encoded (%5B / %5D) to be strictly standards-compliant, though most servers accept the raw form.
3. Rails dot notation. ?user.name=alex&user.age=30. Less common today; appears in some older Ruby and .NET APIs.
4. Comma-separated. ?tag=js,ts. Common in REST APIs (OpenAPI lists it as the form/explode=false style). Concise but ambiguous if values themselves can contain commas.

Pick one convention per API and document it. The most painful bugs in this area come from a client that emits brackets talking to a server that expects repeated keys — both seem to work, but one of them silently keeps only the last value.

CJK, emoji, and other non-ASCII

Every character outside ASCII must be encoded as its UTF-8 bytes, then percent-encoded byte by byte. The character 北 (U+5317) is the three bytes E5 8C 97 in UTF-8, which become %E5%8C%97 in a URL. Emoji follow the same rule: 🔥 (U+1F525) is the four bytes F0 9F 94 A5, encoded as %F0%9F%94%A5.

Legacy systems sometimes emit the same character in a different encoding — GBK for Chinese, Shift-JIS for Japanese, Windows-1251 for Cyrillic. If you are integrating with a pre-2010 backend and your decoded text comes out as garbage, try interpreting the bytes as Windows-1252 or the locale's native encoding before assuming the URL is corrupt.

Internationalised domain names

Host names follow a different rule. Non-ASCII host names are encoded as Punycode (RFC 3492), not percent-encoded. The domain 例え.jp becomes xn--r8jz45g.jp in the actual DNS query. Browsers display the Unicode form for recognised scripts but transmit the Punycode form on the wire. If you find yourself constructing a URL with a non-ASCII host, convert the host with a Punycode encoder, not the URL encoder.

Common gotchas

• Double encoding. Encoding a value, then passing the result through another encoder, produces %2520 instead of %20. Track each encoding step explicitly; never “encode just to be safe” on a value you did not produce.
• Hash fragments are client-side only. Everything after # is never sent to the server. Putting state in the fragment is a deliberate choice (privacy, single-page-app routing); putting it there by accident hides bugs until the server starts depending on it.
• Pluses in mailto links. mailto:[email protected] works because mailto URLs use RFC 3986 rules, not form encoding. Build mailto links by hand, not with a form-encoder.
• The unreserved set excludes the tilde-historically. RFC 2396 (the predecessor to 3986) treated ~ as reserved. Older encoders still emit %7E; the modern standard considers it unreserved. Both decode the same.
• Length limits are real. Most servers cap URLs at 8 KB. Browsers vary from 2 KB (older IE) to 32 KB (modern Chrome). If your query string approaches that range, move the payload into a POST body.

A practical workflow

When building URLs in code: construct an object representing the components (scheme, host, path segments, query map), then serialise once with a library you trust — URL and URLSearchParams in browsers, the equivalents in every other major language. Hand-assembled URLs are the bugs.

Remember that the encoded URL is only the request line; the same HTTP layer carries headers your code rarely escapes manually but should still treat with care. The Referer header leaks the previous URL (including its query string) to the next host unless you strip it with a referrer policy, the User-Agent string lets servers vary behaviour per client, and the connecting IP address arrives in the TCP layer or — behind a proxy — in an X-Forwarded-For header. Anything you put in the query string is visible to all three.

When debugging an encoded URL someone sent you, paste it into our URL encoder/decoder and toggle the direction. The tool decodes once per click, so if you have to click twice to get readable text you have confirmed double-encoding upstream.

The honest takeaway

Use encodeURIComponent on every interpolated value, never on a complete URL. Match your form encoding (space-as-plus) to your transport context. Pick one array convention per API and document it. Trust UTF-8. And when something looks doubly encoded, it almost certainly is — trace the second encoder rather than papering over it with a decode pass.