Character encoding converter and mojibake diagnostics

Encoding

Loading tool

Preparing the tool. The remaining time is not yet known.

All processing for this tool happens in your browser. Your input is not sent to a server.

About this tool

Keep the existing UTF-8, UTF-16LE and UTF-16BE converter for text, hexadecimal and Base64. The separate mojibake diagnostics panel inspects pasted Unicode text or the original bytes of a local file. It lists bounded, explainable candidates while preserving the source; it cannot reliably detect an encoding or establish which wording was intended.

Common uses

Encode known text as UTF-8 or explicitly ordered UTF-16 bytes; decode known hexadecimal or Base64 with the existing converter.
Investigate a single mistaken decoding step, such as UTF-8 bytes previously interpreted as Windows-1252, without replacing the pasted source.
Compare explicit decoding routes for an original local file before its bytes are lost through copying, replacement characters or an editor save.
Review escaped controls, BOM evidence and route assumptions, then deliberately select one candidate to copy or download as UTF-8.

How to use it

1.For ordinary conversion, select Encode or Decode, UTF-8/UTF-16LE/UTF-16BE and Hex/Base64, enter the value, and run the existing converter.
2.For diagnostics, choose pasted text or an original local file and run Diagnose. Text recovery reverses one assumed mistaken decoding step; file diagnostics decode the supplied bytes through explicit routes. Review the unchanged source and all available evidence.
3.Candidates use a stable route order, without confidence percentages or language-based ranking. Select one candidate and confirm that you reviewed it before copying or downloading its complete UTF-8 text. Editing, cancelling or changing the source invalidates old results.

Executable encoding and recovery examples

Existing UTF-8 / Hex conversion

request (ASCII JSON): {"mode":"converter","operation":"encode","encoding":"utf-8","format":"hex","input":"\u4f60\u597d"}

{"output":"E4BDA0E5A5BD"}

The original converter still emits the six UTF-8 bytes for these two characters.

Existing UTF-16BE / Base64 conversion

request (ASCII JSON): {"mode":"converter","operation":"encode","encoding":"utf-16be","format":"base64","input":"A\ud83d\ude00"}

{"output":"AEHYPd4A"}

The bytes 00 41 D8 3D DE 00 contain a complete surrogate pair; byte order is explicitly big-endian.

Existing UTF-16LE / Hex decoding

request (ASCII JSON): {"mode":"converter","operation":"decode","encoding":"utf-16le","format":"hex","input":"41003DD800DE"}

{"output":"A\ud83d\ude00"}

The same text is recovered from explicitly little-endian bytes by the original Decode mode.

Two routes, one café candidate text

request (ASCII JSON): {"mode":"text","text":"caf\u00c3\u00a9"}

{"candidates":[{"id":"windows-1252-to-utf-8","text":"caf\u00e9","byteRoundTrip":true,"stringRoundTrip":true},{"id":"iso-8859-1-to-utf-8","text":"caf\u00e9","byteRoundTrip":true,"stringRoundTrip":true}],"nativeOptional":true,"otherwise":"unsupported: native decoder unavailable or conformance check failed"}

Both single-byte hypotheses reconstruct 63 61 66 C3 A9. Matching output does not tell you which historical decoding occurred. A missing or nonconforming native Windows-1252 decoder is explicitly unsupported; the literal route remains independent.

Literal C1 controls require the literal route

request (ASCII JSON): {"mode":"text","text":"\u00e2\u0080\u0099"}

{"candidates":[{"id":"iso-8859-1-to-utf-8","text":"\u2019","byteRoundTrip":true,"stringRoundTrip":true}],"missing":["windows-1252-to-utf-8"]}

Literal code points E2 80 99 reconstruct the UTF-8 apostrophe bytes. U+0080 cannot be inverted as Windows-1252 byte 80, which maps to U+20AC.

One-step Chinese recovery hypothesis

request (ASCII JSON): {"mode":"text","text":"\u00e4\u00bd\u00a0\u00e5\u00a5\u00bd"}

{"candidates":[{"id":"windows-1252-to-utf-8","text":"\u4f60\u597d","byteRoundTrip":true,"stringRoundTrip":true},{"id":"iso-8859-1-to-utf-8","text":"\u4f60\u597d","byteRoundTrip":true,"stringRoundTrip":true}],"nativeOptional":true,"otherwise":"unsupported: native decoder unavailable or conformance check failed"}

The exact byte transformation yields these characters without consulting language frequency. The displayed source includes escaped U+00A0. A missing or nonconforming native Windows-1252 decoder is explicitly unsupported; the literal route remains independent.

A question mark is not recoverable history

request (ASCII JSON): {"mode":"text","text":"Order?"}

{"candidates":[{"id":"original-text","text":"Order?","unchanged":true}],"warnings":["question-mark-may-be-original"]}

The source stays unchanged. The question mark may be intentional or an earlier replacement; this tool cannot distinguish those histories.

Existing U+FFFD stays visible as an escape

request (ASCII JSON): {"mode":"text","text":"bad\ufffd"}

{"candidates":[{"id":"original-text","text":"bad\ufffd","unchanged":true}],"missing":["windows-1252-to-utf-8","iso-8859-1-to-utf-8"],"warnings":["replacement-may-be-original"]}

No missing letter is fabricated. U+FFFD cannot be encoded by either inverse single-byte route, so the unchanged source is retained.

Reject an unpaired surrogate

request (ASCII JSON): {"mode":"text","text":"\ud800"}

{"error":"unpaired-surrogate"}

ASCII JSON can describe this malformed UTF-16 value safely. Diagnostics reject it before UTF-8 export could replace it silently.

Bidi, newline and NUL remain escaped in previews

request (ASCII JSON): {"mode":"text","text":"A\u202eB\n\u0000"}

{"candidates":[{"id":"original-text","text":"A\u202eB\n\u0000","unchanged":true}],"preview":"A\\u202EB\\n\\u0000"}

The preview is printable ASCII and cannot reorder nearby labels. The manually selected UTF-8 export still contains the original characters; it is not sanitization.

Preserve the UTF-8 BOM code point

request (ASCII JSON): {"mode":"bytes","hex":"EF BB BF 41"}

{"candidates":[{"id":"decode-utf-8","text":"\ufeffA","byteRoundTrip":true}],"bom":"utf-8"}

Diagnostic decoding keeps the leading U+FEFF. UTF-8 export therefore emits EF BB BF 41 without adding or removing characters.

Explicit UTF-16LE with a surrogate pair

request (ASCII JSON): {"mode":"bytes","hex":"FF FE 41 00 3D D8 00 DE"}

{"candidates":[{"id":"decode-utf-16le","text":"\ufeffA\ud83d\ude00","byteRoundTrip":true}],"bom":"utf-16le"}

The LE route reconstructs the original Unicode text and preserves U+FEFF. A diagnostic download is UTF-8, not the original UTF-16 bytes.

Explicit UTF-16BE with the same text

request (ASCII JSON): {"mode":"bytes","hex":"FE FF 00 41 D8 3D DE 00"}

{"candidates":[{"id":"decode-utf-16be","text":"\ufeffA\ud83d\ude00","byteRoundTrip":true}],"bom":"utf-16be"}

The BE route is separate from LE; successful decoding is evidence for that route and does not establish the file history.

Malformed UTF-8 is a failed route

request (ASCII JSON): {"mode":"bytes","hex":"C3 28"}

{"missing":["decode-utf-8"],"invalidAttempts":["decode-utf-8"]}

28 is not a continuation byte after C3. No replacement-based UTF-8 candidate is created; other encodings remain separate hypotheses.

Odd-length UTF-16 is rejected

request (ASCII JSON): {"mode":"bytes","hex":"41"}

{"missing":["decode-utf-16le","decode-utf-16be"],"invalidAttempts":["decode-utf-16le","decode-utf-16be"]}

One byte cannot supply a complete UTF-16 code unit. Both endian routes fail; it may still be valid single-byte or UTF-8 text.

Byte 80: web alias versus literal mapping

request (ASCII JSON): {"mode":"bytes","hex":"80"}

{"candidates":[{"id":"decode-windows-1252","text":"\u20ac","byteRoundTrip":true},{"id":"decode-iso-8859-1-literal","text":"\u0080","byteRoundTrip":true}],"nativeOptional":true,"otherwise":"unsupported: native decoder unavailable or conformance check failed"}

Windows-1252 yields U+20AC, while literal ISO-8859-1 yields C1 control U+0080. The WHATWG ISO-8859-1 alias belongs to the Windows-1252 interpretation. A missing or nonconforming native Windows-1252 decoder is explicitly unsupported; the literal route remains independent.

An explicit WHATWG Shift_JIS reading

request (ASCII JSON): {"mode":"bytes","hex":"82 A0"}

{"candidates":[{"id":"decode-shift_jis","text":"\u3042","byteRoundTrip":null}],"nativeOptional":true,"otherwise":"unsupported: native decoder unavailable or conformance check failed"}

The native Shift_JIS route decodes these bytes. No legacy encoder is supplied, so byte-round-trip evidence is null, not true. If the native decoder is unavailable or fails the conformance check, this route is explicitly unsupported instead.

An explicit GB18030 reading

request (ASCII JSON): {"mode":"bytes","hex":"D6 D0 CE C4"}

{"candidates":[{"id":"decode-gb18030","text":"\u4e2d\u6587","byteRoundTrip":null}],"nativeOptional":true,"otherwise":"unsupported: native decoder unavailable or conformance check failed"}

This common mapping is decoded by the host runtime. The GB18030 revision is not pinned, and no byte-round-trip proof or language ranking is claimed. If the native decoder is unavailable or fails the conformance check, this route is explicitly unsupported instead.

An explicit WHATWG Big5 reading

request (ASCII JSON): {"mode":"bytes","hex":"A4 A4 A4 E5"}

{"candidates":[{"id":"decode-big5","text":"\u4e2d\u6587","byteRoundTrip":null}],"nativeOptional":true,"otherwise":"unsupported: native decoder unavailable or conformance check failed"}

The browser Big5 mapping is used where supported. It is not promised to match every historical Big5 implementation; round-trip evidence remains untested. If the native decoder is unavailable or fails the conformance check, this route is explicitly unsupported instead.

A UTF-32 signature is evidence, not support

request (ASCII JSON): {"mode":"bytes","hex":"FF FE 00 00 41 00 00 00"}

{"bom":"utf-32le","warnings":["utf32-not-supported"]}

The prefix is reported as UTF-32LE evidence, but no UTF-32 decoder is offered. Other valid readings must not be mistaken for automatic UTF-32 recovery.

Interpreting diagnostic results safely

Do not treat the first candidate as the correct answer. Stable route order is not a confidence ranking.
Do not paste hexadecimal into the text-recovery input expecting it to be treated as bytes. Use the existing Hex/Base64 converter, or supply the original file in diagnostic file mode.
Do not confuse the browser ISO-8859-1 alias with the explicitly literal ISO-8859-1 route. C1-range bytes have different interpretations.
An invalid UTF-8 or UTF-16 route is a failed hypothesis; another successful legacy route does not establish the true encoding.
A file beginning with EF BB BF, FF FE or FE FF can provide useful BOM evidence, but explicit endian routes remain distinct and the diagnostic text preserves U+FEFF.
Unpaired surrogate input cannot be losslessly exported as UTF-8. Obtain well-formed source data rather than accepting silent replacement.
Escaped examples below use ASCII JSON. The visible \uXXXX and \n sequences describe characters; they are not live control characters embedded in the page.

Limits and notes

Diagnostics are hypotheses, not reliable encoding detection. A successful strict decode or exact round trip only establishes consistency with a route; several routes can produce valid text, and none proves the original encoding or meaning. Text inverse recovery needs every pasted character to map back to a byte in the assumed legacy encoding, followed by strict UTF-8 decoding. Only supported one-step routes are tried; no recursive repairs, spelling corrections, translation or automatic overwrite occur.
WHATWG TextDecoder labels ISO-8859-1, latin1 and ascii resolve to Windows-1252. The separate literal ISO-8859-1 route maps each byte 00–FF directly to U+0000–U+00FF, including C1 controls, and is deliberately distinct from that browser alias. File routes are UTF-8, UTF-16LE, UTF-16BE, Windows-1252, literal ISO-8859-1, Shift_JIS, GB18030 and Big5. The last three use available native WHATWG decoders without a legacy encoder, so byte-round-trip evidence is untested. Shift_JIS follows web-compatible mappings; GB18030 version is runtime-dependent, and Big5 may differ from historical implementations. UTF-32 is not decoded; an identified prefix is only a warning. Native routes must also pass representative conformance checks; failing checks are shown as unsupported rather than producing a misleading candidate.
Diagnostic Unicode routes reject malformed UTF-8, odd-length UTF-16 and unpaired surrogates. UTF-16LE and UTF-16BE are explicit hypotheses, not automatic endian detection. A decoded leading U+FEFF is retained; EF BB BF, FF FE and FE FF are evidence, not permission to strip data. U+FFFD or a question mark may already stand for discarded information. Their presence cannot reveal the missing bytes or prove damage, since either character can be intentional. The tool never invents that lost history; obtain the original file or a backup where possible.
Diagnostics allow at most 64 KiB (65,536) original file bytes; pasted text must fit both 65,536 UTF-16 code units and 64 KiB of UTF-8. There are at most 12 candidates, each at most 65,536 code units and 262,144 UTF-8 bytes. A cancellable worker has a 5-second timeout; local file reading has a 15-second timeout. Oversized input is rejected rather than silently truncated.
Diagnostic previews escape non-ASCII, control and bidi characters; previews may be shortened, but an explicitly selected export contains the complete candidate, including any controls and U+FEFF. This is not sanitization. No candidate is automatically applied, and the original file is never overwritten. Processing is local, with no input upload or persistent storage. Diagnostic copy/download is UTF-8 only, not a legacy-encoding writer. The separate original converter retains its 4 MiB encoded-text, 8 MiB decoded-byte and 16 MiB encoded-character limits and its existing binary downloads.

Frequently asked questions

What is the difference between pasted text recovery and decoding a file?

Pasted text has already been decoded somewhere. The inverse route assumes a specific earlier error, reconstructs bytes from those characters and strictly decodes them as UTF-8. An original file still supplies actual bytes, so a direct decoding route need not guess that earlier step. Keep a copy of the original bytes whenever possible.

Does a candidate or successful round trip prove the original encoding?

No. ASCII is shared by many encodings; other byte strings may also decode under several routes. A round trip checks only the stated transformation. Candidate order is deterministic, with no probability score, language guess or automatic winner.

Why do Windows-1252 and literal ISO-8859-1 differ?

The web-compatible ISO-8859-1 label is an alias for Windows-1252: byte 80 maps to U+20AC there. The separately named literal route maps 80 to U+0080. An inverse route using the wrong mapping can fail or produce a different candidate; inspect the escaped code points and provenance.

Can lost characters be recovered, and what does a download preserve?

Not from those characters alone. Multiple different inputs can collapse into the same replacement, so the missing history is unknowable here. Existing U+FFFD and question marks are warnings to check original files, not evidence that the tool can reconstruct the omitted content. Diagnostic decoding preserves a leading U+FEFF instead of silently removing it. Escaped previews keep bidi and control characters from changing the surrounding display. After manual selection and review, copy or download contains the candidate itself as UTF-8, including preserved characters; inspect it before using it in another application.