UnicodeEncodeError: 'utf-8' codec can't encode character

What this error means

You tried to turn text into bytes using a codec that cannot represent one of the characters. Note the direction: encode means text → bytes, which is the opposite of the UnicodeDecodeError people meet more often.

Traceback (most recent call last):
  File "demo.py", line 2, in <module>
    print("price: ₹100")
UnicodeEncodeError: 'ascii' codec can't encode character '\u20b9' in position 8: ordinal not in range(128)

Three parts of the message matter: the codec that failed, the character it could not handle (shown as an escape), and the position. Together they identify the problem precisely.

Why utf-8 in the message is almost never the real culprit

This is the confusing part, and it is worth being precise. UTF-8 can encode every Unicode character that exists. So a genuine 'utf-8' codec can't encode character error is rare, and usually means one of two narrow things: a lone surrogate, or a codec that is *named* utf-8 but is not really being used.

# 1 — a lone surrogate, which UTF-8 genuinely cannot encode
bad = '\ud800'                     # half of a surrogate pair
bad.encode('utf-8')                 # 💥 'utf-8' codec can't encode
                                    #    character '\ud800'
# this usually comes from decoding with errors='surrogateescape'
# or from a broken source; fix the source, not the encoder

# 2 — the message says ascii or cp1252, which is the common real case
'₹'.encode('ascii')                 # 💥 'ascii' codec can't encode
'₹'.encode('cp1252')                # 💥 'cp1252' codec can't encode
'₹'.encode('utf-8')                 # ✅ b'\xe2\x82\xb9' — utf-8 is fine

So when you see this error, read the codec name first. If it says ascii or cp1252 or latin-1, the problem is that the target encoding is too narrow — not that your text is unusual.

The Windows console case

The most common real-world report. On Windows the console historically defaults to a legacy code page such as cp1252, so printing non-Latin text fails even though the file itself is fine:

# ❌ crashes on a cp1252 console
print("कीमत: ₹100")

# ✅ tell Python explicitly which encoding to use
import sys
sys.stdout.reconfigure(encoding="utf-8")     # Python 3.7+
print("कीमत: ₹100")

# or set it for the whole process before Python starts
#   set PYTHONIOENCODING=utf-8        (cmd)
#   $env:PYTHONIOENCODING="utf-8"    (PowerShell)

# Windows Terminal and modern PowerShell 7 default to UTF-8 already

sys.stdout.reconfigure is the clean fix when you control the program. Setting PYTHONIOENCODING is better when you do not, because it fixes the behaviour for every script in that environment.

Writing to a file

The second most common case. The default encoding for open() is platform-dependent, so code that works on macOS or Linux fails on Windows:

# ❌ uses the platform default — cp1252 on Windows
with open("out.txt", "w") as f:
    f.write("कीमत: ₹100")            # 💥 on a cp1252 system

# ✅ always state the encoding
with open("out.txt", "w", encoding="utf-8") as f:
    f.write("कीमत: ₹100")            # works everywhere

# reading has exactly the same trap
with open("data.txt", encoding="utf-8") as f:
    text = f.read()

Always pass encoding= to open(). Relying on the platform default is the single most portable-code bug in Python — it passes every test on the author's machine and fails on the first Windows user's.

The errors= argument, and why to avoid it

text = "price: ₹100"

text.encode("ascii", errors="replace")    # b'price: ?100'
text.encode("ascii", errors="ignore")     # b'price: 100'
text.encode("ascii", errors="xmlcharrefreplace")   # b'price: ₹100'
text.encode("ascii", errors="backslashreplace")    # b'price: \\u20b9100'

These are escape hatches, not fixes. errors='ignore' deletes data silently, and errors='replace' substitutes ? — so a currency amount becomes ?100 and nobody notices until someone reads the output. Use them only when lossy output is genuinely acceptable, such as a log line that must never crash a process.

If you are reaching for errors=, ask first whether the real fix is to write UTF-8 instead. Almost always it is.

Diagnosing systematically

1. Read the codec name in the message. utf-8 means something unusual such as a lone surrogate; ascii or cp1252 means the target is too narrow.
2. Check what the default encoding actually is: python -c "import sys; print(sys.stdout.encoding, sys.getdefaultencoding())".
3. If it is a console problem, reconfigure stdout or set PYTHONIOENCODING.
4. If it is a file problem, add encoding="utf-8" to every open() call — reads and writes.
5. If it is a subprocess or pipe, pass encoding="utf-8" to subprocess.run() as well.
6. Only then consider errors=, and prefer backslashreplace so the lost character stays visible.

import subprocess

# subprocess pipes have the same platform-default trap
result = subprocess.run(["cmd"], capture_output=True, text=True,
                        encoding="utf-8", errors="backslashreplace")

Related messages worth knowing

• UnicodeDecodeError: 'utf-8' codec can't decode byte — the mirror image — bytes to text, usually a binary file or wrong encoding.
• ValueError — the parent class of both Unicode errors.
• AttributeError — what you get calling encode() on bytes instead of str.