what difference does redirection make?

[Benjamin Kaplan]

On Sun, Oct 17, 2010 at 3:04 PM, Nikola Skoric <nick at fly.srk.fer.hr> wrote:
> When I execute
> nick at rilmir:~/code/simplepyged/docs/examples$ python latex.py
> I get expected output (bunch of latex markup).
>
> But, when I add a redirection, I get:
> nick at rilmir:~/code/simplepyged/docs/examples$ python latex.py > foo.tex
>  File "latex.py", line 87, in <module>
>    print  mytemplate.render_unicode(stack=stack, index=latex_index(stack), pages=pages)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 625: ordinal not in range(128)
>
> Now, how does python even know I added a redirection?! And then... why
> would it care?
>
> Code of latex.py can be seen here:
> http://github.com/dijxtra/simplepyged/blob/5ad8e0f14e93bde74520b502e5af71d78ed55bf0/docs/examples/latex.py
>

You're trying to write Unicode to a file. But there's no such thing as
a Unicode file- everything has to be stored on disk as a sequence of
bytes. So Python needs to encode the unicode string using a sequence
of bytes. When stdout points to a terminal (you can check the source
code to see how it figures that out), it can use some environment
variables to figure out what your terminal's encoding is. Your file
however does not have anything stating what encoding you expect it to
be in. So Python refuses to make a guess and just defaults to the
lowest common denominator- ASCII. Since you have a byte that can't be
encoded as ASCII, Python gives up and throws that error.

To avoid this, you'll need to explicitly set the encoding of what you
want to print out. For instance, add .encode("utf-8") or
.encode("cp1252") or whatever encoding you want to use to the end of
the mytemplate.render_unicode(...) call on the last line.

> --
> "Now the storm has passed over me
> I'm left to drift on a dead calm sea
> And watch her forever through the cracks in the beams
> Nailed across the doorways of the bedrooms of my dreams"
> --
> http://mail.python.org/mailman/listinfo/python-list
>