Program works great, except under less, cron or execl (Unicode?)
samslists at gmail.com
Fri Sep 19 07:34:48 CEST 2008
Diez for the win... :)
sys.stdout.encoding does indeed have the proper value when called from
the command line of UTF-8.
But when piped into anything or called from anywhere it's None.
Just for completeness, here's my test program:
And here are the results:
$ ./encoding.py | cat
Really, really annoying!
So how can I set sys.stdout.encoding so it's UTF-8 when piped through
cat (or anything else).
I tried assigning to it, but no dice.
On Sep 18, 2:12 am, "Diez B. Roggisch" <de... at nospam.web.de> wrote:
> Sam wrote:
> > I have a program which works great when run from the command line.
> > But when I run it combined with something else such as:
> > - piping it through less
> > - cron
> > - execl (i.e. calling it from another python program)
> > it gives me a unicode error
> > File "../myparser.py", line 261, in set_attributes
> > print "self.atd['Name'] is: ", self.atd['Name']
> > UnicodeEncodeError: 'ascii' codec can't encode character u'\xeb' in
> > position 7: ordinal not in range(128)
> > I'd post the whole program here, except it involves weird Unicode
> > strings.
> > I could probably change the program to get it working under less/cron/
> > etc.
> > But I'd rather understand exactly what the issue is. Why does it work
> > fine when run directly from the command line, but not otherwise?
> Most probably because when to running directly inside a terminal, it gets
> it's stdin/stdout as pipes - and python can't attempt to guess the proper
> encoding on that, as it does on a terminal.
> And thus, when you print unicode to the pipe, it can't decide which encoding
> to use.
> To circumvene this, try & wrap stdout into a codecs-module wrapper with a
> proper encoding applied (e.g. utf-8).
> You might make that conditionally based on the sys.stdout.encoding-variable
> being set or not, albeit I'm not 100% sure to what it actually gets set
> when used in a subprocess. But this should give you the idea where to look.
More information about the Python-list