Problem archiving Japanese messages
![](https://secure.gravatar.com/avatar/3776a851667c7496df1dfc9dab66623d.jpg?s=120&d=mm&r=g)
Hello, I have started to use Mailman v1.1 with exim on Debian GNU/Linux 2.1.
On an archived message from our Japanese mail messages must be converted into the Extended Unix Code (EUC) from the ordinary transporting Japanese text (JIS). Indeed I have had some problems to archive my test message in Japanese.
So I have tried to insert a filter which convert JIS to EUC before post by the wrapper. And Mailman works fine.
One Japanese charater needs two 7 bit code within the special quotation in JIS, and I understood that scanning JIS code to convert to HTML text may mistake. But in EUC one character needs two 8 bit code.
But the posted message from this test mailing list have delivered in EUC. The Japanese mail messages must be written in JIS.
I need to know where and/or how I insert this filter(*) in the Mailman. Probably it will be before scanning the message to convert to HTML text for archiving the posted message.
(*)The filter named nkf can also convert mime headers
Please give me any suggestions.
Thanks in advance.
-- Akira Tagata <tagata@rand.co.jp> R&D Computer Co., Ltd. / Tokyo
![](https://secure.gravatar.com/avatar/676065518e0b6162b1861c16535139d8.jpg?s=120&d=mm&r=g)
In /home/mailman/Mailman/mm_cfg.py I use
PUBLIC_EXTERNAL_ARCHIVER = '(/usr/local/bin/nkf -e |/usr/bin/mhonarc -rcfile /us r/lib/perl5/site_perl/5.005/mhonarc.rc -add -umask 022 -outdir /home/mailman/arc hives/public/%(listname)s -multipg -expireage 2592000 -idxsize 15 -reverse -id xfname index.html)'
Oskar
Tue, 30 May 2000 17:07:56 +0900 Akira Tagata <tagata@rand.co.jp> said: <snip>
I need to know where and/or how I insert this filter(*) in the Mailman. <snip>
![](https://secure.gravatar.com/avatar/3776a851667c7496df1dfc9dab66623d.jpg?s=120&d=mm&r=g)
Date: Tue, 30 May 2000 17:52:04 +0900 From: Oskar Bartenstein <oskar@ifcomputer.co.jp>
Thanks for your suggestion.
I have modified Archiver.py to invoke nkf from Mailman before the internal archiver process as follows. Your hint was so valuable.
| $ diff Mailman/Archiver/Archiver.py Mailman/Archiver/Archiver.py.20000531 | 233,241d232 | < ########## nkf before internal archiver ########## | < import popen2 | < (nkfout, nkfin) = popen2.popen2('/usr/bin/nkf -me') | < nkfin.write(txt) | < nkfin.close() | < tmptxt = nkfout.read() | < nkfout.close() | < txt = tmptxt | < ########## nkf before internal archiver ##########
The original code is from Mailman v1.1 on the Debian GNU/Linux 2.1 distribution.
This may not be a good manner because I am only a newbie for Python.
-- Akira Tagata <tagata@rand.co.jp> R&D Computer Co., Ltd. / Tokyo
![](https://secure.gravatar.com/avatar/3776a851667c7496df1dfc9dab66623d.jpg?s=120&d=mm&r=g)
I have just rewritten the process:
| $ diff Mailman/Archiver/Archiver.py Mailman/Archiver/Archiver.py.20000531 | 233,244d232 | < ########## nkf before internal archiver ########## | < import tempfile | < outputfile=tempfile.mktemp() | < filter = '/usr/bin/nkf -me >' + outputfile | < tofilter=os.popen(filter, 'w') | < tofilter.write(txt) | < tofilter.close() | < fromfilter=open(outputfile) | < txt=fromfilter.read() | < fromfilter.close() | < os.remove(outputfile) | < ########## nkf before internal archiver ##########
The original code (Archiver.py.20000531) is from Mailman v1.1 on the Debian GNU/Linux 2.1 distribution.
This is so ugly....
[snip]
[snip]
A large text process was blocked.
[snip]
-- Akira Tagata <tagata@rand.co.jp> R&D Computer Co., Ltd. / Tokyo
![](https://secure.gravatar.com/avatar/676065518e0b6162b1861c16535139d8.jpg?s=120&d=mm&r=g)
In /home/mailman/Mailman/mm_cfg.py I use
PUBLIC_EXTERNAL_ARCHIVER = '(/usr/local/bin/nkf -e |/usr/bin/mhonarc -rcfile /us r/lib/perl5/site_perl/5.005/mhonarc.rc -add -umask 022 -outdir /home/mailman/arc hives/public/%(listname)s -multipg -expireage 2592000 -idxsize 15 -reverse -id xfname index.html)'
Oskar
Tue, 30 May 2000 17:07:56 +0900 Akira Tagata <tagata@rand.co.jp> said: <snip>
I need to know where and/or how I insert this filter(*) in the Mailman. <snip>
![](https://secure.gravatar.com/avatar/3776a851667c7496df1dfc9dab66623d.jpg?s=120&d=mm&r=g)
Date: Tue, 30 May 2000 17:52:04 +0900 From: Oskar Bartenstein <oskar@ifcomputer.co.jp>
Thanks for your suggestion.
I have modified Archiver.py to invoke nkf from Mailman before the internal archiver process as follows. Your hint was so valuable.
| $ diff Mailman/Archiver/Archiver.py Mailman/Archiver/Archiver.py.20000531 | 233,241d232 | < ########## nkf before internal archiver ########## | < import popen2 | < (nkfout, nkfin) = popen2.popen2('/usr/bin/nkf -me') | < nkfin.write(txt) | < nkfin.close() | < tmptxt = nkfout.read() | < nkfout.close() | < txt = tmptxt | < ########## nkf before internal archiver ##########
The original code is from Mailman v1.1 on the Debian GNU/Linux 2.1 distribution.
This may not be a good manner because I am only a newbie for Python.
-- Akira Tagata <tagata@rand.co.jp> R&D Computer Co., Ltd. / Tokyo
![](https://secure.gravatar.com/avatar/3776a851667c7496df1dfc9dab66623d.jpg?s=120&d=mm&r=g)
I have just rewritten the process:
| $ diff Mailman/Archiver/Archiver.py Mailman/Archiver/Archiver.py.20000531 | 233,244d232 | < ########## nkf before internal archiver ########## | < import tempfile | < outputfile=tempfile.mktemp() | < filter = '/usr/bin/nkf -me >' + outputfile | < tofilter=os.popen(filter, 'w') | < tofilter.write(txt) | < tofilter.close() | < fromfilter=open(outputfile) | < txt=fromfilter.read() | < fromfilter.close() | < os.remove(outputfile) | < ########## nkf before internal archiver ##########
The original code (Archiver.py.20000531) is from Mailman v1.1 on the Debian GNU/Linux 2.1 distribution.
This is so ugly....
[snip]
[snip]
A large text process was blocked.
[snip]
-- Akira Tagata <tagata@rand.co.jp> R&D Computer Co., Ltd. / Tokyo
participants (2)
-
Akira Tagata
-
Oskar Bartenstein