[Tutor] need help in how to make my script read arabic lang

[enas khalil]

hello ,
  i want to know if yu please how can i use python code in tagging arabic text file 
  my code is as follow :
  # -*- coding: cp1256 -*-
import codecs
from nltk.tagger import *
from nltk.corpus import brown
from nltk.tokenizer import WhitespaceTokenizer
from nltk import *
from nltk.tokenreader.tagged import TaggedTokenReader
  # Tokenize ten texts from the Brown Corpus
train_tokens = []
  text_str = (open('fataha2.txt').read())
#codecs.encode(text_str,'cp1256')
reader = TaggedTokenReader(SUBTOKENS='WORDS')
text_token = reader.read_token(text_str)
print text_token['WORDS']
  for l in text_token['WORDS']:
    train_tokens.append(l)
   
  #Initialise and train a unigram tagger
mytagger = UnigramTagger(SUBTOKENS='WORDS')
  for xx in train_tokens:
    cc = reader.read_token(xx['TEXT'])
   # print cc.keys()
    cc['SUBTOKENS']= cc['WORDS']
    
    mytagger.train(cc)
    
#Once a UnigramTagger has been trained, the tag() method can be used to tag new text: 
text_token = Token(TEXT="ÇáÍãÏ ááå ÑÈ ÇáÚÇáãíä")
WhitespaceTokenizer(SUBTOKENS='WORDS').tokenize(text_token)
mytagger.tag(text_token)
#print 'The first example : Using Unigram Tagger the reseults are : '
print
print text_token
   
  and i  got the following error :
   
  Traceback (most recent call last):
  File "I:/examples/unigramgtag1update1.py", line 13, in ?
    codecs.encode(text_str,'cp1256')
  File "C:\Python24\lib\encodings\cp1256.py", line 18, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc8 in position 0: ordinal not in range(128)
   
   please help

		
---------------------------------
 Yahoo! FareChase - Search multiple travel sites in one click.  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/tutor/attachments/20051120/1e29fceb/attachment.htm