Pickle in a POST/GET request give EOFError
Romaric DEFAUX
rde at audaxis.com
Thu Nov 18 05:22:30 EST 2010
Hi again,
I try simplejson, but I have a problem. (I just replace pickle
everywhere by simplejson)
Here's a test I did :
# python
Python 2.5.2 (r252:60911, Jan 20 2010, 21:48:48)
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import simplejson
>>> disk_list = []
>>> disk = {}
>>> disk['name'] = "disk1"
>>> disk['size'] = "52428800"
>>> disk_list.append(disk)
>>> print disk_list
[{'name': 'disk1', 'size': '52428800'}]
>>> disk = {}
>>> disk['name'] = "disk2"
>>> disk['size'] = "647648"
>>> disk_list.append(disk)
>>> print disk_list
[{'name': 'disk1', 'size': '52428800'}, {'name': 'disk2', 'size': '647648'}]
>>> disk_list_json = simplejson.dumps(disk_list)
>>> print disk_list_json
[{"name": "disk1", "size": "52428800"}, {"name": "disk2", "size": "647648"}]
>>> disk_list2 = simplejson.loads(disk_list_json)
>>> print disk_list2
[{u'name': u'disk1', u'size': u'52428800'}, {u'name': u'disk2', u'size':
u'647648'}]
>>> print repr(disk_list_json)
'[{"name": "disk1", "size": "52428800"}, {"name": "disk2", "size":
"647648"}]'
Explanation :
I "dumps" with json a list of dictionnaries into disk_list_json.
When I "loads" it, I don't get my list of dictionnaries like before (see
disk_list2). It adds "u" letter everywhere.
Why ?
Thanks for help !
Romaric Defaux
Le 18/11/2010 09:43, Michael Ricordeau a écrit :
> Hi,
>
> you can use json for passing list and dict .
> Pickle is dangerous .
>
> Instead of pickle.loads/pickle.dumps use json.loads and json.dumps
> (using stdlib json in python>= 2.6 or simplejson in python< 2.6)
>
> Regards
>
>
>
> Le Thu, 18 Nov 2010 09:29:00 +0100,
> Romaric DEFAUX<rde at audaxis.com> a écrit :
>
>
>> Le 17/11/2010 18:52, geremy condra a écrit :
>>
>>> On Wed, Nov 17, 2010 at 6:44 AM, Romaric DEFAUX<rde at audaxis.com> wrote:
>>>
>>>
>>>> Le 16/11/2010 17:47, Romaric DEFAUX a écrit :
>>>>
>>>>
>>>>> Hi everybody !
>>>>>
>>>>> First time I write to this mailing list :)
>>>>> I started writing in python last week, that's probably why I can't
>>>>> understand the following problem...
>>>>>
>>>>>
>>>>> I create a list called web_site_list.
>>>>> This list contain dictionaries called web_site.
>>>>> And some values in this dictionaries are list too.
>>>>>
>>>>> I do that in a function and I return this :
>>>>> return pickle.dumps(web_site_list)
>>>>>
>>>>> This is working fine :)
>>>>>
>>>>> If I do :
>>>>> print "%s" % pickle.loads(system.get_web_site_list())
>>>>>
>>>>> I've got the right stuffs. For example it returns :
>>>>> [{'documentroot_size': '120', 'servername': '---default---', 'client':
>>>>> 'undefined', 'documentroot': '/var/www/', 'client_contact': 'undefined',
>>>>> 'serveralias': []}]
>>>>>
>>>>> I send this to a web service. I send it like that :
>>>>> #I put it in params
>>>>> def system_updateweb_site(server, login, password):
>>>>> params = {}
>>>>> params['login'] = login
>>>>> params['password'] = password
>>>>> params['action'] = 'updateweb_site'
>>>>> params['servername'] = get_servername()
>>>>> params['hosted_web_site'] = get_web_site_list()
>>>>> return call_system_ws(server, params)
>>>>>
>>>>> #Here's how I send it (I tried in GET and POST)
>>>>> def call_system_ws(host, params):
>>>>> query_string = urllib.urlencode(params)
>>>>> #GET
>>>>> # f = urllib.urlopen("http://%s/ws?%s" % (host, query_string))
>>>>> #POST
>>>>> f = urllib.urlopen("http://%s/ws" % (host), query_string)
>>>>> result = f.readline().strip()
>>>>> if result == 'ERROR':
>>>>> msg = f.readline().strip()
>>>>> return (False, msg)
>>>>> return (True, result)
>>>>>
>>>>>
>>>>> On the server side :
>>>>> if action == 'updateweb_site':
>>>>> if not (fields.has_key('servername') and
>>>>> fields.has_key('hosted_web_site')):
>>>>> raise WSError('missing parameter :
>>>>> servername or hosted_web_site')
>>>>> log ('ERROR : missing parameter :
>>>>> servername or hosted_web_site')
>>>>> else:
>>>>>
>>>>> servername=g.db.escape_string(fields['servername'])
>>>>>
>>>>> hosted_web_site=g.db.escape_string(fields['hosted_web_site'])
>>>>> output =
>>>>> systemserver.updateweb_site(cursor, servername, hosted_web_site)
>>>>>
>>>>> In systemserver.py :
>>>>> def updateweb_site(cursor, host, hosted_web_site):
>>>>> web_site_list = pickle.loads(hosted_web_site)
>>>>> return "%s" % (web_site_list)
>>>>>
>>>>> I catch this error :*
>>>>>
>>>>> <type 'exceptions.EOFError'>*:
>>>>>
>>>>> args = ()
>>>>> message = ''
>>>>>
>>>>> Why ?
>>>>>
>>>>> If I just print hosted_web_site, I get this on my web page :
>>>>>
>>>>>
>>>>> (lp0\n(dp1\nS\'documentroot_size\'\np2\nS\'120\'\np3\nsS\'servername\'\np4\nS\'default\'\np5\nsS\'client\'\np6\nS\'undefined\'\np7\nsS\'documentroot\'\np8\nS\'/var/www/\'\np9\nsS\'client_contact\'\np10\ng7\nsS\'serveralias\'\np11\n(lp12\nsa.
>>>>>
>>>>> It's the "pickled view" of
>>>>> [{'documentroot_size': '120', 'servername': '---default---', 'client':
>>>>> 'undefined', 'documentroot': '/var/www/', 'client_contact': 'undefined',
>>>>> 'serveralias': []}]
>>>>>
>>>>> Can someone help me please ? I spend my afternoon to google to try to find
>>>>> a solution...
>>>>>
>>>>>
>>>>> Thanks in advance !!!
>>>>>
>>>>> Romaric Defaux
>>>>>
>>>>>
>>>>>
>>>> After entirely rewrite my code to not use Web service but socket (a real
>>>> client/server program) I finally found the problem... And it's not linked to
>>>> the POST or GET method...
>>>> It's because of that :
>>>> g.db.escape_string(fields['hosted_web_site'])
>>>> (escape_string is the function in MySQLdb library)
>>>> It escapes the simple quote of the pickled object, and break it...
>>>>
>>>> It's good to know, NEVER escape a pickled object :)
>>>>
>>>> Romaric Defaux
>>>>
>>>>
>>> I'm not sure I understand what you're doing here, but I trust you've
>>> read about and understand the security problems with pickle?
>>>
>>> Geremy Condra
>>>
>>>
>> I read quickly the security problems with pickle. But I don't feel
>> concern about that because I run my program in a private network, not
>> over internet. And now I use socket to communicate on a non-standard
>> port, not anymore web service on the 80 port. If I plan to run it
>> through wan, I will encrypt datas for sure with SSL or something like
>> that :)
>>
>> Romaric Defaux
>>
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5361 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20101118/8ab93537/attachment.bin>
More information about the Python-list
mailing list