[Baypiggies] urllib2, basic HTTP authentication, and tr.im

Stephen Cattaneo stephen.cattaneo at gmail.com
Thu Mar 12 00:05:26 CET 2009


As Damon suggested; get wireshark (formerly called ethereal) it is typically
the best place to start when debugging weird network issues.

cheers,

S

On Wed, Mar 11, 2009 at 3:37 PM, Damon McCormick <damonmc at gmail.com> wrote:

> I've used a  tool called Charles <http://www.charlesproxy.com/> for http,
> and Wireshark <http://www.wireshark.org/> is the standard for general
> protocol analysis.  Both run on OS X.
>
> -Damon
>
>
> On Wed, Mar 11, 2009 at 3:14 PM, Daryl Spitzer <daryl.spitzer at gmail.com>wrote:
>
>> Thanks Benjamin.  I should have wrote sooner to add that I've
>> confirmed that curl -u works.  So the problem is definitely with my
>> Python basic HTTP authentication code.
>>
>> I also should have explained that I want to use urllib2 because (as
>> Michael Foord wrote in
>> http://www.voidspace.org.uk/python/articles/urllib2.shtml#proxies),
>> urllib2 will auto-detect proxy settings and use them; curl--as far as
>> I know--doesn't auto-detect proxy settings.  (So my code using query
>> string parameters works fine at work through a proxy server and at
>> home.)
>>
>> Also, I'd just like to learn why my code doesn't work!
>>
>> I guess I should take jj's advice and look at what is being sent over
>> the wire.  (And compare that to what's going over the wire with curl.)
>>  I've never done that before.  Can anyone recommend a tool for that
>> (preferably that works on Mac OS X)?
>>
>> --
>> Daryl
>>
>>
>> On Wed, Mar 11, 2009 at 2:07 PM, Benjamin Sergeant <bsergean at gmail.com>
>> wrote:
>> > Warning: quick and dirty solution with no error checking using curl and
>> a
>> > unix pipe.
>> >
>> > - Benjamin
>> >
>> > [bsergean at localhost bin]$ cat trim.py
>> > #!/usr/bin/env python
>> >
>> > import os, sys
>> > from getpass import getpass
>> > import xml.dom.minidom
>> >
>> > if len(sys.argv) != 2:
>> >     print 'Usage: trim.py <url to shorten>'
>> >     sys.exit(0)
>> >
>> > user = raw_input('username: ')
>> > passwd = getpass('password: ')
>> >
>> > # See http://tr.im/api
>> > url = 'http://api.tr.im/api/trim_url.xml?url=%s&newtrim=yes' %
>> sys.argv[1]
>> >
>> > # Warning: --silent don't show error messages
>> > cmd  = 'curl --silent --basic -u %s:%s ' % (user, passwd)
>> > cmd += url
>> >
>> > fd = os.popen(cmd, 'r')
>> > response = fd.read()
>> > print response
>> > fd.close()
>> >
>> > Doc = xml.dom.minidom.parseString(response)
>> > N = Doc.getElementsByTagName("url")
>> > shortened_url = str((N[0].childNodes)[0].data)
>> >
>> > print 'shortened url:', shortened_url
>> >
>> >
>> > [bsergean at localhost bin]$ python trim.py
>> > Usage: trim.py <url to shorten>
>> > [bsergean at localhost bin]$ python trim.py
>> > http://www.reddit.com/r/programming/
>> > username: bsergean
>> > password:
>> > <?xml version="1.0" encoding="UTF-8"?>
>> > <trim>
>> >   <status result="OK" code="200" message="tr.im URL Added."/>
>> >   <url>http://tr.im/hglC</url>
>> >   <reference>ZfCAq7qcyouYXwfR2WsuBR6v3V4q7L</reference>
>> >   <trimpath>hglC</trimpath>
>> > </trim>
>> >
>> > shortened url: http://tr.im/hglC
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Tue, Mar 10, 2009 at 11:51 PM, Shannon -jj Behrens <jjinux at gmail.com
>> >
>> > wrote:
>> >>
>> >> It may help to look at what is being sent over the wire.
>> >>
>> >> -jj
>> >>
>> >> On Tue, Mar 10, 2009 at 8:26 AM, Daryl Spitzer <
>> daryl.spitzer at gmail.com>
>> >> wrote:
>> >> > I'm playing around, trying to write some code to use the
>> http://tr.im
>> >> > APIs (http://tr.im/api/) to shorten a URL.
>> >> >
>> >> > After reading http://docs.python.org/library/urllib2.html, I tried:
>> >> >
>> >> >    TRIM_API_URL = 'http://api.tr.im/api'
>> >> >    auth_handler = urllib2.HTTPBasicAuthHandler()
>> >> >    auth_handler.add_password(realm='tr.im',
>> >> >                              uri=TRIM_API_URL,
>> >> >                              user=USERNAME,
>> >> >                              passwd=PASSWORD)
>> >> >    opener = urllib2.build_opener(auth_handler)
>> >> >    urllib2.install_opener(opener)
>> >> >    response = urllib2.urlopen('%s/trim_simple?url=%s'
>> >> >                               % (TRIM_API_URL, url_to_trim))
>> >> >    url = response.read().strip()
>> >> >
>> >> > response.code is 200 (I think it should be 202).  url is valid, but
>> >> > the basic HTTP authentication doesn't seem to have worked, because
>> the
>> >> > shortened URL isn't in my list of URLs (at http://tr.im/?page=1).
>> >> >
>> >> > After reading
>> >> >
>> http://www.voidspace.org.uk/python/articles/authentication.shtml#doing-it-properly
>> >> > I also tried:
>> >> >
>> >> >    TRIM_API_URL = 'api.tr.im/api'
>> >> >    password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
>> >> >    password_mgr.add_password(None, TRIM_API_URL, USERNAME, PASSWORD)
>> >> >    auth_handler = urllib2.HTTPBasicAuthHandler(password_mgr)
>> >> >    opener = urllib2.build_opener(auth_handler)
>> >> >    urllib2.install_opener(opener)
>> >> >    response = urllib2.urlopen('http://%s/trim_simple?url=%s'
>> >> >                               % (TRIM_API_URL, url_to_trim))
>> >> >    url = response.read().strip()
>> >> >
>> >> > But I get the same results.  (response.code is 200 and url is valid,
>> >> > but not recorded in my account at http://tr.im/.)
>> >> >
>> >> > If I use query string parameters instead of basic HTTP
>> authentication,
>> >> > like this:
>> >> >
>> >> >    TRIM_API_URL = 'http://api.tr.im/api'
>> >> >    response =
>> >> > urllib2.urlopen('%s/trim_simple?url=%s&username=%s&password=%s'
>> >> >                               % (TRIM_API_URL,
>> >> >                                  url_to_trim,
>> >> >                                  USERNAME,
>> >> >                                  PASSWORD))
>> >> >    url = response.read().strip()
>> >> >
>> >> > ...then not only is url valid but it's recorded in my tr.im account.
>> >> > (Though response.code is still 200.)
>> >> >
>> >> > Either there must be something wrong in my authentication code, or in
>> >> > tr.im's.  (I've written api at tr.im to see if it's a known problem and
>> >> > haven't received a response.)
>> >> >
>> >> > Can anyone spot any problems?
>> >> >
>> >> > --
>> >> > Daryl
>> >> > _______________________________________________
>> >> > Baypiggies mailing list
>> >> > Baypiggies at python.org
>> >> > To change your subscription options or unsubscribe:
>> >> > http://mail.python.org/mailman/listinfo/baypiggies
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> In this life we cannot do great things. We can only do small things
>> >> with great love. -- Mother Teresa
>> >> http://jjinux.blogspot.com/
>> >> _______________________________________________
>> >> Baypiggies mailing list
>> >> Baypiggies at python.org
>> >> To change your subscription options or unsubscribe:
>> >> http://mail.python.org/mailman/listinfo/baypiggies
>> >
>> >
>> _______________________________________________
>> Baypiggies mailing list
>> Baypiggies at python.org
>> To change your subscription options or unsubscribe:
>> http://mail.python.org/mailman/listinfo/baypiggies
>>
>
>
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies
>



-- 
---
Failures are finger posts on the road to achievement.

-- C.S. Lewis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20090311/5396a8e0/attachment.htm>


More information about the Baypiggies mailing list