[Tutor] Is anybody out there who could help me with URL Authentication?

[Federo]

Hi ..

I have to admit that Python is really surprising me. It was lucky day a few 
weeks ago I firts time start using Python. Lot's of things realy can be done 
with short learning curve. Your user guieds was best place to start!

Below is problem I am unable to solve. I would appreciate your advice or 
even better code sample. The problem is URL authorisation. I try to approaches 
but no luck so far. Two problems:

1.) Being able to logon using Python code
2.) Being able to send data to a form and get back server reply

ad 1.)
The Two pages I try to open:

Login:
https://investor.firstrade.com/firstrade/login.do
Main page after login:
https://investor.firstrade.com/firstrade/mainmenu.do

I have tried bellow code (with currect user name and password of course):
>>> import urllib2
>>> password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
>>> protocol = 'https://'
>>> username = 'user'
>>> password = 'pass'
>>> top_level_url = "investor.firstrade.com/firstrade/login.do"
>>> password_mgr.add_password(None, top_level_url, username, password)
>>> handler = urllib2.HTTPBasicAuthHandler(password_mgr)
>>> opener = urllib2.build_opener(handler)
>>> a_url = "https://investor.firstrade.com/firstrade/login.do"
>>> f = opener.open(a_url)
>>> a = f.read()
>>> print (a)

but it replied login page after login. It looks like they are using cookie. 
Bellow line can be find in header
Set-Cookie: JSESSIONID=351CE048D9FEAD01E21604129D38725C; Path=/; Secure

On the web I found user guide explaining coookie login. However I do not 
understand how to combine now password and cookie login if this will solve the 
problem at first place... 
http://www.voidspace.org.uk/python/articles/cookielib.shtml#fetching-webpages

CODE SAMPLE to be able to solve the problem very appreciated..

There is by pass solution using Web Recorder Macro. However it will be much 
better to BE ABLE TO handle entire process with Python..

ad 2.) Filling data to form / getting beck server values. I need server reply 
from the same web side we are discussing in point 1.

Here is Code sample:
import urllib
import urllib2  

    url = 'https://investor.firstrade.com/firstrade/mainmenu.do'
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'    
    values = {'symbol' : 'xsnx'}

    data = urllib.urlencode(values)
    req = urllib2.Request(url, data)
    response = urllib2.urlopen(req)
    the_page = response.read()

a.) I do not understand where can I find currect fields names for values 
in the above code sample (in our case "Symbol" is written before Text box (look 
attached print screen. HTML code of main side is attached file 
Firstrade_MainSideHTML.rtf) 
where you can write stock ticker and than press a button. Server replied real 
time stock data which I would like to read)?
b.) Probably code will be more complex as sample code above as code has to take 
care also for proper authorisation..

Sorry for taking your time. Your help would be very useful as I stacked with 
this. Being able to handle URL requests is very important now days...

Cheers, Fedo

____________________
http://www.email.si/

____________________
http://www.email.si/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Firstrade_MainSide.jpg
Type: image/pjpeg
Size: 45800 bytes
Desc: Firstrade_MainSide.jpg
URL: <http://mail.python.org/pipermail/tutor/attachments/20080801/3b4ac3cd/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Firstrade_MainSideHTML.rtf
Type: application/msword
Size: 41489 bytes
Desc: Firstrade_MainSideHTML.rtf
URL: <http://mail.python.org/pipermail/tutor/attachments/20080801/3b4ac3cd/attachment-0001.wiz>