From szybalski at gmail.com Wed Sep 2 00:04:09 2009 From: szybalski at gmail.com (Lukasz Szybalski) Date: Tue, 1 Sep 2009 17:04:09 -0500 Subject: [Chicago] [RFC]ootools with pyuno In-Reply-To: <804e5c70909011229g7bcad119pc76ebfcae582c3a9@mail.gmail.com> References: <804e5c70909011229g7bcad119pc76ebfcae582c3a9@mail.gmail.com> Message-ID: <804e5c70909011504r6e57a785w1957b7f118b5e035@mail.gmail.com> Hello, Based on some work that was done by http://www.linuxjournal.com/content/starting-stopping-and-connecting-openoffice-python I've created a ootools package that allows you to start openoffice in just 2 lines of python code. It requires openoffice -headless and oo2.4+. Install it and give it a try. ---How to install it --- easy_install ootools or sudo easy_install ootools -- How to use it -- python import ootools oor=ootools.OORunner() oor.start() # open office headless should start on port 8100. #Check using command: netstat -atpen | grep soffice #To stop it do: oor.stop() -- How to get desktop object -- import ootools oor=ootools.OORunner() oor.start() desktop=oor.connect() #Do something with the desktop. #When done oor.close() Give it a try and let me know. It works correctly under Debian lenny 32bit. What os you have system, path to open office,etc. Thanks, Lucas -- Using rsync. How to setup rsyncd. http://lucasmanual.com/mywiki/rsync OpenLdap - From start to finish. http://lucasmanual.com/mywiki/OpenLdap From bray at sent.com Sat Sep 5 01:15:53 2009 From: bray at sent.com (Brian Ray) Date: Fri, 4 Sep 2009 18:15:53 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting Message-ID: Good News, we have secured a great new venue for this month's meeting... Tribune Tower. No details yet; however, this really may be the best venue ever. Yay!!! Topics, Topics, Topics ??? Should we tackle something related to journalism? Should we live channel to/from Djangocon... so many questions, yet such a great meeting this will be. Brian Ray From ccf3 at mindspring.com Sat Sep 5 03:22:35 2009 From: ccf3 at mindspring.com (Clyde Forrester) Date: Fri, 04 Sep 2009 20:22:35 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: Message-ID: <4AA1BD5B.5060202@mindspring.com> I've got my "Hello, and Beyond" talk ready to go. It covers simple programs in a number of languages. The idea is to cover: 1. How to get started. 2. Some early lessons. 3. How languages compare. and 4. Some other programming fundamentals. It will be an ongoing project for me, but it has grown past the minimum amount of stuff I need to be able to say that it's ready to go. Clyde Forrester Brian Ray wrote: > Good News, we have secured a great new venue for this month's > meeting... Tribune Tower. No details yet; however, this really may be > the best venue ever. Yay!!! > > > Topics, Topics, Topics ??? Should we tackle something related to > journalism? Should we live channel to/from Djangocon... so many > questions, yet such a great meeting this will be. > > > Brian Ray From dgriff1 at gmail.com Sat Sep 5 04:01:01 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Fri, 4 Sep 2009 21:01:01 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: <4AA1BD5B.5060202@mindspring.com> References: <4AA1BD5B.5060202@mindspring.com> Message-ID: <3db160680909041901q7055a1bao39329d7ff171680b@mail.gmail.com> I brought it up a few months back but I have been working out a presentation on Somewhat Advanced SQLAlchemy that I could do. Dan On Fri, Sep 4, 2009 at 8:22 PM, Clyde Forrester wrote: > I've got my "Hello, and Beyond" talk ready to go. It covers simple programs > in a number of languages. The idea is to cover: 1. How to get started. 2. > Some early lessons. 3. How languages compare. and 4. Some other programming > fundamentals. > > It will be an ongoing project for me, but it has grown past the minimum > amount of stuff I need to be able to say that it's ready to go. > > Clyde Forrester > > > Brian Ray wrote: > >> Good News, we have secured a great new venue for this month's meeting... >> Tribune Tower. No details yet; however, this really may be the best venue >> ever. Yay!!! >> >> >> Topics, Topics, Topics ??? Should we tackle something related to >> journalism? Should we live channel to/from Djangocon... so many questions, >> yet such a great meeting this will be. >> >> >> Brian Ray >> > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shekay at pobox.com Mon Sep 7 16:45:10 2009 From: shekay at pobox.com (sheila miguez) Date: Mon, 7 Sep 2009 09:45:10 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: Message-ID: On Fri, Sep 4, 2009 at 6:15 PM, Brian Ray wrote: > Good News, we have secured a great new venue for this month's meeting... > Tribune Tower. ?No details yet; however, this really may be the best venue > ever. Yay!!! > > > Topics, Topics, Topics ??? ?Should we tackle something related to > journalism? Should we live channel to/from Djangocon... so many questions, > yet such a great meeting this will be. Did Carl already make arrangements for someone to run video? If not, do we have anyone who is familiar with setting up the video equipment? Carl's at djangocon this week. I'll contact some of the people who helped with pycon but don't regularly attend chipy to see if they can help. I don't have a car, but could rent one. I know there is some equipment held at Thoughtworks, but I don't know who to contact. Cosmin did, but he's traveling right now. I've got pager duty this week, so if something critical happens at work, I can't really help much. -- sheila From carl at personnelware.com Mon Sep 7 17:30:20 2009 From: carl at personnelware.com (Carl Karsten) Date: Mon, 7 Sep 2009 10:30:20 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: Message-ID: <549053140909070830g2f9f6aebs2bea2673c84d5bd2@mail.gmail.com> On Mon, Sep 7, 2009 at 9:45 AM, sheila miguez wrote: > On Fri, Sep 4, 2009 at 6:15 PM, Brian Ray wrote: >> Good News, we have secured a great new venue for this month's meeting... >> Tribune Tower. ?No details yet; however, this really may be the best venue >> ever. Yay!!! >> >> >> Topics, Topics, Topics ??? ?Should we tackle something related to >> journalism? Should we live channel to/from Djangocon... so many questions, >> yet such a great meeting this will be. > > Did Carl already make arrangements for someone to run video? I did not. -- Carl K From mdipierro at cs.depaul.edu Mon Sep 7 17:40:33 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Mon, 7 Sep 2009 10:40:33 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: <549053140909070830g2f9f6aebs2bea2673c84d5bd2@mail.gmail.com> References: <549053140909070830g2f9f6aebs2bea2673c84d5bd2@mail.gmail.com> Message-ID: <19BB8F8C-D0E4-4FE3-988D-DDEC749EFB81@cs.depaul.edu> what's on the agenda? About the previous meeting. Is there a recording of the talk by Dr. John Hunter? John, are you a member of this mailing list? Massimo On Sep 7, 2009, at 10:30 AM, Carl Karsten wrote: > On Mon, Sep 7, 2009 at 9:45 AM, sheila miguez wrote: >> On Fri, Sep 4, 2009 at 6:15 PM, Brian Ray wrote: >>> Good News, we have secured a great new venue for this month's >>> meeting... >>> Tribune Tower. No details yet; however, this really may be the >>> best venue >>> ever. Yay!!! >>> >>> >>> Topics, Topics, Topics ??? Should we tackle something related to >>> journalism? Should we live channel to/from Djangocon... so many >>> questions, >>> yet such a great meeting this will be. >> >> Did Carl already make arrangements for someone to run video? > > I did not. > > > -- > Carl K > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From shekay at pobox.com Mon Sep 7 17:55:21 2009 From: shekay at pobox.com (sheila miguez) Date: Mon, 7 Sep 2009 10:55:21 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: <19BB8F8C-D0E4-4FE3-988D-DDEC749EFB81@cs.depaul.edu> References: <549053140909070830g2f9f6aebs2bea2673c84d5bd2@mail.gmail.com> <19BB8F8C-D0E4-4FE3-988D-DDEC749EFB81@cs.depaul.edu> Message-ID: On Mon, Sep 7, 2009 at 10:40 AM, Massimo Di Pierro wrote: > what's on the agenda? > > About the previous meeting. Is there a recording of the talk by Dr. John > Hunter? John, are you a member of this mailing list? yes, but it did not convert to flash, so it is an ogg file. which means blip will want to run an applet to view it. You can download hte ogg file if you don't want to do that. it's at Carl's page. http://carlfk.blip.tv So is the talk on bcfg2. -- sheila From joe at germuska.com Mon Sep 7 19:19:39 2009 From: joe at germuska.com (Joe Germuska) Date: Mon, 7 Sep 2009 12:19:39 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: Message-ID: On Sep 4, 2009, at 6:15 PM, Brian Ray wrote: > Topics, Topics, Topics ??? Should we tackle something related to > journalism? Should we live channel to/from Djangocon... so many > questions, yet such a great meeting this will be. Looks like there may be enough stuff, but since we are hosting, I could show my work-in-progress porting the Perl Geo::StreetAddress::US library to Python. I would guess it would only be 15 minutes or less, but it might be of interest. The Perl module uses some pretty arcane RegExp magic. And maybe someone will see what I'm missing in the last three test cases! It may also be time to open the discussion for where to take the after- event. The Billy Goat is obvious (and with merit) but I might also suggest CND Gyros & Lounge, which has the wood paneling to make the Resi's fans feel right at home. Joe -- Joe Germuska Joe at Germuska.com * http://blog.germuska.com "Participation. That's what's gonna save the human race." --Pete Seeger From smv260985 at gmail.com Mon Sep 7 19:40:22 2009 From: smv260985 at gmail.com (Subodhini) Date: Mon, 7 Sep 2009 10:40:22 -0700 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: Message-ID: Hello Everyone, I am new to this group and I am glad to meet you here on this platform. I am interested to meet you in Tribune Tower. Could you guys put any light on what is the entry fee? Do I need to pay kind of a charge to attend this meeting? I would be pleased, if you guys provide me this information. Thanks & Regards, Subodhini Chopde On Mon, Sep 7, 2009 at 10:19 AM, Joe Germuska wrote: > On Sep 4, 2009, at 6:15 PM, Brian Ray wrote: > >> Topics, Topics, Topics ??? Should we tackle something related to >> journalism? Should we live channel to/from Djangocon... so many questions, >> yet such a great meeting this will be. >> > > > Looks like there may be enough stuff, but since we are hosting, I could > show my work-in-progress porting the Perl Geo::StreetAddress::US library to > Python. I would guess it would only be 15 minutes or less, but it might be > of interest. The Perl module uses some pretty arcane RegExp magic. And > maybe someone will see what I'm missing in the last three test cases! > > It may also be time to open the discussion for where to take the > after-event. The Billy Goat is obvious (and with merit) but I might also > suggest CND Gyros & Lounge, which has the wood paneling to make the Resi's > fans feel right at home. > > Joe > -- > Joe Germuska > Joe at Germuska.com * http://blog.germuska.com > > "Participation. That's what's gonna save the human race." --Pete Seeger > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -- Subodhini Chopde -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdipierro at cs.depaul.edu Mon Sep 7 20:45:03 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Mon, 7 Sep 2009 13:45:03 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: <549053140909070830g2f9f6aebs2bea2673c84d5bd2@mail.gmail.com> <19BB8F8C-D0E4-4FE3-988D-DDEC749EFB81@cs.depaul.edu> Message-ID: thanks. Massimo On Sep 7, 2009, at 10:55 AM, sheila miguez wrote: > On Mon, Sep 7, 2009 at 10:40 AM, Massimo Di > Pierro wrote: >> what's on the agenda? >> >> About the previous meeting. Is there a recording of the talk by Dr. >> John >> Hunter? John, are you a member of this mailing list? > > yes, but it did not convert to flash, so it is an ogg file. which > means blip will want to run an applet to view it. You can download hte > ogg file if you don't want to do that. > > it's at Carl's page. http://carlfk.blip.tv > > So is the talk on bcfg2. > > > -- > sheila > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From robkapteyn at gmail.com Mon Sep 7 21:11:59 2009 From: robkapteyn at gmail.com (Rob Kapteyn) Date: Mon, 7 Sep 2009 14:11:59 -0500 Subject: [Chicago] Great New Venue for Sept 10th Monthly Meeting In-Reply-To: References: Message-ID: <7D26EA1C-4241-4118-839F-0CAC976DFD05@gmail.com> Subodhini : ChiPy meetings are open to everyone with no charge or membership fee. Hope to see you there on Thursday ! -Rob On Sep 7, 2009, at 12:40 PM, Subodhini wrote: > Hello Everyone, > > I am new to this group and I am glad to meet you here on this > platform. I am interested to meet you in Tribune Tower. Could you > guys put any light on what is the entry fee? Do I need to pay kind > of a charge to attend this meeting? > > I would be pleased, if you guys provide me this information. > > Thanks & Regards, > Subodhini Chopde > > On Mon, Sep 7, 2009 at 10:19 AM, Joe Germuska > wrote: > On Sep 4, 2009, at 6:15 PM, Brian Ray wrote: > Topics, Topics, Topics ??? Should we tackle something related to > journalism? Should we live channel to/from Djangocon... so many > questions, yet such a great meeting this will be. > > > Looks like there may be enough stuff, but since we are hosting, I > could show my work-in-progress porting the Perl > Geo::StreetAddress::US library to Python. I would guess it would > only be 15 minutes or less, but it might be of interest. The Perl > module uses some pretty arcane RegExp magic. And maybe someone will > see what I'm missing in the last three test cases! > > It may also be time to open the discussion for where to take the > after-event. The Billy Goat is obvious (and with merit) but I might > also suggest CND Gyros & Lounge, which has the wood paneling to make > the Resi's fans feel right at home. > > Joe > -- > Joe Germuska > Joe at Germuska.com * http://blog.germuska.com > > "Participation. That's what's gonna save the human race." --Pete > Seeger > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > > > -- > Subodhini Chopde > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From bray at sent.com Tue Sep 8 16:21:29 2009 From: bray at sent.com (Brian Ray) Date: Tue, 8 Sep 2009 09:21:29 -0500 Subject: [Chicago] September Meeting Callout; DjangoCon; October In-Reply-To: References: Message-ID: On Aug 25, 2009, at 1:27 PM, Brian Ray wrote: > So, I open up the floor for Non-Django related topics (even neo- > django, if you must) for the September meeting. Likewise, we are > looking for are host for this meeting. So if anyone has an > organization capable of handling teams of rowdy Python People, > please let us know. What about plug computing and AMQP? I think > Garrett and Pete Fein have some pretty exciting stuff on this and > may be ready for September !? I am putting together the topic lists and I noticed we had some topics left from before. Any comments on these? Brian Ray From g at rre.tt Tue Sep 8 16:28:48 2009 From: g at rre.tt (Garrett Smith) Date: Tue, 8 Sep 2009 09:28:48 -0500 Subject: [Chicago] September Meeting Callout; DjangoCon; October In-Reply-To: References: Message-ID: Pencil me in for "Not using AMQP to super charge your Django apps!" I need more than 15 minutes though to properly cover not doing this. On Tue, Sep 8, 2009 at 9:21 AM, Brian Ray wrote: > > On Aug 25, 2009, at 1:27 PM, Brian Ray wrote: > >> So, I open up the floor for Non-Django related topics (even neo-django, if >> you must) for the September meeting. ?Likewise, we are looking for are host >> for this meeting. So if anyone has an organization capable of handling teams >> of rowdy Python People, please let us know. ?What about plug computing and >> AMQP? I think Garrett and Pete Fein have some pretty exciting stuff on this >> and may be ready for September !? > > > I am putting together the topic lists and I noticed we had some topics left > from before. ?Any comments on these? > > Brian Ray > > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From pfein at pobox.com Tue Sep 8 17:06:22 2009 From: pfein at pobox.com (Pete) Date: Tue, 8 Sep 2009 10:06:22 -0500 Subject: [Chicago] September Meeting Callout; DjangoCon; October In-Reply-To: References: Message-ID: <3F2F3D93-5138-4E26-BBFA-6EBF5BAC8A76@pobox.com> On Sep 8, 2009, at 9:21 AM, Brian Ray wrote: > > On Aug 25, 2009, at 1:27 PM, Brian Ray wrote: > >> So, I open up the floor for Non-Django related topics (even neo- >> django, if you must) for the September meeting. Likewise, we are >> looking for are host for this meeting. So if anyone has an >> organization capable of handling teams of rowdy Python People, >> please let us know. What about plug computing and AMQP? I think >> Garrett and Pete Fein have some pretty exciting stuff on this and >> may be ready for September !? > > > I am putting together the topic lists and I noticed we had some > topics left from before. Any comments on these? I can't make it this month. From shekay at pobox.com Tue Sep 8 17:42:29 2009 From: shekay at pobox.com (sheila miguez) Date: Tue, 8 Sep 2009 10:42:29 -0500 Subject: [Chicago] help pick up vid equipment Message-ID: Dudes, Josh has equipment stored for us at Thoughtworks. I need help getting it by tomorrow early afternoon (because I also volunteered it for the Erlang meeting tomorrow at 6, why did I do that?). Anyway, Erlang meeting is here in the Ogilvy transportation center, and Thoughtworks is not to far away. Maybe I can cab it. Or rent a car. Or have someone help. I don't remember how much equipment is there. If it too much for one person to carry; I can make multiple trips. If someone would help me I would be happier. There is a book reading group for Richard Powers, The Echo Maker tonight which I would like to go to. He is one of my favorite authors of all time. If I pick up stuff today it would have to be in time for being able to make it to Lincoln Square by 7:30. If you help, Carl will buy you a beer and food (or not a beer if you don't like it). I would, but I am awkwardly non-talkative* so you wouldn't enjoy me buying you a beer as much as you would Carl, who is very gregarious. * I bought this shirt to help http://imgs.xkcd.com/store/imgs/just_shy_square_0.png -- sheila From bray at sent.com Wed Sep 9 06:02:38 2009 From: bray at sent.com (Brian Ray) Date: Tue, 8 Sep 2009 23:02:38 -0500 Subject: [Chicago] [ANN] Chicago Python User Group ChiPy September Meeting Message-ID: <582AEAED-0E84-4730-8BBC-59ADA8241755@sent.com> Chicago Python User Group ========================= History, Tradition, Technology and Journalism meet this Thursday at one of our nations most historical landmarks, Tribune Tower--right off the Chicago River on the Magnificent Mile. This *new* venue speaks to the movement of journalism into technology and technology into Python. So, this will be one of most interesting venues and will provide a solid platform for the best user group meeting in the history of newspapers. We may even channel DjangoCon going on at the same time on the West coast. Bring a friend. This will is going to be great! RSVP Right NOW to: brian (at) hackerjournalist.net if you even think you might make it. Topics ------ * (30 min) Somewhat Advanced SQLAlchemy - Daniel Griffin * (30 min) Python port of Geo::StreetAddress::US library - Joe Germuska * (30 min) "Not using AMQP to super charge your Django apps!" -- Garrett Smith * (15 min) "Hello, and Beyond" -- Clyde Forrester When ---- Thursday, September 10th, ~7pm Location -------- Tribune Tower, 435 N. Michigan Ave. RSVP to brian (at) hackerjournalist.net Other Meetings -------------- (before) Kinzie Chop House (some plan to gather around 4:30) (after) Billy Goat is obvious (and with merit) OR CND Gyros & Lounge About ChiPy ----------- ChiPy is a group of Chicago Python Programmers, l33t, and n00bs. Meetings are held monthly at various locations around Chicago. Also, ChiPy is a proud sponsor of many Open Source and Educational efforts in Chicago. Stay tuned to the mailing list for more info. ChiPy website: ChiPy Mailing List: ChiPy Announcement *ONLY* Mailing List: Python website: From brian at sixthw.com Thu Sep 10 16:21:00 2009 From: brian at sixthw.com (Brian Boyer) Date: Thu, 10 Sep 2009 09:21:00 -0500 Subject: [Chicago] [ANN] Chicago Python User Group ChiPy September Meeting In-Reply-To: <582AEAED-0E84-4730-8BBC-59ADA8241755@sent.com> References: <582AEAED-0E84-4730-8BBC-59ADA8241755@sent.com> Message-ID: <85ab0bd40909100721x5187b724y494422d4c430c895@mail.gmail.com> Beer, pizza and pop will be served at this evening's ChiPy at the Tribune Tower. If you think you might attend, please RSVP to brian *at* hackerjournalist.net. This will be the best meeting, ever. Hope to see you there! Brian On Tue, Sep 8, 2009 at 11:02 PM, Brian Ray wrote: > Chicago Python User Group > ========================= > > History, Tradition, Technology and Journalism meet this Thursday at one of > our nations most historical landmarks, Tribune Tower--right off the Chicago > River on the Magnificent Mile. ?This *new* venue speaks to the movement of > journalism into technology and technology into Python. > > So, this will be one of most interesting venues and will provide a solid > platform for the best user group meeting in the ?history of newspapers. We > may even channel DjangoCon going on at the same time on the West coast. > > Bring a friend. This will is going to be great! > > RSVP Right NOW to: brian (at) hackerjournalist.net if you even think you > might make it. > > > Topics > ------ > > * (30 min) Somewhat Advanced SQLAlchemy ?- Daniel Griffin > * (30 min) ?Python port of Geo::StreetAddress::US library - Joe Germuska > * (30 min) ?"Not using AMQP to super charge your Django apps!" ?-- Garrett > Smith > * (15 min) ?"Hello, and Beyond" -- Clyde Forrester > > When > ---- > > Thursday, September 10th, ~7pm > > Location > -------- > > Tribune Tower, ? 435 N. Michigan Ave. > > RSVP to brian ?(at) hackerjournalist.net > > > Other Meetings > -------------- > > (before) Kinzie Chop House (some plan to gather around 4:30) > (after) Billy Goat is obvious (and with merit) OR CND Gyros & Lounge > > > About ChiPy > ----------- > > ChiPy is a group of Chicago Python Programmers, l33t, and n00bs. > Meetings are held monthly at various locations around Chicago. > Also, ChiPy is a proud sponsor of many Open Source and Educational > efforts in Chicago. Stay tuned to the mailing list for more info. > > ChiPy website: > ChiPy Mailing List: > ChiPy Announcement *ONLY* Mailing List: > > Python website: > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From brian at sixthw.com Thu Sep 10 16:27:19 2009 From: brian at sixthw.com (Brian Boyer) Date: Thu, 10 Sep 2009 09:27:19 -0500 Subject: [Chicago] Hacker wanted: Code in the public interest, save journalism, in sunny Chicago, Illinois Message-ID: <85ab0bd40909100727t1956c378w7770ce249e22aba9@mail.gmail.com> As luck would have it, this evening's ChiPy coincides with a job opportunity on our team! (And it was just good luck, honest! We were planning on hosting next month!) Details here: http://apps.chicagotribune.com/2009/09/hacker-wanted-code-in-the-public-interest-save-journalism-in-sunny-chicago-illinois/ If this sounds like fun, then you might find it interesting to meet the folks you'd be working with. :) Hope to see y'all tonight. Thanks. Brian From kumar.mcmillan at gmail.com Thu Sep 10 17:55:39 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Thu, 10 Sep 2009 10:55:39 -0500 Subject: [Chicago] [ANN] Chicago Python User Group ChiPy September Meeting In-Reply-To: <85ab0bd40909100721x5187b724y494422d4c430c895@mail.gmail.com> References: <582AEAED-0E84-4730-8BBC-59ADA8241755@sent.com> <85ab0bd40909100721x5187b724y494422d4c430c895@mail.gmail.com> Message-ID: On Thu, Sep 10, 2009 at 9:21 AM, Brian Boyer wrote: > Beer, pizza and pop will be served at this evening's ChiPy at the Tribune Tower. > > If you think you might attend, please RSVP to brian *at* hackerjournalist.net. > > This will be the best meeting, ever. > > Hope to see you there! Is it going to be obvious where to go when we give our names at the front desk? Is there a floor or department we should tell them we're headed to? > > Brian > > > On Tue, Sep 8, 2009 at 11:02 PM, Brian Ray wrote: >> Chicago Python User Group >> ========================= >> >> History, Tradition, Technology and Journalism meet this Thursday at one of >> our nations most historical landmarks, Tribune Tower--right off the Chicago >> River on the Magnificent Mile. ?This *new* venue speaks to the movement of >> journalism into technology and technology into Python. >> >> So, this will be one of most interesting venues and will provide a solid >> platform for the best user group meeting in the ?history of newspapers. We >> may even channel DjangoCon going on at the same time on the West coast. >> >> Bring a friend. This will is going to be great! >> >> RSVP Right NOW to: brian (at) hackerjournalist.net if you even think you >> might make it. >> >> >> Topics >> ------ >> >> * (30 min) Somewhat Advanced SQLAlchemy ?- Daniel Griffin >> * (30 min) ?Python port of Geo::StreetAddress::US library - Joe Germuska >> * (30 min) ?"Not using AMQP to super charge your Django apps!" ?-- Garrett >> Smith >> * (15 min) ?"Hello, and Beyond" -- Clyde Forrester >> >> When >> ---- >> >> Thursday, September 10th, ~7pm >> >> Location >> -------- >> >> Tribune Tower, ? 435 N. Michigan Ave. >> >> RSVP to brian ?(at) hackerjournalist.net >> >> >> Other Meetings >> -------------- >> >> (before) Kinzie Chop House (some plan to gather around 4:30) >> (after) Billy Goat is obvious (and with merit) OR CND Gyros & Lounge >> >> >> About ChiPy >> ----------- >> >> ChiPy is a group of Chicago Python Programmers, l33t, and n00bs. >> Meetings are held monthly at various locations around Chicago. >> Also, ChiPy is a proud sponsor of many Open Source and Educational >> efforts in Chicago. Stay tuned to the mailing list for more info. >> >> ChiPy website: >> ChiPy Mailing List: >> ChiPy Announcement *ONLY* Mailing List: >> >> Python website: >> >> >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From brian at sixthw.com Thu Sep 10 18:03:43 2009 From: brian at sixthw.com (Brian Boyer) Date: Thu, 10 Sep 2009 11:03:43 -0500 Subject: [Chicago] [ANN] Chicago Python User Group ChiPy September Meeting In-Reply-To: References: <582AEAED-0E84-4730-8BBC-59ADA8241755@sent.com> <85ab0bd40909100721x5187b724y494422d4c430c895@mail.gmail.com> Message-ID: <85ab0bd40909100903h6a2f2010jac84f00bf3c5cb69@mail.gmail.com> The folks at the front desk should point you in the right direction. The meeting room is on Lower Level 2 -- we'll have signs up to make sure you know where to go. b On Thu, Sep 10, 2009 at 10:55 AM, Kumar McMillan wrote: > On Thu, Sep 10, 2009 at 9:21 AM, Brian Boyer wrote: >> Beer, pizza and pop will be served at this evening's ChiPy at the Tribune Tower. >> >> If you think you might attend, please RSVP to brian *at* hackerjournalist.net. >> >> This will be the best meeting, ever. >> >> Hope to see you there! > > Is it going to be obvious where to go when we give our names at the > front desk? ?Is there a floor or department we should tell them we're > headed to? > >> >> Brian >> >> >> On Tue, Sep 8, 2009 at 11:02 PM, Brian Ray wrote: >>> Chicago Python User Group >>> ========================= >>> >>> History, Tradition, Technology and Journalism meet this Thursday at one of >>> our nations most historical landmarks, Tribune Tower--right off the Chicago >>> River on the Magnificent Mile. ?This *new* venue speaks to the movement of >>> journalism into technology and technology into Python. >>> >>> So, this will be one of most interesting venues and will provide a solid >>> platform for the best user group meeting in the ?history of newspapers. We >>> may even channel DjangoCon going on at the same time on the West coast. >>> >>> Bring a friend. This will is going to be great! >>> >>> RSVP Right NOW to: brian (at) hackerjournalist.net if you even think you >>> might make it. >>> >>> >>> Topics >>> ------ >>> >>> * (30 min) Somewhat Advanced SQLAlchemy ?- Daniel Griffin >>> * (30 min) ?Python port of Geo::StreetAddress::US library - Joe Germuska >>> * (30 min) ?"Not using AMQP to super charge your Django apps!" ?-- Garrett >>> Smith >>> * (15 min) ?"Hello, and Beyond" -- Clyde Forrester >>> >>> When >>> ---- >>> >>> Thursday, September 10th, ~7pm >>> >>> Location >>> -------- >>> >>> Tribune Tower, ? 435 N. Michigan Ave. >>> >>> RSVP to brian ?(at) hackerjournalist.net >>> >>> >>> Other Meetings >>> -------------- >>> >>> (before) Kinzie Chop House (some plan to gather around 4:30) >>> (after) Billy Goat is obvious (and with merit) OR CND Gyros & Lounge >>> >>> >>> About ChiPy >>> ----------- >>> >>> ChiPy is a group of Chicago Python Programmers, l33t, and n00bs. >>> Meetings are held monthly at various locations around Chicago. >>> Also, ChiPy is a proud sponsor of many Open Source and Educational >>> efforts in Chicago. Stay tuned to the mailing list for more info. >>> >>> ChiPy website: >>> ChiPy Mailing List: >>> ChiPy Announcement *ONLY* Mailing List: >>> >>> Python website: >>> >>> >>> _______________________________________________ >>> Chicago mailing list >>> Chicago at python.org >>> http://mail.python.org/mailman/listinfo/chicago >>> >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From kumar.mcmillan at gmail.com Thu Sep 10 19:52:09 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Thu, 10 Sep 2009 12:52:09 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado Message-ID: This may be of interest to fellow Chipynauts: http://developers.facebook.com/news.php?blog=1&story=301 "FriendFeed's web server is a relatively simple, non-blocking web server written in Python. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the non-blocking web server and tools." Docs: http://www.tornadoweb.org/documentation It uses epoll (from Linux kernel), interesting. From mdipierro at cs.depaul.edu Fri Sep 11 07:32:04 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Fri, 11 Sep 2009 00:32:04 -0500 Subject: [Chicago] SQLAlchemy presentation In-Reply-To: <3db160680907092009q3f29ac73qb9196b99b6c70b75@mail.gmail.com> References: <3db160680907092009q3f29ac73qb9196b99b6c70b75@mail.gmail.com> Message-ID: Hi Daniel, thank you for your SQLAlchemy presentation. It was inspiring. In fact, I did not know SQLAlchemy could handle "on delete cascade" with sqlite. This is an important feature that web2py (currently) does not have. So I have now implemented this feature in web2py and it is in trunk. Here is an example: python web2py.py -S welcome -M >>> db.define_table('a', Field('name')) >>> db.define_table('b', Field('name'), Field('a',db.a,ondelete='CASCADE')) >>> ida = db.a.insert(name='xxx') >>> db.b.insert(name='yyy', a=ida) 1 >>> db(db.a.id == ida).delete() 1 From robkapteyn at gmail.com Fri Sep 11 18:59:11 2009 From: robkapteyn at gmail.com (Rob Kapteyn) Date: Fri, 11 Sep 2009 11:59:11 -0500 Subject: [Chicago] Python a Weekend language ? Message-ID: Here is a link to the Slashdot article I mentioned last night, titled: "C# and Java Weekday Languages, Python and Ruby For Weekends?" http://tech.slashdot.org/story/09/08/15/1849217/C-and-Java-Weekday-Languages-Python-and-Ruby-For-Weekends -Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsudlow at gmail.com Fri Sep 11 20:39:42 2009 From: jsudlow at gmail.com (Jon Sudlow) Date: Fri, 11 Sep 2009 13:39:42 -0500 Subject: [Chicago] Python a Weekend language ? In-Reply-To: References: Message-ID: > I hope this will change in the future! After writing in IBM system/370 > assembler, I can really appreciate what a joy python/ruby really are. My > humble poker playing robot is almost all python, with only object c and c++ > used for hooking the remote processes. Will it ever be possible for python > to control the hardware and memory like C++ does? > -Jon > > > http://tech.slashdot.org/story/09/08/15/1849217/C-and-Java-Weekday-Languages-Python-and-Ruby-For-Weekends > > -Rob > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shekay at pobox.com Fri Sep 11 20:41:05 2009 From: shekay at pobox.com (sheila miguez) Date: Fri, 11 Sep 2009 13:41:05 -0500 Subject: [Chicago] video apologies Message-ID: Sorry for not showing up last night to record. I did get generous offers to help carry equipment, and thanks for those. but, all of the people who normally record events were either out of town, or unavailable. Due to being on pager duty, I didn't want to run things all by myself hence canceled. -- sheila From g at rre.tt Fri Sep 11 23:35:14 2009 From: g at rre.tt (Garrett Smith) Date: Fri, 11 Sep 2009 16:35:14 -0500 Subject: [Chicago] AMQP links Message-ID: Some folks asked me for some of the AMQP references from last night. AMQP specs: http://www.amqp.org Apache Qpid: http://qpid.apache.org RabbitMQ: http://www.rabbitmq.com Python AMQP (0.8, works with Rabbit): http://code.google.com/p/py-amqplib Ah, heck -- this has got everything you need: http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol If anyone's got any questions, just shoot me an email off list. Garrett From bray at sent.com Sun Sep 13 02:30:56 2009 From: bray at sent.com (Brian Ray) Date: Sat, 12 Sep 2009 19:30:56 -0500 Subject: [Chicago] Tornado, shines good on Python? Message-ID: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> Any thoughts on this... http://www.tornadoweb.org/ From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, Facebook's Director of Products: """Tornado also offers much higher performance than existing Python web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz AMD Opteron system managed 8213 web requests per second, while a single threaded version managed 3353. This compares with Django at 2223 requests per second, Web.py at 2066 and CherryPy at 785.""" -- Brian Ray From maney at two14.net Sun Sep 13 07:28:54 2009 From: maney at two14.net (Martin Maney) Date: Sun, 13 Sep 2009 00:28:54 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> Message-ID: <20090913052854.GA25938@furrr.two14.net> On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: > Any thoughts on this... > > http://www.tornadoweb.org/ > > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, > Facebook's Director of Products: > > """Tornado also offers much higher performance than existing Python > web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz AMD > Opteron system managed 8213 web requests per second, while a single > threaded version managed 3353. This compares with Django at 2223 requests > per second, Web.py at 2066 and CherryPy at 785.""" http://bret.appspot.com/entry/tornado-web-server http://www.apparatusproject.org/blog/2009/09/twisted-web-vs-tornado-performance-test/ http://glyph.twistedmatrix.com/2009/09/what-i-wish-tornado-were.html http://dustin.github.com/2009/09/12/tornado.html hackernews: who needs slashdot except for games and celebrity news, now? -- Man's mind, once stretched by a new idea, never regains its original dimensions. -- Oliver Wendell Holmes From mdipierro at cs.depaul.edu Sun Sep 13 08:42:10 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Sun, 13 Sep 2009 01:42:10 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> Message-ID: <3B1F936D-6577-4B31-A534-544B91013BB8@cs.depaul.edu> Is the web server the bottle neck in web applications? My experience is that the bottle neck is the database. My understanding is that Tornado is not a multithreaded server. It can accept concurrent requests but executes them sequentially for each process. It is know that this kind of design is fast because there are no threads but it is not appropriate for all web apps in particular not if the requests may take time to run. It works great in "hello world" type of benchmarks. Another issue is that Tornado is probably using 4 python instances thus taking advantages of the 4 cores. The Django and CherryPy are probably not doing that because they would require a load balancer like Pound. I doubt they did so I am not convinced they are comparing apples to apples. Am I wrong? Massimo On Sep 12, 2009, at 7:30 PM, Brian Ray wrote: > > Any thoughts on this... > > http://www.tornadoweb.org/ > > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, > Facebook's Director of Products: > > """Tornado also offers much higher performance than existing > Python web frameworks. A multiple process Tornado server on a 4 core > 2.4Ghz AMD Opteron system managed 8213 web requests per second, while > a single threaded version managed 3353. This compares with Django at > 2223 requests per second, Web.py at 2066 and CherryPy at 785.""" > > -- Brian Ray > > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From maney at two14.net Sun Sep 13 16:10:24 2009 From: maney at two14.net (Martin Maney) Date: Sun, 13 Sep 2009 09:10:24 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <20090913052854.GA25938@furrr.two14.net> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> Message-ID: <20090913141024.GA14910@furrr.two14.net> On Sun, Sep 13, 2009 at 12:28:54AM -0500, Martin Maney wrote: > On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: > > Any thoughts on this... > > > > http://www.tornadoweb.org/ Apparently Python hackers labor through the night to bring us yet more news and benchmarks. (hmmmm... lies, news and benchmarks? just free-associating while the caffeine osmoses) http://antoniocangiano.com/2009/09/13/benchmarking-tornado-vs-twisted-web-vs-tornado-on-twisted-vs-unicorn/ http://www.youtube.com/watch?v=LYubpuIe3cw (tornado vs. rails: pun) ...and the race continues. -- "What's so funny? That I'm sitting on a deserted beach at night, nibbling at gourmet meals, and rereading every book I've ever loved? Can you think of anything more worthwhile?" -- Michael Flynn From kumar.mcmillan at gmail.com Sun Sep 13 19:56:21 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Sun, 13 Sep 2009 12:56:21 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> Message-ID: On Sat, Sep 12, 2009 at 7:30 PM, Brian Ray wrote: > > Any thoughts on this... > > ? ?http://www.tornadoweb.org/ > > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, > Facebook's Director of Products: > > ? ?"""Tornado also offers much higher performance than existing Python web > frameworks. A multiple process Tornado server on a 4 core 2.4Ghz AMD Opteron > system managed 8213 web requests per second, while a single threaded version > managed 3353. This compares with Django at 2223 requests per second, Web.py > at 2066 and CherryPy at 785.""" It's pretty interesting since event based architecture performs well. However, I believe it won't be possible to simply "drop in" 3rd party modules because most are not written as non-blocking code. After a brief glance at the Tornado source it seems that they have written a lot of custom low level components to account for this. I assume part of their open source initiative is to begin building their empire of non-blocking Python libraries. Hmm, this is pretty much what Twisted had to do as well. So, unfortunately, Tornado may turn into yet another isolated silo of Python libraries. > > -- Brian Ray > > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From dgriff1 at gmail.com Sun Sep 13 20:51:02 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Sun, 13 Sep 2009 13:51:02 -0500 Subject: [Chicago] Best practices for this concurrent problem? Message-ID: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> Hi, I have been playing with a program which essentially reads some elements from a database then starts a "thread" for each element, this can be a few elements or thousands. Inside that thread it does some processing then starts n sockets to do tasks. The life of the thread is generally a few seconds but can stretch up to hours or even days. I think my options are to use threading, multi-processing or some sort of async thing like twisted. I am really unsure which one to choose. Here are the results of my testing so far: threads - GIL death when I spin up a ton of threads, I can limit how many threads I make but I know I am not getting anywhere near full utilization of the box. processes - I dont think its a good idea to make processes that are short lived, it seems too expensive. This is even worse on windows. async - I would have to re-factor to make this work and havent tried yet. summary - What is the best way to deal with (sometimes) large numbers of "threads" that do a small amount of processing and a large amount of socket io? Thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfein at pobox.com Mon Sep 14 01:24:41 2009 From: pfein at pobox.com (Pete) Date: Sun, 13 Sep 2009 18:24:41 -0500 Subject: [Chicago] Best practices for this concurrent problem? In-Reply-To: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> References: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> Message-ID: On Sep 13, 2009, at 1:51 PM, Daniel Griffin wrote: > processes - I dont think its a good idea to make processes that are > short lived, it seems too expensive. This is even worse on windows. I don't know diddly about windows, but the process creation overhead on Linux is pretty minimal - the differences b/w threads & processes are slightly. > async - I would have to re-factor to make this work and havent tried > yet. > > summary - What is the best way to deal with (sometimes) large > numbers of "threads" that do a small amount of processing and a > large amount of socket io? Sounds like the typical case for async, actually, but since I don't know anything about your problem, I can't really say. Async's also a pit of a pain to write/read/maintain, as you mention. Have you thought about refactoring to use long-lived processes/ threads? A better design is often to create workers which receive tasks (typically off a queue) and process them one at a time. This usually gives better results than spawning a new thread for each task. --Pete From dgriff1 at gmail.com Mon Sep 14 01:50:17 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Sun, 13 Sep 2009 18:50:17 -0500 Subject: [Chicago] Best practices for this concurrent problem? In-Reply-To: References: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> Message-ID: <3db160680909131650i748266b3ibbc4e3e80fb5c0ae@mail.gmail.com> I didnt even see the multiprocessing Pool class. I think that will work perfectly. The big difference in overhead for Processes is that you need to re-init certain things for each process which is a lot of work. Since Windows doesn't have fork it actually just calls your program again and gets to the point where you make a process which is pretty rough. Dan On Sun, Sep 13, 2009 at 6:24 PM, Pete wrote: > On Sep 13, 2009, at 1:51 PM, Daniel Griffin wrote: > > processes - I dont think its a good idea to make processes that are short >> lived, it seems too expensive. This is even worse on windows. >> > > I don't know diddly about windows, but the process creation overhead on > Linux is pretty minimal - the differences b/w threads & processes are > slightly. > > async - I would have to re-factor to make this work and havent tried yet. >> >> summary - What is the best way to deal with (sometimes) large numbers of >> "threads" that do a small amount of processing and a large amount of socket >> io? >> > > Sounds like the typical case for async, actually, but since I don't know > anything about your problem, I can't really say. Async's also a pit of a > pain to write/read/maintain, as you mention. > > Have you thought about refactoring to use long-lived processes/threads? A > better design is often to create workers which receive tasks (typically off > a queue) and process them one at a time. This usually gives better results > than spawning a new thread for each task. > > --Pete > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.mcavoy at gmail.com Mon Sep 14 20:32:54 2009 From: chris.mcavoy at gmail.com (Chris McAvoy) Date: Mon, 14 Sep 2009 13:32:54 -0500 Subject: [Chicago] Day of Cloud Message-ID: <3096c19d0909141132h67e03a3al1336155731c7bc7a@mail.gmail.com> Hi All, There's a conference coming up on October 16th aimed at developers using big fluffy clouds. AppEngine will be there, and I'll be talking about Amazon, with a Python twist. The website for the conference is at http://www.dayofcloud.com/ Chris From g at rre.tt Tue Sep 15 03:11:50 2009 From: g at rre.tt (Garrett Smith) Date: Mon, 14 Sep 2009 20:11:50 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: I'm maintaining an informal list of benchmark results of a simple "Hello World" style app for various web servers. I just updated the list to include Tornado. These benchmarks are part of a web framework I'm working on in Erlang called Landshark. The app must print the test "You're that clever shark, aren't you" in either HTML or plain text. The numbers are from Apache Benchmark running for 10 seconds under different concurrency levels. The numbers below are not rigorous by any stretch. In fact, please consider them, officially, damned lies. Based on these lies, however, here's my personal takeaway, at least as far as the Python servers are concerned: - CherryPy is very, very fast and has the upside of being threaded, which means you don't need to deal with weird async/event based models (see Kumar's post to this thread, which is spot on) - Fapws, which spells "Fast Asynchronous Python Web Server" is very, very fast indeed, as it's mostly written in C -- however it's event based (used libev) so beware of blocking operations (e.g. network or file access) and slow Python code, which will bring your numbers down to numbers to mortal levels - Tornado is an impressive contender here -- solid throughput even at high levels of concurrent requests; however, it's only slightly faster than CherryPy within "reasonable" levels of concurrent requests (10,000 or less) and, if you're a speed freak, you should consider Fapws, which blew the field away Obviously there's a lot behind the numbers, such as configuration, system under test, etc. Like I said, they're lies. If you have questions or, in particular, can help me make these go faster with smarter configuration, I'll happily rerun and publish. If you can make WEBrick (a Ruby server) go faster than my friend's golden retriever, which is trained to serve dynamic HTML, I'll buy you a beer. As a final note, the tests themselves will be available with the Landshark download, once that's posted, so people can re-run the tests for themselves. Of course, it's not that hard to run ab against a hello world app. RPS = requests per second Time = average time in milliseconds to server each request Benchmark = `ab -t 10 -c -r http://localhost` 100 Concurrent Requests ----------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 7174 14 Landshark 4479 22 PHP-5 4191 24 Tomcat 6 3554 28 Tornado 2641 38 CherryPy WSGI 2102 48 Jetty 6 937 107 Django WSGI 785 129 WEBrick 43 2350 =============== ====== =========== 1,000 Concurrent Requests ------------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 5359 187 Landshark 4477 224 PHP 5 3062 345 Tomcat 6 3014 345 Tornado 2452 409 CherryPy WSGI 2126 470 Jetty 6 1095 949 Django WSGI 953 1057 WEBrick 50 20607 =============== ====== =========== 10,000 Concurrent Requests -------------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 5213 1920 Landshark 4239 2361 Tomcat 6 2369 4752 Tornado 2265 4439 PHP 5 2239 4282 CherryPy WSGI 1731 5786 Jetty 6 794 13210 Django WSGI 890 12330 WEBrick 84 163425 =============== ====== =========== 20,000 Concurrent Requests -------------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 4788 4178 Landshark 2936 6823 Tornado 2153 9315 PHP 5 1728 11578 Tomcat 6 1362 15102 CherryPy WSGI 1294 15477 Django WSGI 790 27633 Jetty 6 616 32492 WEBrick 63 462464 =============== ====== =========== On Thu, Sep 10, 2009 at 12:52 PM, Kumar McMillan wrote: > This may be of interest to fellow Chipynauts: > http://developers.facebook.com/news.php?blog=1&story=301 > > "FriendFeed's web server is a relatively simple, non-blocking web > server written in Python. The FriendFeed application is written using > a web framework that looks a bit like web.py or Google's webapp, but > with additional tools and optimizations to take advantage of the > non-blocking web server and tools." > > Docs: http://www.tornadoweb.org/documentation > > It uses epoll (from Linux kernel), interesting. > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kumar.mcmillan at gmail.com Tue Sep 15 03:52:09 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Mon, 14 Sep 2009 20:52:09 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: On Mon, Sep 14, 2009 at 8:11 PM, Garrett Smith wrote: > I'm maintaining an informal list of benchmark results of a simple "Hello > World" style app for various web servers. I just updated the list to include > Tornado. > > These benchmarks are part of a web framework I'm working on in Erlang called > Landshark. The app must print the test "You're that clever shark, aren't > you" in either HTML or plain text. The numbers are from Apache Benchmark > running for 10 seconds under different concurrency levels. > > The numbers below are not rigorous by any stretch. In fact, please consider > them, officially, damned lies. > > Based on these lies, however, here's my personal takeaway, at least as far > as the Python servers are concerned: > > CherryPy is very, very fast and has the upside of being threaded, which > means you don't need to deal with weird async/event based models (see > Kumar's post to this thread, which is spot on) > > Fapws, which spells "Fast Asynchronous Python Web Server" is very, very fast > indeed, as it's mostly written in C -- however it's event based (used libev) > so beware of blocking operations (e.g. network or file access) and slow > Python code, which will bring your numbers down to numbers to mortal levels > > Tornado is an impressive contender here -- solid throughput even at high > levels of concurrent requests; however, it's only slightly faster than > CherryPy within "reasonable" levels of concurrent requests (10,000 or less) > and, if you're a speed freak, you should consider Fapws, which blew the > field away > > Obviously there's a lot behind the numbers, such as configuration, system > under test, etc. Like I said, they're lies. If you have questions or, in > particular, can help me make these go faster with smarter configuration, > I'll happily rerun and publish. If you can make WEBrick (a Ruby server) go > faster than my friend's golden retriever, which is trained to serve dynamic > HTML, I'll buy you a beer. Benchmarking Ruby in WEBRick is like benchmarking SimpleHTTPServer in Python. In other words, it's not really fair :) You should set up Phusion Passenger in Apache. It's just as easy as Django / mod_wsgi http://hackd.thrivesmarthq.com/how-to-setup-a-linux-server-for-ruby-on-rails-with-github-and-phusion-passenger (you can probably ignore the SSH config) btw, you don't need to use Rails for your example. You can just make a Rack application: http://www.modrails.com/documentation/Users%20guide.html#_deploying_a_rack_based_ruby_application Rack is more or less Python's WSGI 2.0 ported to Ruby so it will be familiar. > > As a final note, the tests themselves will be available with the Landshark > download, once that's posted, so people can re-run the tests for themselves. > Of course, it's not that hard to run ab against a hello world app. > > RPS = requests per second > Time = average time in milliseconds to server each request > Benchmark = `ab -t 10 -c -r http://localhost` > > 100 Concurrent Requests > ----------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 7174 ? ? ? ? ?14 > Landshark ? ? ? ? 4479 ? ? ? ?? 22 > PHP-5 ? ? ? ? ? ? 4191 ? ? ? ?? 24 > Tomcat 6 ? ? ? ? ?3554 ? ? ? ? ?28 > Tornado ? ? ? ? ? 2641 ? ? ? ? ?38 > CherryPy WSGI ? ? 2102 ? ? ? ?? 48 > Jetty 6 ? ? ? ? ? ?937 ? ? ? ? 107 > Django WSGI ? ? ? ?785 ? ? ? ? 129 > WEBrick ? ? ? ? ? ? 43 ? ? ?? 2350 > =============== ====== =========== > > 1,000 Concurrent Requests > ------------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 5359 ? ? ? ? 187 > Landshark ? ? ? ? 4477 ? ? ? ? 224 > PHP 5 ? ? ? ? ? ? 3062 ? ? ? ? 345 > Tomcat 6 ? ? ? ? ?3014 ? ? ? ? 345 > Tornado ? ? ? ? ? 2452 ? ? ? ? 409 > CherryPy WSGI ? ? 2126 ? ? ? ? 470 > Jetty 6 ? ? ? ? ? 1095 ? ? ? ? 949 > Django WSGI ? ? ? ?953 ? ? ? ?1057 > WEBrick ? ? ? ? ? ? 50 ? ? ? 20607 > =============== ====== =========== > > 10,000 Concurrent Requests > -------------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 5213 ? ? ? ?1920 > Landshark ? ? ? ? 4239 ? ? ? ?2361 > Tomcat 6 ? ? ? ? ?2369 ? ? ? ?4752 > Tornado ? ? ? ? ? 2265 ? ? ? ?4439 > PHP 5 ? ? ? ? ? ? 2239 ? ? ? ?4282 > CherryPy WSGI ? ? 1731 ? ? ? ?5786 > Jetty 6 ? ? ? ? ? ?794 ? ? ? 13210 > Django WSGI ? ? ? ?890 ? ? ? 12330 > WEBrick ? ? ? ? ? ? 84 ? ? ?163425 > =============== ====== =========== > > 20,000 Concurrent Requests > -------------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 4788 ? ? ? ?4178 > Landshark ? ? ? ? 2936 ? ? ? ?6823 > Tornado ? ? ? ? ? 2153 ? ? ? ?9315 > PHP 5 ? ? ? ? ? ? 1728 ? ? ? 11578 > Tomcat 6 ? ? ? ? ?1362 ? ? ? 15102 > CherryPy WSGI ? ? 1294 ? ? ? 15477 > Django WSGI ? ? ? ?790 ? ? ? 27633 > Jetty 6 ? ? ? ? ? ?616 ? ? ? 32492 > WEBrick ? ? ? ? ? ? 63 ? ? ?462464 > =============== ====== =========== > > > On Thu, Sep 10, 2009 at 12:52 PM, Kumar McMillan > wrote: >> This may be of interest to fellow Chipynauts: >> http://developers.facebook.com/news.php?blog=1&story=301 >> >> "FriendFeed's web server is a relatively simple, non-blocking web >> server written in Python. The FriendFeed application is written using >> a web framework that looks a bit like web.py or Google's webapp, but >> with additional tools and optimizations to take advantage of the >> non-blocking web server and tools." >> >> Docs: http://www.tornadoweb.org/documentation >> >> It uses epoll (from Linux kernel), interesting. >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > From chris.mcavoy at gmail.com Tue Sep 15 04:25:10 2009 From: chris.mcavoy at gmail.com (Chris McAvoy) Date: Mon, 14 Sep 2009 21:25:10 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: <3096c19d0909141925k418412fcub0739523ae7f92f2@mail.gmail.com> This is interesting Garrett...thanks. Did you use the stdlib WSGI runner? Or are the wsgi apps behind mod_wsgi? Chris On Mon, Sep 14, 2009 at 8:11 PM, Garrett Smith wrote: > I'm maintaining an informal list of benchmark results of a simple "Hello > World" style app for various web servers. I just updated the list to include > Tornado. > > These benchmarks are part of a web framework I'm working on in Erlang called > Landshark. The app must print the test "You're that clever shark, aren't > you" in either HTML or plain text. The numbers are from Apache Benchmark > running for 10 seconds under different concurrency levels. > > The numbers below are not rigorous by any stretch. In fact, please consider > them, officially, damned lies. > > Based on these lies, however, here's my personal takeaway, at least as far > as the Python servers are concerned: > > CherryPy is very, very fast and has the upside of being threaded, which > means you don't need to deal with weird async/event based models (see > Kumar's post to this thread, which is spot on) > > Fapws, which spells "Fast Asynchronous Python Web Server" is very, very fast > indeed, as it's mostly written in C -- however it's event based (used libev) > so beware of blocking operations (e.g. network or file access) and slow > Python code, which will bring your numbers down to numbers to mortal levels > > Tornado is an impressive contender here -- solid throughput even at high > levels of concurrent requests; however, it's only slightly faster than > CherryPy within "reasonable" levels of concurrent requests (10,000 or less) > and, if you're a speed freak, you should consider Fapws, which blew the > field away > > Obviously there's a lot behind the numbers, such as configuration, system > under test, etc. Like I said, they're lies. If you have questions or, in > particular, can help me make these go faster with smarter configuration, > I'll happily rerun and publish. If you can make WEBrick (a Ruby server) go > faster than my friend's golden retriever, which is trained to serve dynamic > HTML, I'll buy you a beer. > > As a final note, the tests themselves will be available with the Landshark > download, once that's posted, so people can re-run the tests for themselves. > Of course, it's not that hard to run ab against a hello world app. > > RPS = requests per second > Time = average time in milliseconds to server each request > Benchmark = `ab -t 10 -c -r http://localhost` > > 100 Concurrent Requests > ----------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 7174 ? ? ? ? ?14 > Landshark ? ? ? ? 4479 ? ? ? ?? 22 > PHP-5 ? ? ? ? ? ? 4191 ? ? ? ?? 24 > Tomcat 6 ? ? ? ? ?3554 ? ? ? ? ?28 > Tornado ? ? ? ? ? 2641 ? ? ? ? ?38 > CherryPy WSGI ? ? 2102 ? ? ? ?? 48 > Jetty 6 ? ? ? ? ? ?937 ? ? ? ? 107 > Django WSGI ? ? ? ?785 ? ? ? ? 129 > WEBrick ? ? ? ? ? ? 43 ? ? ?? 2350 > =============== ====== =========== > > 1,000 Concurrent Requests > ------------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 5359 ? ? ? ? 187 > Landshark ? ? ? ? 4477 ? ? ? ? 224 > PHP 5 ? ? ? ? ? ? 3062 ? ? ? ? 345 > Tomcat 6 ? ? ? ? ?3014 ? ? ? ? 345 > Tornado ? ? ? ? ? 2452 ? ? ? ? 409 > CherryPy WSGI ? ? 2126 ? ? ? ? 470 > Jetty 6 ? ? ? ? ? 1095 ? ? ? ? 949 > Django WSGI ? ? ? ?953 ? ? ? ?1057 > WEBrick ? ? ? ? ? ? 50 ? ? ? 20607 > =============== ====== =========== > > 10,000 Concurrent Requests > -------------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 5213 ? ? ? ?1920 > Landshark ? ? ? ? 4239 ? ? ? ?2361 > Tomcat 6 ? ? ? ? ?2369 ? ? ? ?4752 > Tornado ? ? ? ? ? 2265 ? ? ? ?4439 > PHP 5 ? ? ? ? ? ? 2239 ? ? ? ?4282 > CherryPy WSGI ? ? 1731 ? ? ? ?5786 > Jetty 6 ? ? ? ? ? ?794 ? ? ? 13210 > Django WSGI ? ? ? ?890 ? ? ? 12330 > WEBrick ? ? ? ? ? ? 84 ? ? ?163425 > =============== ====== =========== > > 20,000 Concurrent Requests > -------------------------- > =============== ====== =========== > App Server ? ? ? ? RPS ? Time (ms) > =============== ====== =========== > Fapws ? ? ? ? ? ? 4788 ? ? ? ?4178 > Landshark ? ? ? ? 2936 ? ? ? ?6823 > Tornado ? ? ? ? ? 2153 ? ? ? ?9315 > PHP 5 ? ? ? ? ? ? 1728 ? ? ? 11578 > Tomcat 6 ? ? ? ? ?1362 ? ? ? 15102 > CherryPy WSGI ? ? 1294 ? ? ? 15477 > Django WSGI ? ? ? ?790 ? ? ? 27633 > Jetty 6 ? ? ? ? ? ?616 ? ? ? 32492 > WEBrick ? ? ? ? ? ? 63 ? ? ?462464 > =============== ====== =========== > > > On Thu, Sep 10, 2009 at 12:52 PM, Kumar McMillan > wrote: >> This may be of interest to fellow Chipynauts: >> http://developers.facebook.com/news.php?blog=1&story=301 >> >> "FriendFeed's web server is a relatively simple, non-blocking web >> server written in Python. The FriendFeed application is written using >> a web framework that looks a bit like web.py or Google's webapp, but >> with additional tools and optimizations to take advantage of the >> non-blocking web server and tools." >> >> Docs: http://www.tornadoweb.org/documentation >> >> It uses epoll (from Linux kernel), interesting. >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > From g at rre.tt Tue Sep 15 04:36:09 2009 From: g at rre.tt (Garrett Smith) Date: Mon, 14 Sep 2009 21:36:09 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: <3096c19d0909141925k418412fcub0739523ae7f92f2@mail.gmail.com> References: <3096c19d0909141925k418412fcub0739523ae7f92f2@mail.gmail.com> Message-ID: On Mon, Sep 14, 2009 at 9:25 PM, Chris McAvoy wrote: > This is interesting Garrett...thanks. > > Did you use the stdlib WSGI runner? Or are the wsgi apps behind mod_wsgi? > > Chris > With the exception of PHP, which was run under Apache MPM + mod_php, everything was run as a standalone server. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kumar.mcmillan at gmail.com Tue Sep 15 05:22:55 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Mon, 14 Sep 2009 22:22:55 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: <3096c19d0909141925k418412fcub0739523ae7f92f2@mail.gmail.com> Message-ID: On Mon, Sep 14, 2009 at 9:36 PM, Garrett Smith wrote: > On Mon, Sep 14, 2009 at 9:25 PM, Chris McAvoy > wrote: >> >> This is interesting Garrett...thanks. >> >> Did you use the stdlib WSGI runner? ?Or are the wsgi apps behind mod_wsgi? >> >> Chris > > With the exception of PHP, which was run under Apache MPM + mod_php, > everything was run as a standalone server. Oh, I thought the Django one was done with mod_wsgi. Hmm, I see what you are trying to do. I don't know of any pure Ruby web server that anyone would ever consider using in production. I don't think there is one. I bet there is a pure PHP web server though. you didn't look hard enough! http://sourceforge.net/projects/astahttpd/ Bam!!! http://webscripts.softpedia.com/script/PHP-Clases/bib-server-50643.html And Bam again! You can thank me later. > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > From maney at two14.net Tue Sep 15 05:38:55 2009 From: maney at two14.net (Martin Maney) Date: Mon, 14 Sep 2009 22:38:55 -0500 Subject: [Chicago] Best practices for this concurrent problem? In-Reply-To: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> References: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> Message-ID: <20090915033855.GA3572@furrr.two14.net> On Sun, Sep 13, 2009 at 01:51:02PM -0500, Daniel Griffin wrote: > I have been playing with a program which essentially reads some elements > from a database then starts a "thread" for each element, this can be a few > elements or thousands. Inside that thread it does some processing then > starts n sockets to do tasks. The life of the thread is generally a few > seconds but can stretch up to hours or even days. > threads - GIL death when I spin up a ton of threads, I can limit how many > threads I make but I know I am not getting anywhere near full utilization of > the box. Yes, native Python threads don't use multiple processors. That's why Tornado's setup was to run 4 instances of Torndao (on a quad core box) behind nginx running to proxy the incoming requests to those worker processes round robin. Something like that is probably a reasonable approach for Python. > processes - I dont think its a good idea to make processes that are short > lived, it seems too expensive. This is even worse on windows. Linux's forking is very fast - in fact forking a new process and starting a "real" thread are the same process with a few options set differently IIRC. OTOH, I believe Python's threading is the user-mode sort, which should be even quicker to start, but with more limitations (such as the often-complained about way Python's threading doesn't scale to multiple cores). Windows is, indeed, a whole different story, and if the codebase has to be performant on both Linux and Windows you may well need to provide different modes of operation. > async - I would have to re-factor to make this work and havent tried yet. Async is the state machine approach. It can yield very good performance, but its Achille's heel is that it doesn't deal well with "long" computations even if they're rare (they cause latency for all pending requests or an often painful refactoring to break the work into small chunks). That's the design choice twisted and Tornado make. > summary - What is the best way to deal with (sometimes) large numbers of > "threads" that do a small amount of processing and a large amount of socket > io? There is no best way, there are only a choice of design tradeoffs. >From where you are now, the simplest might be something like Tornado's deployment model - run a fairly small number of threaded workers with a work manager that dispatches tasks (data sets or queries or whatever exactly they are) to these workers. That should scale at least to around one worker per core; maybe further if the thread bog-down is due to internal overhead in Python's threading. -- Beer does more than Pascal can to justify Date's ways to man. From maney at two14.net Tue Sep 15 05:48:36 2009 From: maney at two14.net (Martin Maney) Date: Mon, 14 Sep 2009 22:48:36 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: <20090915034836.GB3572@furrr.two14.net> On Mon, Sep 14, 2009 at 08:11:50PM -0500, Garrett Smith wrote: > I'm maintaining an informal list of benchmark results of a simple "Hello > World" style app for various web servers. I just updated the list to include > Tornado. > > These benchmarks are part of a web framework I'm working on in Erlang called Nice work, although a trivial "Hello World" type of request seems likely to measure the wrong thing for a web framework, at least as I think of that sort of thing. It does do a fair job of measuring the degenerate case of a bunch of components, but says nothing at all about how it would scale when there were thousands of differnt URLs being requested (hence presumably exercising internal dispatching) and something other than static content being returned (because you should alwys be able to beat a more general framework for static content using nginx or the like). I know, that's very much more trouble, which is why it's rarely done. But these "Hello World" tests of frameworks designed for dynamic web sites do put one to mind of the joke about the drunk who was looking for his keys under the streetlight rather than "over there in the dark" where he dropped them because it was so much easier to see under the streetlight. :-/ -- Trouble rather the tiger in his lair than the sage among his books. For to you kingdoms and their armies are things mighty and enduring, but to him they are but toys of the moment, to be overturned with the flick of a finger. -- Gordon Dickson From dgriff1 at gmail.com Tue Sep 15 05:49:28 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Mon, 14 Sep 2009 22:49:28 -0500 Subject: [Chicago] Best practices for this concurrent problem? In-Reply-To: <20090915033855.GA3572@furrr.two14.net> References: <3db160680909131151j6e992cc4nbf73bcf2c250ec1c@mail.gmail.com> <20090915033855.GA3572@furrr.two14.net> Message-ID: <3db160680909142049j3384ec8djd71cb580f3c3cced@mail.gmail.com> Well, processes and threads are very similar on Unixey OSs but fork makes a copy of your current process and then certain things might break(like DB connections). Python threads or "real" threads except that only 1 thread can use the interpreter at a time. I wrote a test app using multiprocessing pools and I think it will be the best I can hope for, I can start say 8 processes and use them in a pool. There is still the chance I get caught up on 8 long running processes but I think I can risk that. Dan On Mon, Sep 14, 2009 at 10:38 PM, Martin Maney wrote: > On Sun, Sep 13, 2009 at 01:51:02PM -0500, Daniel Griffin wrote: > > I have been playing with a program which essentially reads some elements > > from a database then starts a "thread" for each element, this can be a > few > > elements or thousands. Inside that thread it does some processing then > > starts n sockets to do tasks. The life of the thread is generally a few > > seconds but can stretch up to hours or even days. > > > threads - GIL death when I spin up a ton of threads, I can limit how many > > threads I make but I know I am not getting anywhere near full utilization > of > > the box. > > Yes, native Python threads don't use multiple processors. That's why > Tornado's setup was to run 4 instances of Torndao (on a quad core box) > behind nginx running to proxy the incoming requests to those worker > processes round robin. Something like that is probably a reasonable > approach for Python. > > > processes - I dont think its a good idea to make processes that are short > > lived, it seems too expensive. This is even worse on windows. > > Linux's forking is very fast - in fact forking a new process and > starting a "real" thread are the same process with a few options set > differently IIRC. OTOH, I believe Python's threading is the user-mode > sort, which should be even quicker to start, but with more limitations > (such as the often-complained about way Python's threading doesn't > scale to multiple cores). Windows is, indeed, a whole different story, > and if the codebase has to be performant on both Linux and Windows you > may well need to provide different modes of operation. > > > async - I would have to re-factor to make this work and havent tried yet. > > Async is the state machine approach. It can yield very good > performance, but its Achille's heel is that it doesn't deal well with > "long" computations even if they're rare (they cause latency for all > pending requests or an often painful refactoring to break the work into > small chunks). That's the design choice twisted and Tornado make. > > > summary - What is the best way to deal with (sometimes) large numbers of > > "threads" that do a small amount of processing and a large amount of > socket > > io? > > There is no best way, there are only a choice of design tradeoffs. > >From where you are now, the simplest might be something like Tornado's > deployment model - run a fairly small number of threaded workers with a > work manager that dispatches tasks (data sets or queries or whatever > exactly they are) to these workers. That should scale at least to > around one worker per core; maybe further if the thread bog-down is due > to internal overhead in Python's threading. > > -- > Beer does more than Pascal can to justify Date's ways to man. > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g at rre.tt Tue Sep 15 07:23:34 2009 From: g at rre.tt (Garrett Smith) Date: Tue, 15 Sep 2009 00:23:34 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: On Mon, Sep 14, 2009 at 8:52 PM, Kumar McMillan wrote: > > Benchmarking Ruby in WEBRick is like benchmarking SimpleHTTPServer in > Python. In other words, it's not really fair :) You should set up > Phusion Passenger in Apache. It's just as easy as Django / mod_wsgi > > http://hackd.thrivesmarthq.com/how-to-setup-a-linux-server-for-ruby-on-rails-with-github-and-phusion-passenger > (you can probably ignore the SSH config) > Yay! Ruby web apps *can* serve HTML faster than my friend's golden retriver ;) I owe Kumar a beer! I added a couple new servers to the lot: - Phusion (Ruby rack app + mod_passenger + Apache MPM) - modwsgi I should clarify a few things about the damned lies I'm spreading here. The goal is not to benchmark frameworks, but servers. That's why it's a hello world app. There are no templates, no databases, no sessions, no routing layers -- just an HTTP server using a program over some interface to serve a simple message to the client. I dropped WEBrick, because it was pretty hilarious and web server stats should be dry and boring. Obviously it's not intended for production use. Neither is the Django wsgi server, but I left it in for comparison to the other wsgi server implementations. I should probably drop it altogether since it is probably misleading readers to conclude that Django is really slow. Again, not a framework test, but I can appreciate how one might think it was. I still find it amazing that CherryPy, a pure Python server, holds its own in this group. As a single module that you can include in your source tree, it makes it super simple to build a performant dynamic web app. I think these numbers (lies) should also dispel the myth that threaded servers don't handle high levels of concurrency well. All of the servers were threaded with the exception of Landshark (Erlang processes), Fapws (libev events), and Tornado (epoll events). 100 Concurrent Requests ----------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 7174 14 Landshark 4479 22 PHP-5 4191 24 modwsgi 3651 27 Tomcat 6 3554 28 Tornado 2641 38 CherryPy WSGI 2102 48 Phusion 1873 54 Jetty 6 937 107 Django WSGI 785 129 =============== ====== =========== 1,000 Concurrent Requests ------------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 5359 187 Landshark 4477 224 modwsgi 3449 290 PHP 5 3062 345 Tomcat 6 3014 345 Tornado 2452 409 CherryPy WSGI 2126 470 Phusion 1585 636 Jetty 6 1095 949 Django WSGI 953 1057 =============== ====== =========== 10,000 Concurrent Requests -------------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 5213 1920 Landshark 4239 2361 Tomcat 6 2369 4752 Tornado 2265 4439 PHP 5 2239 4282 modwsgi 2115 4736 CherryPy WSGI 1731 5786 Phusion 1247 8082 Jetty 6 794 13210 Django WSGI 890 12330 =============== ====== =========== 20,000 Concurrent Requests -------------------------- =============== ====== =========== App Server RPS Time (ms) =============== ====== =========== Fapws 4788 4178 Landshark 2936 6823 Tornado 2214 9061 PHP 5 1728 11578 modwsgi 1374 14880 Tomcat 6 1362 15102 CherryPy WSGI 1294 15477 Phusion 961 20894 Django WSGI 790 27633 Jetty 6 616 32492 =============== ====== =========== -------------- next part -------------- An HTML attachment was scrubbed... URL: From g at rre.tt Tue Sep 15 08:38:04 2009 From: g at rre.tt (Garrett Smith) Date: Tue, 15 Sep 2009 01:38:04 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <20090913052854.GA25938@furrr.two14.net> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> Message-ID: On Sun, Sep 13, 2009 at 12:28 AM, Martin Maney wrote: > On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: > > Any thoughts on this... > > > > http://www.tornadoweb.org/ > > > > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, > > Facebook's Director of Products: > > > > """Tornado also offers much higher performance than existing Python > > web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz AMD > > Opteron system managed 8213 web requests per second, while a single > > threaded version managed 3353. This compares with Django at 2223 requests > > per second, Web.py at 2066 and CherryPy at 785.""" > I just saw this. I think these guys are taking the wrong tone. I was saddened to read Glyph's post in that he had to defend Twisted from some (apparently) less than sensitive remarks from the Tornado team. Unfortunate. It's easy to get into numbers wars -- I'm afraid my posts may feed more of this nonsense. Hopefully they wont be misconstrued to be saying that one server is better than another. My interest in this is simply to gain a better understanding of low level performance characteristics when running the simplest of web apps. As has been pointed out by others on the list, there are a *ton* of factors that one should consider when looking a web platform. Hello world benchmarks give you insight into only one facet of a complex system. So, just to show how crazy stats can be, here's my results, using the same ab test the Tornado team used: ======== ====== ======== ======= ======= Server Run 1 Run 2 Run 3 Avg ======== ====== ======== ======= ======= Faps 6046 6335 6319 6233 Modwsgi 3326 3381 3452 3386 Tornado 2273 2257 2132 2221 CherryPy 2112 2044 2048 2068 ======== ====== ======== ======= ======= Pretty different story. Who knows, it's all lies anyway, seriously. I do think they should tone down their claims as they can easily be misinterpreted if not hurtful to the outstanding projects they're comparing themselves with. In any case, having another well supported, excellent codebase like Tornado is a big win for the Python community (and WSGI!), no matter the silly numbers games we play. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cwebber at dustycloud.org Tue Sep 15 15:51:15 2009 From: cwebber at dustycloud.org (Christopher Allan Webber) Date: Tue, 15 Sep 2009 08:51:15 -0500 Subject: [Chicago] Talk about origins and philosophy of free and open source software, within modern philosophy Message-ID: <87eiq8nwoc.fsf@dustycloud.org> Cross-posting this to ChiPy and ChiGlug. I've been considering giving this talk at both. I've been long meaning to give a talk about the origins of free and open source software, as well as the philosophical differences between those terms. I've been thinking of taking it a step further and throwing in . From cwebber at dustycloud.org Tue Sep 15 16:06:36 2009 From: cwebber at dustycloud.org (Christopher Allan Webber) Date: Tue, 15 Sep 2009 09:06:36 -0500 Subject: [Chicago] Talk about origins and philosophy of free and open source software, within modern philosophy In-Reply-To: <87eiq8nwoc.fsf@dustycloud.org> (Christopher Allan Webber's message of "Tue, 15 Sep 2009 08:51:15 -0500") References: <87eiq8nwoc.fsf@dustycloud.org> Message-ID: <878wggnvyr.fsf@dustycloud.org> Sigh, I actually decided to retract this message and save it in my drafts until I was done finalizing the idea for this talk. Instead of saving it in my drafts I accidentally hit send. Sorry. :( Christopher Allan Webber writes: > Cross-posting this to ChiPy and ChiGlug. I've been considering giving > this talk at both. > > I've been long meaning to give a talk about the origins of free and open > source software, as well as the philosophical differences between those > terms. I've been thinking of taking it a step further and throwing in . > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From fasteliteprogrammer at yahoo.com Wed Sep 16 06:12:54 2009 From: fasteliteprogrammer at yahoo.com (Craig) Date: Tue, 15 Sep 2009 21:12:54 -0700 (PDT) Subject: [Chicago] (no subject) Message-ID: <116971.50156.qm@web36508.mail.mud.yahoo.com> Does anyone have a blackberry and use the blackberry messenger? From mandric at gmail.com Wed Sep 16 06:38:58 2009 From: mandric at gmail.com (Milan Andric) Date: Tue, 15 Sep 2009 23:38:58 -0500 Subject: [Chicago] Talk about origins and philosophy of free and open source software, within modern philosophy In-Reply-To: <878wggnvyr.fsf@dustycloud.org> References: <87eiq8nwoc.fsf@dustycloud.org> <878wggnvyr.fsf@dustycloud.org> Message-ID: <536089f30909152138y27093f66ua980c774f1c6cad5@mail.gmail.com> +1 ;) On Tue, Sep 15, 2009 at 9:06 AM, Christopher Allan Webber wrote: > Sigh, I actually decided to retract this message and save it in my > drafts until I was done finalizing the idea for this talk. ?Instead of > saving it in my drafts I accidentally hit send. ?Sorry. ?:( > > Christopher Allan Webber writes: > >> Cross-posting this to ChiPy and ChiGlug. ?I've been considering giving >> this talk at both. >> >> I've been long meaning to give a talk about the origins of free and open >> source software, as well as the philosophical differences between those >> terms. ?I've been thinking of taking it a step further and throwing in . >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From kumar.mcmillan at gmail.com Wed Sep 16 17:41:21 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Wed, 16 Sep 2009 10:41:21 -0500 Subject: [Chicago] [ChicagoLinux] Talk about origins and philosophy of free and open source software, within modern philosophy In-Reply-To: <87eiq8nwoc.fsf@dustycloud.org> References: <87eiq8nwoc.fsf@dustycloud.org> Message-ID: On Tue, Sep 15, 2009 at 8:51 AM, Christopher Allan Webber wrote: > Cross-posting this to ChiPy and ChiGlug. ?I've been considering giving > this talk at both. > > I've been long meaning to give a talk about the origins of free and open > source software, as well as the philosophical differences between those > terms. ?I've been thinking of taking it a step further and throwing in . If you were serious about giving this talk then you have my whole hearted plus one. Great topic. Did everyone see Ian's article about this from Djangocon? http://blog.ianbicking.org/2009/09/10/a-new-self-definition-for-foss/ I didn't see the presentation but it was a great read. From david.durham.jr at gmail.com Wed Sep 16 20:35:54 2009 From: david.durham.jr at gmail.com (David Durham) Date: Wed, 16 Sep 2009 13:35:54 -0500 Subject: [Chicago] [ChicagoLinux] Talk about origins and philosophy of free and open source software, within modern philosophy In-Reply-To: References: <87eiq8nwoc.fsf@dustycloud.org> Message-ID: On Wed, Sep 16, 2009 at 10:41 AM, Kumar McMillan wrote: > > Did everyone see Ian's article about this from Djangocon? > http://blog.ianbicking.org/2009/09/10/a-new-self-definition-for-foss/ > I didn't see the presentation but it was a great read. Good blog post .. insane number of grammatical mistakes / typos in it though. -Dave From ianb at colorstudy.com Wed Sep 16 21:28:15 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 16 Sep 2009 14:28:15 -0500 Subject: [Chicago] [ChicagoLinux] Talk about origins and philosophy of free and open source software, within modern philosophy In-Reply-To: References: <87eiq8nwoc.fsf@dustycloud.org> Message-ID: On Wed, Sep 16, 2009 at 1:35 PM, David Durham wrote: > On Wed, Sep 16, 2009 at 10:41 AM, Kumar McMillan > wrote: > > > > Did everyone see Ian's article about this from Djangocon? > > http://blog.ianbicking.org/2009/09/10/a-new-self-definition-for-foss/ > > I didn't see the presentation but it was a great read. > > Good blog post .. insane number of grammatical mistakes / typos in it > though. > I'll defend the grammatical mistakes by noting I wrote it to be read out loud, where all the rules are different ;) The typos... well, maybe I should get someone to proof it again. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker -------------- next part -------------- An HTML attachment was scrubbed... URL: From fasteliteprogrammer at yahoo.com Thu Sep 17 19:02:39 2009 From: fasteliteprogrammer at yahoo.com (Craig) Date: Thu, 17 Sep 2009 10:02:39 -0700 (PDT) Subject: [Chicago] python error question Message-ID: <369418.69838.qm@web36501.mail.mud.yahoo.com> Why do i get this error when i try to run a python hello world for? Traceback (most recent call last): File "C:\Users\Administrator\Documents\NetBeansProjects\NewPythonProject\src\setup.py", line 4, in from setuptools import setup,find_packages ImportError: No module named setuptools Try to learn to program in netbeans on windows vista:) From dgriff1 at gmail.com Thu Sep 17 19:04:21 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Thu, 17 Sep 2009 12:04:21 -0500 Subject: [Chicago] python error question In-Reply-To: <369418.69838.qm@web36501.mail.mud.yahoo.com> References: <369418.69838.qm@web36501.mail.mud.yahoo.com> Message-ID: <3db160680909171004u3fc27b00n39ba9b5efeacd6ee@mail.gmail.com> Because you dont have setuptools installed. On Thu, Sep 17, 2009 at 12:02 PM, Craig wrote: > Why do i get this error when i try to run a python hello world for? > > > Traceback (most recent call last): > File > "C:\Users\Administrator\Documents\NetBeansProjects\NewPythonProject\src\setup.py", > line 4, in > from setuptools import setup,find_packages > ImportError: No module named setuptools > > Try to learn to program in netbeans on windows vista:) > > > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g at rre.tt Thu Sep 17 19:19:42 2009 From: g at rre.tt (Garrett Smith) Date: Thu, 17 Sep 2009 12:19:42 -0500 Subject: [Chicago] python error question In-Reply-To: <369418.69838.qm@web36501.mail.mud.yahoo.com> References: <369418.69838.qm@web36501.mail.mud.yahoo.com> Message-ID: As far as I know, this is all still current: http://thinkhole.org/wp/2007/02/01/howto-install-setuptools-in-windows/ It's a bit of a pain in Windows, unfortunately. But once you have easy install installed and on your path, getting Python packages is super easy. On Thu, Sep 17, 2009 at 12:02 PM, Craig wrote: > Why do i get this error when i try to run a python hello world for? > > > Traceback (most recent call last): > ?File "C:\Users\Administrator\Documents\NetBeansProjects\NewPythonProject\src\setup.py", line 4, in > ? ?from setuptools import setup,find_packages > ImportError: No module named setuptools > > Try to learn to program in netbeans on windows vista:) > > > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From david.durham.jr at gmail.com Fri Sep 18 00:48:38 2009 From: david.durham.jr at gmail.com (David Durham) Date: Thu, 17 Sep 2009 17:48:38 -0500 Subject: [Chicago] [ChicagoLinux] Talk about origins and philosophy of free and open source software, within modern philosophy In-Reply-To: References: <87eiq8nwoc.fsf@dustycloud.org> Message-ID: > I'll defend the grammatical mistakes by noting I wrote it to be read out > loud, where all the rules are different ;) ?The typos... well, maybe I > should get someone to proof it again. How can you defend a statement like this: "When I read this statement I was immediate head-over-heels in love with this concept." Get it fixed, Mr. Bicking. I'll swoop back around tomorrow and check on your progress / encourage you to continue. :) -Dave From kumar.mcmillan at gmail.com Fri Sep 18 17:02:20 2009 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Fri, 18 Sep 2009 10:02:20 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> Message-ID: On Tue, Sep 15, 2009 at 1:38 AM, Garrett Smith wrote: > On Sun, Sep 13, 2009 at 12:28 AM, Martin Maney wrote: >> >> On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: >> > Any thoughts on this... >> > >> > ? ? http://www.tornadoweb.org/ >> > >> > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, >> > Facebook's Director of Products: >> > >> > ? ? """Tornado also offers much higher performance than existing Python >> > web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz AMD >> > Opteron system managed 8213 web requests per second, while a single >> > threaded version managed 3353. This compares with Django at 2223 >> > requests >> > per second, Web.py at 2066 and CherryPy at 785.""" > > I just saw this. I think these guys are taking the wrong tone. The best response to Twisted / Tornado camps I've seen yet: http://teddziuba.com/2009/09/twisted-vs-tornado-youre-both.html "You're both idiots" > > I was saddened to read Glyph's post in that he had to defend Twisted from > some (apparently) less than sensitive remarks from the Tornado team. > Unfortunate. > > It's easy to get into numbers wars -- I'm afraid my posts may feed more of > this nonsense. Hopefully they wont be misconstrued to be saying that one > server is better than another. My interest in this is simply to gain a > better understanding of low level performance characteristics when running > the simplest of web apps. > > As has been pointed out by others on the list, there are a *ton* of factors > that one should consider when looking a web platform. Hello world benchmarks > give you insight into only one facet of a complex system. > > So, just to show how crazy stats can be, here's my results, using the same > ab test the Tornado team used: > > ======== ====== ======== ======= ======= > Server??? Run 1??? Run 2?? Run 3 ? ? Avg > ======== ====== ======== ======= ======= > Faps?????? 6046???? 6335??? 6319 ?? 6233 > Modwsgi??? 3326???? 3381??? 3452??? 3386 > Tornado??? 2273???? 2257??? 2132??? 2221 > CherryPy?? 2112 ??? 2044??? 2048??? 2068 > ======== ====== ======== ======= ======= > > Pretty different story. Who knows, it's all lies anyway, seriously. I do > think they should tone down their claims as they can easily be > misinterpreted if not hurtful to the outstanding projects they're comparing > themselves with. > > In any case, having another well supported, excellent codebase like Tornado > is a big win for the Python community (and WSGI!), no matter the silly > numbers games we play. > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > From dgriff1 at gmail.com Fri Sep 18 17:07:41 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Fri, 18 Sep 2009 10:07:41 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> Message-ID: <3db160680909180807l726103e3yf12b153231ae68e@mail.gmail.com> "Twisted is probably the douchiest programming library out there. Every time I open up that code, I feel like I've wandered into a late-night bar on the Jersey Shore where everybody's drinking Jager-bombs, and nobody is wearing a shirt." Its also by far the funniest commentary about the whole thing. Who would have thought that a simple web server would cause such a ruckus. Dan On Fri, Sep 18, 2009 at 10:02 AM, Kumar McMillan wrote: > On Tue, Sep 15, 2009 at 1:38 AM, Garrett Smith wrote: > > On Sun, Sep 13, 2009 at 12:28 AM, Martin Maney wrote: > >> > >> On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: > >> > Any thoughts on this... > >> > > >> > http://www.tornadoweb.org/ > >> > > >> > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, > >> > Facebook's Director of Products: > >> > > >> > """Tornado also offers much higher performance than existing > Python > >> > web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz > AMD > >> > Opteron system managed 8213 web requests per second, while a single > >> > threaded version managed 3353. This compares with Django at 2223 > >> > requests > >> > per second, Web.py at 2066 and CherryPy at 785.""" > > > > I just saw this. I think these guys are taking the wrong tone. > > > The best response to Twisted / Tornado camps I've seen yet: > http://teddziuba.com/2009/09/twisted-vs-tornado-youre-both.html > "You're both idiots" > > > > > > I was saddened to read Glyph's post in that he had to defend Twisted from > > some (apparently) less than sensitive remarks from the Tornado team. > > Unfortunate. > > > > It's easy to get into numbers wars -- I'm afraid my posts may feed more > of > > this nonsense. Hopefully they wont be misconstrued to be saying that one > > server is better than another. My interest in this is simply to gain a > > better understanding of low level performance characteristics when > running > > the simplest of web apps. > > > > As has been pointed out by others on the list, there are a *ton* of > factors > > that one should consider when looking a web platform. Hello world > benchmarks > > give you insight into only one facet of a complex system. > > > > So, just to show how crazy stats can be, here's my results, using the > same > > ab test the Tornado team used: > > > > ======== ====== ======== ======= ======= > > Server Run 1 Run 2 Run 3 Avg > > ======== ====== ======== ======= ======= > > Faps 6046 6335 6319 6233 > > Modwsgi 3326 3381 3452 3386 > > Tornado 2273 2257 2132 2221 > > CherryPy 2112 2044 2048 2068 > > ======== ====== ======== ======= ======= > > > > Pretty different story. Who knows, it's all lies anyway, seriously. I do > > think they should tone down their claims as they can easily be > > misinterpreted if not hurtful to the outstanding projects they're > comparing > > themselves with. > > > > In any case, having another well supported, excellent codebase like > Tornado > > is a big win for the Python community (and WSGI!), no matter the silly > > numbers games we play. > > > > > > _______________________________________________ > > Chicago mailing list > > Chicago at python.org > > http://mail.python.org/mailman/listinfo/chicago > > > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlspelich at gmail.com Fri Sep 18 17:21:59 2009 From: rlspelich at gmail.com (Robert Spelich) Date: Fri, 18 Sep 2009 10:21:59 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <3db160680909180807l726103e3yf12b153231ae68e@mail.gmail.com> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> <3db160680909180807l726103e3yf12b153231ae68e@mail.gmail.com> Message-ID: <303c9ec10909180821s3e1710dcudd55a772e1630de7@mail.gmail.com> I'd love to take Tornado for a spin but the install on a Windows machine seems to be more difficult than it should be. Actually its not a Tornado thing. Anyone have a tip for installing pycurl and CURL. pycurl requires the path to libcurl.lib but there is no .lib in the CURL packages I get from the download site. I am certain I am doing something wrong or just don't quite understand what's at play. accomplish. Help?-Bob On Fri, Sep 18, 2009 at 10:07 AM, Daniel Griffin wrote: > "Twisted is probably the douchiest programming library out there. Every > time I open up that code, I feel like I've wandered into a late-night bar on > the Jersey Shore where everybody's drinking Jager-bombs, and nobody is > wearing a shirt." > > Its also by far the funniest commentary about the whole thing. Who would > have thought that a simple web server would cause such a ruckus. > > Dan > > On Fri, Sep 18, 2009 at 10:02 AM, Kumar McMillan > wrote: > >> On Tue, Sep 15, 2009 at 1:38 AM, Garrett Smith wrote: >> > On Sun, Sep 13, 2009 at 12:28 AM, Martin Maney wrote: >> >> >> >> On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: >> >> > Any thoughts on this... >> >> > >> >> > http://www.tornadoweb.org/ >> >> > >> >> > From http://www.heise.de/english/newsticker/news/145214 Bret Taylor, >> >> > Facebook's Director of Products: >> >> > >> >> > """Tornado also offers much higher performance than existing >> Python >> >> > web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz >> AMD >> >> > Opteron system managed 8213 web requests per second, while a single >> >> > threaded version managed 3353. This compares with Django at 2223 >> >> > requests >> >> > per second, Web.py at 2066 and CherryPy at 785.""" >> > >> > I just saw this. I think these guys are taking the wrong tone. >> >> >> The best response to Twisted / Tornado camps I've seen yet: >> http://teddziuba.com/2009/09/twisted-vs-tornado-youre-both.html >> "You're both idiots" >> >> >> > >> > I was saddened to read Glyph's post in that he had to defend Twisted >> from >> > some (apparently) less than sensitive remarks from the Tornado team. >> > Unfortunate. >> > >> > It's easy to get into numbers wars -- I'm afraid my posts may feed more >> of >> > this nonsense. Hopefully they wont be misconstrued to be saying that one >> > server is better than another. My interest in this is simply to gain a >> > better understanding of low level performance characteristics when >> running >> > the simplest of web apps. >> > >> > As has been pointed out by others on the list, there are a *ton* of >> factors >> > that one should consider when looking a web platform. Hello world >> benchmarks >> > give you insight into only one facet of a complex system. >> > >> > So, just to show how crazy stats can be, here's my results, using the >> same >> > ab test the Tornado team used: >> > >> > ======== ====== ======== ======= ======= >> > Server Run 1 Run 2 Run 3 Avg >> > ======== ====== ======== ======= ======= >> > Faps 6046 6335 6319 6233 >> > Modwsgi 3326 3381 3452 3386 >> > Tornado 2273 2257 2132 2221 >> > CherryPy 2112 2044 2048 2068 >> > ======== ====== ======== ======= ======= >> > >> > Pretty different story. Who knows, it's all lies anyway, seriously. I do >> > think they should tone down their claims as they can easily be >> > misinterpreted if not hurtful to the outstanding projects they're >> comparing >> > themselves with. >> > >> > In any case, having another well supported, excellent codebase like >> Tornado >> > is a big win for the Python community (and WSGI!), no matter the silly >> > numbers games we play. >> > >> > >> > _______________________________________________ >> > Chicago mailing list >> > Chicago at python.org >> > http://mail.python.org/mailman/listinfo/chicago >> > >> > >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> > > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > -- -Bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgriff1 at gmail.com Fri Sep 18 17:23:55 2009 From: dgriff1 at gmail.com (Daniel Griffin) Date: Fri, 18 Sep 2009 10:23:55 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <303c9ec10909180821s3e1710dcudd55a772e1630de7@mail.gmail.com> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> <3db160680909180807l726103e3yf12b153231ae68e@mail.gmail.com> <303c9ec10909180821s3e1710dcudd55a772e1630de7@mail.gmail.com> Message-ID: <3db160680909180823x2771cfds3a8fe741dde38990@mail.gmail.com> Tornado is also very dependent on epoll which is exclusive to Linux. On Fri, Sep 18, 2009 at 10:21 AM, Robert Spelich wrote: > I'd love to take Tornado for a spin but the install on a Windows machine > seems to be more difficult than it should be. > > Actually its not a Tornado thing. Anyone have a tip for installing pycurl > and CURL. pycurl requires the path to libcurl.lib but there is no .lib in > the CURL packages I get from the download site. I am certain I am doing > something wrong or just don't quite understand what's at play. accomplish. > > Help?-Bob > > > On Fri, Sep 18, 2009 at 10:07 AM, Daniel Griffin wrote: > >> "Twisted is probably the douchiest programming library out there. Every >> time I open up that code, I feel like I've wandered into a late-night bar on >> the Jersey Shore where everybody's drinking Jager-bombs, and nobody is >> wearing a shirt." >> >> Its also by far the funniest commentary about the whole thing. Who would >> have thought that a simple web server would cause such a ruckus. >> >> Dan >> >> On Fri, Sep 18, 2009 at 10:02 AM, Kumar McMillan < >> kumar.mcmillan at gmail.com> wrote: >> >>> On Tue, Sep 15, 2009 at 1:38 AM, Garrett Smith wrote: >>> > On Sun, Sep 13, 2009 at 12:28 AM, Martin Maney >>> wrote: >>> >> >>> >> On Sat, Sep 12, 2009 at 07:30:56PM -0500, Brian Ray wrote: >>> >> > Any thoughts on this... >>> >> > >>> >> > http://www.tornadoweb.org/ >>> >> > >>> >> > From http://www.heise.de/english/newsticker/news/145214 Bret >>> Taylor, >>> >> > Facebook's Director of Products: >>> >> > >>> >> > """Tornado also offers much higher performance than existing >>> Python >>> >> > web frameworks. A multiple process Tornado server on a 4 core 2.4Ghz >>> AMD >>> >> > Opteron system managed 8213 web requests per second, while a single >>> >> > threaded version managed 3353. This compares with Django at 2223 >>> >> > requests >>> >> > per second, Web.py at 2066 and CherryPy at 785.""" >>> > >>> > I just saw this. I think these guys are taking the wrong tone. >>> >>> >>> The best response to Twisted / Tornado camps I've seen yet: >>> http://teddziuba.com/2009/09/twisted-vs-tornado-youre-both.html >>> "You're both idiots" >>> >>> >>> > >>> > I was saddened to read Glyph's post in that he had to defend Twisted >>> from >>> > some (apparently) less than sensitive remarks from the Tornado team. >>> > Unfortunate. >>> > >>> > It's easy to get into numbers wars -- I'm afraid my posts may feed more >>> of >>> > this nonsense. Hopefully they wont be misconstrued to be saying that >>> one >>> > server is better than another. My interest in this is simply to gain a >>> > better understanding of low level performance characteristics when >>> running >>> > the simplest of web apps. >>> > >>> > As has been pointed out by others on the list, there are a *ton* of >>> factors >>> > that one should consider when looking a web platform. Hello world >>> benchmarks >>> > give you insight into only one facet of a complex system. >>> > >>> > So, just to show how crazy stats can be, here's my results, using the >>> same >>> > ab test the Tornado team used: >>> > >>> > ======== ====== ======== ======= ======= >>> > Server Run 1 Run 2 Run 3 Avg >>> > ======== ====== ======== ======= ======= >>> > Faps 6046 6335 6319 6233 >>> > Modwsgi 3326 3381 3452 3386 >>> > Tornado 2273 2257 2132 2221 >>> > CherryPy 2112 2044 2048 2068 >>> > ======== ====== ======== ======= ======= >>> > >>> > Pretty different story. Who knows, it's all lies anyway, seriously. I >>> do >>> > think they should tone down their claims as they can easily be >>> > misinterpreted if not hurtful to the outstanding projects they're >>> comparing >>> > themselves with. >>> > >>> > In any case, having another well supported, excellent codebase like >>> Tornado >>> > is a big win for the Python community (and WSGI!), no matter the silly >>> > numbers games we play. >>> > >>> > >>> > _______________________________________________ >>> > Chicago mailing list >>> > Chicago at python.org >>> > http://mail.python.org/mailman/listinfo/chicago >>> > >>> > >>> _______________________________________________ >>> Chicago mailing list >>> Chicago at python.org >>> http://mail.python.org/mailman/listinfo/chicago >>> >> >> >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> >> > > > -- > -Bob > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tal.liron at threecrickets.com Fri Sep 18 17:48:59 2009 From: tal.liron at threecrickets.com (Tal Liron) Date: Fri, 18 Sep 2009 10:48:59 -0500 Subject: [Chicago] Tornado, shines good on Python? In-Reply-To: <3db160680909180807l726103e3yf12b153231ae68e@mail.gmail.com> References: <635F08B9-EDDC-439C-A218-D2C47722EB1A@sent.com> <20090913052854.GA25938@furrr.two14.net> <3db160680909180807l726103e3yf12b153231ae68e@mail.gmail.com> Message-ID: <4AB3ABEB.2040201@threecrickets.com> Wow, everybody's all crazy about non-blocking I/O these days! I'm jumping the gun a bit here, but the RESTful platform I'm working on, Prudence, is based on a non-blocking I/O server (Grizzly) over the JVM. (In my ChiPy presentation, I called it "Restoration". That had to be renamed. Sounded like a backup solution product!) The update since my presentation is that Prudence now works with Jython 2.5, kinda. Jython 2.5 ended up being released without support for JSR-223, which Prudence requires. However, it does have support in svn. HOWEVER, this support is all kinds of broken! So, I've included a patched version of Jython 2.6svn in Prudence, and am joining the Jython team, I guess, to try to fix this good. Details, details. I'm also working on having Prudence available in different "flavors". The Python flavor, which will be the first, will feature a nice demo web app, with Prudence as the "frontend" and SQLAlchemy as the "backend". For those interested, I will likely be presenting it at ACM on Nov 12. Until then, go for the svn snapshot of Prudence to get it with Jython 2.5: http://threecrickets.com/prudence/ For the record, I think non-blocking I/O is useful for a very, very small group of applications, which either truly hit the "c10k users" roof, or are web services in extreme volume environments (such as in the trading industry). For anybody else, please repeat to yourself over and over that "scalability does not equal performance". In fact, many applications that are designed to scale well perform poorly. (memcached comes to mind. This will be part of my next talk at ChiPy, about Django performance.) All this to say: Tornado is interesting, Twisted is interesting, Prudence is (of course!) interesting, but their non-blocking features might be irrelevant to you. Actually, Prudence lets you switch its web server to blocking Jetty, which can be good. Non-blocking I/O is tricky to code, and these libraries are notorious for very hard to find, and weird bugs. I can provide non-joyful anecdotes. Bottom line: robustness may be more important than scalability for you. Final note: there are extremely scalable web servers out there that use /blocking/ I/O. When you need to scale: test, and don't believe the hype. -Tal From ianb at colorstudy.com Fri Sep 18 21:54:44 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 18 Sep 2009 14:54:44 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: On Mon, Sep 14, 2009 at 8:52 PM, Kumar McMillan wrote: > Benchmarking Ruby in WEBRick is like benchmarking SimpleHTTPServer in > Python. ?In other words, it's not really fair :) There's nothing that unfair about benchmarking SimpleHTTPServer. paste.httpserver uses SimpleHTTPServer, and it's not as fast as CherryPy (which is still pure Python and I think not entirely unlike SimpleHTTPServer, just more streamlined), but it's not terrible. It's certainly nothing like WEBrick. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker From mdipierro at cs.depaul.edu Fri Sep 18 22:03:50 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Fri, 18 Sep 2009 15:03:50 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: Message-ID: <7FE408FF-608C-409F-9416-9AFCB9F6AE74@cs.depaul.edu> I can contribute one more: http://web2py.com/examples/static/web2pyserver.py - api compatible with cherrypy and very much inspired by it. - works with cherrypy ssl_handler (to be tested) but will soon have its own. - multithreaded - can handle requests and responses via chunking (like cherrypy) (but not tested yet!) - should work with python 3 (but not tried yet!) - 30-50% faster then cherrypy in my tests. I could use some independent tests and benchmarks. Massimo On Sep 18, 2009, at 2:54 PM, Ian Bicking wrote: > On Mon, Sep 14, 2009 at 8:52 PM, Kumar McMillan > wrote: >> Benchmarking Ruby in WEBRick is like benchmarking SimpleHTTPServer in >> Python. In other words, it's not really fair :) > > There's nothing that unfair about benchmarking SimpleHTTPServer. > paste.httpserver uses SimpleHTTPServer, and it's not as fast as > CherryPy (which is still pure Python and I think not entirely unlike > SimpleHTTPServer, just more streamlined), but it's not terrible. It's > certainly nothing like WEBrick. > > -- > Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From maney at two14.net Fri Sep 18 23:51:20 2009 From: maney at two14.net (Martin Maney) Date: Fri, 18 Sep 2009 16:51:20 -0500 Subject: [Chicago] Django Debug Toolbar: the music video Message-ID: <20090918215120.GA1900@furrr.two14.net> Okay, not really - it's just a screencast (animated version of screenshots) with a soundtrack. Someone's been doing some serious revising to the interface while I wasn't looking! http://robhudson.github.com/django-debug-toolbar/ -- It isn't that secrets are never needed in security. It's that they are never desirable. -- Whitfield Diffie From g at rre.tt Sat Sep 19 01:41:15 2009 From: g at rre.tt (Garrett Smith) Date: Fri, 18 Sep 2009 18:41:15 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: <7FE408FF-608C-409F-9416-9AFCB9F6AE74@cs.depaul.edu> References: <7FE408FF-608C-409F-9416-9AFCB9F6AE74@cs.depaul.edu> Message-ID: On Fri, Sep 18, 2009 at 3:03 PM, Massimo Di Pierro wrote: > I can contribute one more: > > ? ? http://web2py.com/examples/static/web2pyserver.py > > - api compatible with cherrypy and very much inspired by it. > - works with cherrypy ssl_handler (to be tested) but will soon have its own. > - multithreaded > - can handle requests and responses via chunking (like cherrypy) (but not > tested yet!) > - should work with python 3 (but not tried yet!) > - 30-50% faster then cherrypy in my tests. > > I could use some independent tests and benchmarks. I've confirmed these results. The new web2py WSGI server is a playah! web2server is on par with CherryPy at lowish levels of concurrency (< 1000) but is far better at handling very high levels of concurrency requests (> 10,000). I was skeptical of Massimos 50% faster claims, but it shows up at these very high levels. It's *very* comparable, per my benchmarks, to Tornado. And it's threaded. I hope word gets out about this. As folks have been saying here for a while now -- it's really not a good idea to plunge into async/event concurrency models for web apps. Hell, even crazy highly concurrent apps like instant messaging or simulators can probably get by perfectly well using threads, provided the stack size is kept reasonable. If you need a hundred thousand concurrent, long running processes, fine. But you'll probably want something like Stackless anyway. Or Erlang :) Nice work Massimo! P.S. The LGPL is kind of a pain for people that want to throw this module into their source tree and not worry about the particulars of that license. Massimo, if you plan to keep this as a single module (hope so), would you consider an alternative or dual license? From mdipierro at cs.depaul.edu Sat Sep 19 03:45:46 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Fri, 18 Sep 2009 20:45:46 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: <7FE408FF-608C-409F-9416-9AFCB9F6AE74@cs.depaul.edu> Message-ID: Thank you Garrett for the tests. Can you tell us about what hardware/os you used? Massimo Could you tell us what machine you are using for the test? Massimo On Sep 18, 2009, at 6:41 PM, Garrett Smith wrote: > On Fri, Sep 18, 2009 at 3:03 PM, Massimo Di Pierro > wrote: >> I can contribute one more: >> >> http://web2py.com/examples/static/web2pyserver.py >> >> - api compatible with cherrypy and very much inspired by it. >> - works with cherrypy ssl_handler (to be tested) but will soon have >> its own. >> - multithreaded >> - can handle requests and responses via chunking (like cherrypy) >> (but not >> tested yet!) >> - should work with python 3 (but not tried yet!) >> - 30-50% faster then cherrypy in my tests. >> >> I could use some independent tests and benchmarks. > > I've confirmed these results. The new web2py WSGI server is a playah! > > web2server is on par with CherryPy at lowish levels of concurrency (< > 1000) but is far better at handling very high levels of concurrency > requests (> 10,000). I was skeptical of Massimos 50% faster claims, > but it shows up at these very high levels. > > It's *very* comparable, per my benchmarks, to Tornado. And it's > threaded. > > I hope word gets out about this. > > As folks have been saying here for a while now -- it's really not a > good idea to plunge into async/event concurrency models for web apps. > Hell, even crazy highly concurrent apps like instant messaging or > simulators can probably get by perfectly well using threads, provided > the stack size is kept reasonable. > > If you need a hundred thousand concurrent, long running processes, > fine. But you'll probably want something like Stackless anyway. Or > Erlang :) > > Nice work Massimo! > > P.S. The LGPL is kind of a pain for people that want to throw this > module into their source tree and not worry about the particulars of > that license. Massimo, if you plan to keep this as a single module > (hope so), would you consider an alternative or dual license? > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From mdipierro at cs.depaul.edu Sat Sep 19 10:25:37 2009 From: mdipierro at cs.depaul.edu (Massimo Di Pierro) Date: Sat, 19 Sep 2009 03:25:37 -0500 Subject: [Chicago] Facebook open sources FriendFeed's real-time Python web framework, Tornado In-Reply-To: References: <7FE408FF-608C-409F-9416-9AFCB9F6AE74@cs.depaul.edu> Message-ID: <88B3B42A-BEB7-4EBB-A8B1-96D030E846AD@cs.depaul.edu> Here is a better version: http://web2py.com/examples/static/sneaky.py and a Python 3.0 version: http://web2py.com/examples/static/sneaky3.py They may still need some more work but they seem to work fine. I did not test SSL and chunked uploads yet. Massimo On Sep 18, 2009, at 8:45 PM, Massimo Di Pierro wrote: > Thank you Garrett for the tests. > > Can you tell us about what hardware/os you used? > > Massimo > > Could you tell us what machine you are using for the test? > > Massimo > > On Sep 18, 2009, at 6:41 PM, Garrett Smith wrote: > >> On Fri, Sep 18, 2009 at 3:03 PM, Massimo Di Pierro >> wrote: >>> I can contribute one more: >>> >>> http://web2py.com/examples/static/web2pyserver.py >>> >>> - api compatible with cherrypy and very much inspired by it. >>> - works with cherrypy ssl_handler (to be tested) but will soon have >>> its own. >>> - multithreaded >>> - can handle requests and responses via chunking (like cherrypy) >>> (but not >>> tested yet!) >>> - should work with python 3 (but not tried yet!) >>> - 30-50% faster then cherrypy in my tests. >>> >>> I could use some independent tests and benchmarks. >> >> I've confirmed these results. The new web2py WSGI server is a playah! >> >> web2server is on par with CherryPy at lowish levels of concurrency (< >> 1000) but is far better at handling very high levels of concurrency >> requests (> 10,000). I was skeptical of Massimos 50% faster claims, >> but it shows up at these very high levels. >> >> It's *very* comparable, per my benchmarks, to Tornado. And it's >> threaded. >> >> I hope word gets out about this. >> >> As folks have been saying here for a while now -- it's really not a >> good idea to plunge into async/event concurrency models for web apps. >> Hell, even crazy highly concurrent apps like instant messaging or >> simulators can probably get by perfectly well using threads, provided >> the stack size is kept reasonable. >> >> If you need a hundred thousand concurrent, long running processes, >> fine. But you'll probably want something like Stackless anyway. Or >> Erlang :) >> >> Nice work Massimo! >> >> P.S. The LGPL is kind of a pain for people that want to throw this >> module into their source tree and not worry about the particulars of >> that license. Massimo, if you plan to keep this as a single module >> (hope so), would you consider an alternative or dual license? >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago From maney at two14.net Sat Sep 19 23:25:28 2009 From: maney at two14.net (Martin Maney) Date: Sat, 19 Sep 2009 16:25:28 -0500 Subject: [Chicago] Google Python Style Guide Message-ID: <20090919212528.GB1900@furrr.two14.net> http://google-styleguide.googlecode.com/svn/trunk/pyguide.html Oh, and there are similar guides for some other languages that no sane person would care about, like C++ and Objective-C - crazy stuff. ;-) -- Not on the wealthy, who buy only what they want when they want it, was the vast superstructure of industry founded and built up, but on those who, aching for a luxury beyond their reach and for a leisure forever denied them, could be bullied or wheedled into spending their few hardly won shillings on whatever might give them, if only for a moment, a leisured and luxurious illusion. -- Dorothy Sayers, _Murder Must Advertise_ From bray at sent.com Sat Sep 19 23:29:40 2009 From: bray at sent.com (Brian Ray) Date: Sat, 19 Sep 2009 16:29:40 -0500 Subject: [Chicago] Google Python Style Guide In-Reply-To: <20090919212528.GB1900@furrr.two14.net> References: <20090919212528.GB1900@furrr.two14.net> Message-ID: <2FD99161-9648-49E4-8BEB-3B5AD736BB08@sent.com> On Sep 19, 2009, at 4:25 PM, Martin Maney wrote: > > http://google-styleguide.googlecode.com/svn/trunk/pyguide.html > > Oh, and there are similar guides for some other languages that no sane > person would care about, like C++ and Objective-C - crazy stuff. ;-) Heh, this is really funny... it is like super charged PEP8. Brian Ray From maney at two14.net Sun Sep 20 02:09:08 2009 From: maney at two14.net (Martin Maney) Date: Sat, 19 Sep 2009 19:09:08 -0500 Subject: [Chicago] Google Python Style Guide In-Reply-To: <2FD99161-9648-49E4-8BEB-3B5AD736BB08@sent.com> References: <20090919212528.GB1900@furrr.two14.net> <2FD99161-9648-49E4-8BEB-3B5AD736BB08@sent.com> Message-ID: <20090920000908.GC1900@furrr.two14.net> On Sat, Sep 19, 2009 at 04:29:40PM -0500, Brian Ray wrote: > Heh, this is really funny... it is like super charged PEP8. With simple yet amusing in-page widgets to "reveal details"! Really, I came across very little that made me say "uhm, wait". One that comes to mind, perhaps because I'd just violated it earlier today, was the IMO over-strict one about limiting list comprehensions. I do not for a moment think that result.extend('%s %s' % (p, s) for p in prefixes for s in tails) would be improved in any reasonable way by unrolling it into multiple lines, yet the rule says Only One Iterable Is Allowed. I think they've just chosen a 90% right condition and turned it into a too-rigid rule that has obvious failings. Of course, the intent - don't write overly involved list comprehensions - is one with which I agree. Another issue they've made me think about is importing non-module objects. "from xyz import *" is easy to argue against, but I was surprised to see the rule (1) "Use imports for packages and modules only." I'll concede that Google probably has more experience than I do about large code bases with many different people working on them, but I can't see what the problem is here, unless they're in the habit of shuffling functions or classes about from one place to another with gay abandon. (I will say that the sort of thing I use from ... import x,y,z with is mostly framework-ish stuff that gets used over and over in some files, such as Django's Model class and field types, and that may be partly because so many of them have long-enough names as it is) Style guides: the biggest, most in need of painting, bikesheds ever! :-) (1) I still find that terse, not clear. The expansion makes it clear: "Use import x for importing packages and modules. Use from x import y only when x is a package and y is a module." -- vi is a microcosm of the Unix world. Don't expect to learn all of it at once; perhaps you shouldn't expect to learn all of it at all. -- Jon Lasser (Think Unix) From lists at durin42.com Sun Sep 20 04:03:24 2009 From: lists at durin42.com (Augie Fackler) Date: Sat, 19 Sep 2009 21:03:24 -0500 Subject: [Chicago] Google Python Style Guide In-Reply-To: <20090920000908.GC1900@furrr.two14.net> References: <20090919212528.GB1900@furrr.two14.net> <2FD99161-9648-49E4-8BEB-3B5AD736BB08@sent.com> <20090920000908.GC1900@furrr.two14.net> Message-ID: <671C5043-CE1A-4380-A730-EF581D5A8AE1@durin42.com> On Sep 19, 2009, at 7:09 PM, Martin Maney wrote: > On Sat, Sep 19, 2009 at 04:29:40PM -0500, Brian Ray wrote: >> Heh, this is really funny... it is like super charged PEP8. > > With simple yet amusing in-page widgets to "reveal details"! > > Really, I came across very little that made me say "uhm, wait". One > that comes to mind, perhaps because I'd just violated it earlier > today, > was the IMO over-strict one about limiting list comprehensions. I do > not for a moment think that > > result.extend('%s %s' % (p, s) for p in prefixes for s in tails) > > would be improved in any reasonable way by unrolling it into multiple > lines, yet the rule says Only One Iterable Is Allowed. Come at it as a C++ or Java programmer though - most projects at Google use Java and C++ primarily, and last I knew the majority of Googlers only know Python at the level of having read _Python in a Nutshell_ on the shuttle to work. The list comprehension is something that a lot of people have a little trouble with if they're coming to Python as an nth programming language. I'll confess I found it non- intuitive (and a little confusing) at first. > I think they've > just chosen a 90% right condition and turned it into a too-rigid rule > that has obvious failings. Of course, the intent - don't write overly > involved list comprehensions - is one with which I agree. The point is that you might write a service in Python which some other poor sucker^W^W engineer will have to use, and if it acts oddly, he might need to be able to read the source. He still has to be able to parse your code if he's not fluent in the nuances of your language. The price for that is that you neuter some of the more useful features of the language. Then again, my worst single piece of Python comes from abusing map, reduce, and list comprehensions. I think it's the single feature of the language which requires the most restraint. > Another issue they've made me think about is importing non-module > objects. "from xyz import *" is easy to argue against, but I was > surprised to see the rule (1) "Use imports for packages and modules > only." I'll concede that Google probably has more experience than I > do > about large code bases with many different people working on them, but > I can't see what the problem is here, unless they're in the habit of > shuffling functions or classes about from one place to another with > gay > abandon. (I will say that the sort of thing I use from ... import > x,y,z with is mostly framework-ish stuff that gets used over and over > in some files, such as Django's Model class and field types, and that > may be partly because so many of them have long-enough names as it is) When you're looking at a 1000 line file (sadly, not uncommon), it's nice to know that the function that just got used is really an import from another module (because it has a foo. prefix) and that you should go look in another file instead of scrolling around wildly or searching for "def ..." looking for a function when it might be a class (this has happened to me even in small codebases) gets tedious. > > Style guides: the biggest, most in need of painting, bikesheds > ever! :-) Indeed! > (1) I still find that terse, not clear. The expansion makes it clear: > "Use import x for importing packages and modules. Use from x import y > only when x is a package and y is a module." > > -- > vi is a microcosm of the Unix world. Don't expect > to learn all of it at once; perhaps you shouldn't expect > to learn all of it at all. -- Jon Lasser (Think Unix) > > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > From mtemkin at speakeasy.net Sun Sep 20 17:47:08 2009 From: mtemkin at speakeasy.net (Marc Temkin) Date: Sun, 20 Sep 2009 10:47:08 -0500 Subject: [Chicago] Chicago Digest, Vol 49, Issue 21 Style Guide Discussion In-Reply-To: References: Message-ID: <9E5D05A497034B2986AFC60CB435C877@Dreamcatcher2> Every new programming language comes out promoting their more direct syntax with equivalent or more usefulness then predecessors. These languages, be it C#, Java, Ruby, Python, others, then offer some less obvious but more powerful syntax. Certain groups of programmers adopt these idioms extensively and weaken the original goal of the language by obscuring the original direct syntax. After years of creeping obfuscation, we get a new language! This stylization has always occurred in literature,music composition, and other arts but the authors and composers create these styles to uniquely identify their work. Marc Temkin From joe.jasinski at gmail.com Mon Sep 21 01:07:33 2009 From: joe.jasinski at gmail.com (Joe Jasinski) Date: Sun, 20 Sep 2009 18:07:33 -0500 Subject: [Chicago] Google Python Style Guide In-Reply-To: <671C5043-CE1A-4380-A730-EF581D5A8AE1@durin42.com> References: <20090919212528.GB1900@furrr.two14.net> <2FD99161-9648-49E4-8BEB-3B5AD736BB08@sent.com> <20090920000908.GC1900@furrr.two14.net> <671C5043-CE1A-4380-A730-EF581D5A8AE1@durin42.com> Message-ID: Hey Augie, Just joined the ChiPy mailing group. Saw your name. How's life been? I assume you are in the city? Joe On Sat, Sep 19, 2009 at 9:03 PM, Augie Fackler wrote: > > On Sep 19, 2009, at 7:09 PM, Martin Maney wrote: > > On Sat, Sep 19, 2009 at 04:29:40PM -0500, Brian Ray wrote: >> >>> Heh, this is really funny... it is like super charged PEP8. >>> >> >> With simple yet amusing in-page widgets to "reveal details"! >> >> Really, I came across very little that made me say "uhm, wait". One >> that comes to mind, perhaps because I'd just violated it earlier today, >> was the IMO over-strict one about limiting list comprehensions. I do >> not for a moment think that >> >> result.extend('%s %s' % (p, s) for p in prefixes for s in tails) >> >> would be improved in any reasonable way by unrolling it into multiple >> lines, yet the rule says Only One Iterable Is Allowed. >> > > Come at it as a C++ or Java programmer though - most projects at Google use > Java and C++ primarily, and last I knew the majority of Googlers only know > Python at the level of having read _Python in a Nutshell_ on the shuttle to > work. The list comprehension is something that a lot of people have a little > trouble with if they're coming to Python as an nth programming language. > I'll confess I found it non-intuitive (and a little confusing) at first. > > I think they've >> just chosen a 90% right condition and turned it into a too-rigid rule >> that has obvious failings. Of course, the intent - don't write overly >> involved list comprehensions - is one with which I agree. >> > > The point is that you might write a service in Python which some other poor > sucker^W^W engineer will have to use, and if it acts oddly, he might need to > be able to read the source. He still has to be able to parse your code if > he's not fluent in the nuances of your language. The price for that is that > you neuter some of the more useful features of the language. > > Then again, my worst single piece of Python comes from abusing map, reduce, > and list comprehensions. I think it's the single feature of the language > which requires the most restraint. > > Another issue they've made me think about is importing non-module >> objects. "from xyz import *" is easy to argue against, but I was >> surprised to see the rule (1) "Use imports for packages and modules >> only." I'll concede that Google probably has more experience than I do >> about large code bases with many different people working on them, but >> I can't see what the problem is here, unless they're in the habit of >> shuffling functions or classes about from one place to another with gay >> abandon. (I will say that the sort of thing I use from ... import >> x,y,z with is mostly framework-ish stuff that gets used over and over >> in some files, such as Django's Model class and field types, and that >> may be partly because so many of them have long-enough names as it is) >> > > When you're looking at a 1000 line file (sadly, not uncommon), it's nice to > know that the function that just got used is really an import from another > module (because it has a foo. prefix) and that you should go look in another > file instead of scrolling around wildly or searching for "def ..." looking > for a function when it might be a class (this has happened to me even in > small codebases) gets tedious. > > >> Style guides: the biggest, most in need of painting, bikesheds ever! :-) >> > > Indeed! > > > (1) I still find that terse, not clear. The expansion makes it clear: >> "Use import x for importing packages and modules. Use from x import y >> only when x is a package and y is a module." >> >> -- >> vi is a microcosm of the Unix world. Don't expect >> to learn all of it at once; perhaps you shouldn't expect >> to learn all of it at all. -- Jon Lasser (Think Unix) >> >> _______________________________________________ >> Chicago mailing list >> Chicago at python.org >> http://mail.python.org/mailman/listinfo/chicago >> >> > _______________________________________________ > Chicago mailing list > Chicago at python.org > http://mail.python.org/mailman/listinfo/chicago > -- Joe J. Jasinski www.joejasinski.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From szybalski at gmail.com Mon Sep 28 22:42:35 2009 From: szybalski at gmail.com (Lukasz Szybalski) Date: Mon, 28 Sep 2009 15:42:35 -0500 Subject: [Chicago] Calendar Server instructions In-Reply-To: <804e5c70909281340jfc757b4yca6ac87cf39c6027@mail.gmail.com> References: <804e5c70909281340jfc757b4yca6ac87cf39c6027@mail.gmail.com> Message-ID: <804e5c70909281342s46526b68waa551ba49e6cf6a6@mail.gmail.com> Kind of off topic, but still running python twisted ...so.... Not sure how many of you need calendar Server in you company, but I just updated instructions on how to setup Darwin Calendar Server on debian. It was as easy as 123. http://lucasmanual.com/mywiki/CalendarServer Now users can share and add events, and tasks inside their mail client. Contents ? 1. Install CalendarServer ? 2. Configure Calendar Server ? 3. Add Calendar To Thunderbird ? 4. Enable Calendar on company network ? 5. Share events and Tasks Enjoy, Lucas -- Setup CalendarServer for your company. http://lucasmanual.com/mywiki/CalendarServer Automotive Recall Database - See if you vehicle has a recall http://lucasmanual.com/recall From maney at two14.net Mon Sep 28 23:21:09 2009 From: maney at two14.net (Martin Maney) Date: Mon, 28 Sep 2009 16:21:09 -0500 Subject: [Chicago] Python (and some other) news on the web Message-ID: <20090928212109.GA16185@furrr.two14.net> PyPy has some early results from their JIT - looking good! http://morepypy.blogspot.com/2009/09/first-results-of-jit.html His solution seems to be all Ruby, but I'm all over his complaints about CSS lacking DRYness. Really, really lacking... http://chriseppstein.github.com/blog/2009/09/20/why-stylesheet-abstraction-matters/ Ever find yourself choosing between open source RDBMs? This may help: http://www.anchor.com.au/hosting/dedicated/mysql_vs_postgres ...okay, where did the other Python-related item go to? Weren't there two of those? Well, while I'm rummaging about, amuse yourselves... Scary: you're never more than 145 miles away from a Big Mac in the continental US! http://strangemaps.wordpress.com/2009/09/26/413-the-mcfarthest-place-145-mi-to-the-nearest-big-mac/ Ah, here we are. Come join us on IRC and ask feihong about it - he keeps mentioning argparse. :-) http://www.python.org/dev/peps/pep-0389/ Cooking may have been a necessary step in the evolution of humans. Who knew? http://www.npr.org/templates/story/story.php?storyId=112334465 -- The phenomenon of financial excess associated with promising novel technologies is a recurring feature of the last two centuries. -- Andrew Odlyzko