From barry at list.org Tue Jul 5 15:06:54 2011 From: barry at list.org (Barry Warsaw) Date: Tue, 5 Jul 2011 09:06:54 -0400 Subject: [mmgsoc] Status Message-ID: <20110705090654.4ab9b865@neurotica.wooz.org> Hi folks, I'm now back from my travels and hoping to catch up on how you guys are doing. What's your status? Do you have anything to demo? Are you stuck on anything? Next week are already the mid-term evaluations. Can you believe it? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From cs5070214 at cse.iitd.ac.in Wed Jul 6 01:45:16 2011 From: cs5070214 at cse.iitd.ac.in (Dushyant Bansal) Date: Wed, 06 Jul 2011 05:15:16 +0530 Subject: [mmgsoc] Status Message-ID: <4E13A20C.6000405@cse.iitd.ac.in> Hi, I have finished the basic implementation of archiver UI using pylons framework and Storm ORM. Though it still requires some minor fixes, it covers: 1. conversation-list view 2. conversation page with indentation and quote hiding feature. Code can be found at: http://code.launchpad.net/~dushyant37/+junk/ArchiverUI Egg package: http://bazaar.launchpad.net/~dushyant37/+junk/ArchiverUI/files/head:/ArchiverUI/dist/ Other info: README.txt It just requires a sqlite database file. But currently, the test database has not been generated from any list archives (mbox file), so, it is not good for demo purpose. That is also the reason I have not been able to test it thoroughly. Maybe in a couple of days, we can get that. Overall, generating pages dynamically seems to be a better way to view archives. I would like to discuss some other issues before proceeding further. 1. Performance: We need to look at the performance of the new archiver in order to improve it as well as to compare this approach of generating pages on the fly with the static one. Is there any specific way to go about it? After that, performance gain can be achieved by proper use of caching which is offered by pylons and storm/sqlite. 2. Interaction with mailman: Right now, the interaction with mailman (archiver part) is through a sqlite database file. So, we just need to update the relevant database files on archiving a message through mailman. If we want to further separate out the archiver from mailman, we can use the methods used by mhonarc and mailarchive interfaces. 3. Search: I also plan to integrate search functionality(work done by Priya) for archives into this pylons project. Please give your feedback. Thanks, Dushyant From dcrodman at gmail.com Wed Jul 6 19:35:14 2011 From: dcrodman at gmail.com (Andrew Rodman) Date: Wed, 6 Jul 2011 13:35:14 -0400 Subject: [mmgsoc] Status Message-ID: Hey, I'm finished with the actual ORM mapping (thanks to some help from dushyant) and am re-implementing the methods in HyperDatabase. Rather than using DumbBTrees, these methods now rely on Storm queries and mappings to achieve persistant storage and access to stored archives. It's not done yet, but I'm steadily making progress and only have a couple of methods to write in HyperDatabase. I see two main objectives that I need to achieve once HyperDatabase is finished: 1. Remove any index references in pipermail.py and replacing them with storm queries where needed 2. The upgrade script (pickles -> sqlite db entries) for the Mm2 to Mm3 upgrade procedure My branch can be accessed here: https://code.launchpad.net/~dcrodman/mailman/storm-upgrade Feedback and suggestions are always welcome. - Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From benedict.stein at googlemail.com Wed Jul 6 20:21:06 2011 From: benedict.stein at googlemail.com (Benedict Stein) Date: Wed, 06 Jul 2011 20:21:06 +0200 Subject: [mmgsoc] Status In-Reply-To: <20110705090654.4ab9b865@neurotica.wooz.org> References: <20110705090654.4ab9b865@neurotica.wooz.org> Message-ID: <1309976466.2371.14.camel@vaio-fe31m> Just to be brief, talked to Flo about this earlier, I had to attend a preparation course for my volunatry year the last few days, I really hope to be back to you soon. IIRC we stopped with a new testing layer which fixes dupliaction bugs during doctests, most of the sites are now rendered in "the site" which does already have menu items which are not yet linked. Overall I'd summarize MM3 Webui got a7 compatible and some minor enhancements, I'm still experimenting on how to display the different pages , but I can proceed tomorrow :-) In case you do have questions feel free to contact Flo or me. Am Dienstag, den 05.07.2011, 09:06 -0400 schrieb Barry Warsaw: > Hi folks, > > I'm now back from my travels and hoping to catch up on how you guys are > doing. What's your status? Do you have anything to demo? Are you stuck on > anything? > > Next week are already the mid-term evaluations. Can you believe it? > > -Barry > _______________________________________________ > MMGSOC mailing list > MMGSOC at python.org > http://mail.python.org/mailman/listinfo/mmgsoc -- Einen sch?nen Tag w?nscht: Benedict Stein -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 1057 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From barry at list.org Tue Jul 12 17:13:45 2011 From: barry at list.org (Barry Warsaw) Date: Tue, 12 Jul 2011 11:13:45 -0400 Subject: [mmgsoc] Status In-Reply-To: References: Message-ID: <20110712111345.182df776@resist.wooz.org> On Jul 06, 2011, at 01:35 PM, Andrew Rodman wrote: >I'm finished with the actual ORM mapping (thanks to some help from dushyant) >and am re-implementing the methods in HyperDatabase. Rather than using >DumbBTrees, these methods now rely on Storm queries and mappings to achieve >persistant storage and access to stored archives. It's not done yet, but I'm >steadily making progress and only have a couple of methods to write in >HyperDatabase. > >I see two main objectives that I need to achieve once HyperDatabase is >finished: > > 1. Remove any index references in pipermail.py and replacing them with > storm queries where needed > 2. The upgrade script (pickles -> sqlite db entries) for the Mm2 to Mm3 > upgrade procedure > >My branch can be accessed here: >https://code.launchpad.net/~dcrodman/mailman/storm-upgrade > >Feedback and suggestions are always welcome. Thanks for the update. I'll grab the branch and take a look soon. (Sadly, I'm now in system-restore mode as my main dev machine lost my $HOME.) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Tue Jul 12 17:18:55 2011 From: barry at list.org (Barry Warsaw) Date: Tue, 12 Jul 2011 11:18:55 -0400 Subject: [mmgsoc] Status In-Reply-To: <4E13A20C.6000405@cse.iitd.ac.in> References: <4E13A20C.6000405@cse.iitd.ac.in> Message-ID: <20110712111855.0add4ded@resist.wooz.org> As I mentioned, my main dev machine is down for the count, but I can look at things once I get that working again. On Jul 06, 2011, at 05:15 AM, Dushyant Bansal wrote: >1. Performance: We need to look at the performance of the new archiver in >order to improve it as well as to compare this approach of generating pages >on the fly with the static one. Is there any specific way to go about it? >After that, performance gain can be achieved by proper use of caching which >is offered by pylons and storm/sqlite. I guess there are a couple of sides to this. How well does it perform for single-message updates? How does it perform for full-archive regeneration? What is the cold render time for a single page, and how does that break down among the various components (e.g. db query, message retrieval, message rendering)? What about warm access in the absence of a cache? >2. Interaction with mailman: Right now, the interaction with mailman >(archiver part) is through a sqlite database file. So, we just need to update >the relevant database files on archiving a message through mailman. > If we want to further separate out the archiver from mailman, we can use the > methods used by mhonarc and mailarchive interfaces. I still think I want to separate the archiver from the core engine. It doesn't have to be done now, but it's worth thinking about. >3. Search: I also plan to integrate search functionality(work done by Priya) >for archives into this pylons project. Cool. To all students: are you writing unit tests and documentation for your changes? Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Tue Jul 12 17:21:09 2011 From: barry at list.org (Barry Warsaw) Date: Tue, 12 Jul 2011 11:21:09 -0400 Subject: [mmgsoc] Status In-Reply-To: <1309976466.2371.14.camel@vaio-fe31m> References: <20110705090654.4ab9b865@neurotica.wooz.org> <1309976466.2371.14.camel@vaio-fe31m> Message-ID: <20110712112109.38d6fdaa@resist.wooz.org> On Jul 06, 2011, at 08:21 PM, Benedict Stein wrote: >IIRC we stopped with a new testing layer which fixes dupliaction bugs >during doctests, >most of the sites are now rendered in "the site" which does already have >menu items which are not yet linked. I don't understand these two items. Can you elaborate? ;) >Overall I'd summarize MM3 Webui got a7 compatible and some minor >enhancements, I'm still experimenting on how to display the different >pages , but I can proceed tomorrow :-) Cool. Remember, I'm always on #mailman during US/Eastern working hours! Question: have you or any of the other students found anything missing in the core that blocks your progress? If so, please let me know and I will prioritize them. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From benedict.stein at googlemail.com Tue Jul 12 17:24:49 2011 From: benedict.stein at googlemail.com (Benedict Stein) Date: Tue, 12 Jul 2011 17:24:49 +0200 Subject: [mmgsoc] Status - doctest In-Reply-To: <20110712111855.0add4ded@resist.wooz.org> References: <4E13A20C.6000405@cse.iitd.ac.in> <20110712111855.0add4ded@resist.wooz.org> Message-ID: <1310484289.8923.8.camel@vaio-fe31m> Yes i do add all newly implemented parts to doctests which do exist. Am Dienstag, den 12.07.2011, 11:18 -0400 schrieb Barry Warsaw: > As I mentioned, my main dev machine is down for the count, but I can look at > things once I get that working again. > > On Jul 06, 2011, at 05:15 AM, Dushyant Bansal wrote: > > >1. Performance: We need to look at the performance of the new archiver in > >order to improve it as well as to compare this approach of generating pages > >on the fly with the static one. Is there any specific way to go about it? > >After that, performance gain can be achieved by proper use of caching which > >is offered by pylons and storm/sqlite. > > I guess there are a couple of sides to this. How well does it perform for > single-message updates? How does it perform for full-archive regeneration? > What is the cold render time for a single page, and how does that break down > among the various components (e.g. db query, message retrieval, message > rendering)? What about warm access in the absence of a cache? > > >2. Interaction with mailman: Right now, the interaction with mailman > >(archiver part) is through a sqlite database file. So, we just need to update > >the relevant database files on archiving a message through mailman. > > If we want to further separate out the archiver from mailman, we can use the > > methods used by mhonarc and mailarchive interfaces. > > I still think I want to separate the archiver from the core engine. It > doesn't have to be done now, but it's worth thinking about. > > >3. Search: I also plan to integrate search functionality(work done by Priya) > >for archives into this pylons project. > > Cool. > > To all students: are you writing unit tests and documentation for your > changes? > > Cheers, > -Barry > _______________________________________________ > MMGSOC mailing list > MMGSOC at python.org > http://mail.python.org/mailman/listinfo/mmgsoc -- Einen sch?nen Tag w?nscht: Benedict Stein -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From benedict.stein at googlemail.com Tue Jul 12 17:42:57 2011 From: benedict.stein at googlemail.com (Benedict Stein) Date: Tue, 12 Jul 2011 17:42:57 +0200 Subject: [mmgsoc] Status - webUI In-Reply-To: <20110712112109.38d6fdaa@resist.wooz.org> References: <20110705090654.4ab9b865@neurotica.wooz.org> <1309976466.2371.14.camel@vaio-fe31m> <20110712112109.38d6fdaa@resist.wooz.org> Message-ID: <1310485377.8923.18.camel@vaio-fe31m> Am Dienstag, den 12.07.2011, 11:21 -0400 schrieb Barry Warsaw: > On Jul 06, 2011, at 08:21 PM, Benedict Stein wrote: > > >IIRC we stopped with a new testing layer which fixes dupliaction bugs > >during doctests, > > >most of the sites are now rendered in "the site" which does already have > >menu items which are not yet linked. > > I don't understand these two items. Can you elaborate? ;) 1. As promissed Flo tought me to implement and use doctest in all parts of the project. But i had issued getting correct tests because running the test would create dummy content into the mm3 DB - which was not running in testing mode when running django test. In the meantime we solved this - Flo amended some lines to the Tests which make sure we do have a mailman testing instance running. Since of then all tests pass successfully, and there is no dummy content in the mm3 DB. 2. Anna and Flo already prepared a few views last year, i moddified them to work together with the new version of REST. Most of these sides were coded for single use (using a "what to do next" link part at the end of the page. I've used my proposal for a possible Menu structre and added a menu above the content block, most of the menu items are not yet linked to the correct view. ? But that's where I'm today, in the meantime i focused on one way of displaying only parts of the settings (using GET var option and section) and could go on linking the menu items to the correct views or parts of the settings. In addition i had problems saving settings after modifying them - these are now gone. At the moment I go on further with menu items, creating the neccessary views / links. In particular I'm modfying the subscription page. Please feel free to download the latest dev version or contact me on IRC. > > >Overall I'd summarize MM3 Webui got a7 compatible and some minor > >enhancements, I'm still experimenting on how to display the different > >pages , but I can proceed tomorrow :-) > > Cool. Remember, I'm always on #mailman during US/Eastern working hours! > > Question: have you or any of the other students found anything missing in the > core that blocks your progress? If so, please let me know and I will > prioritize them. > > -Barry -- Einen sch?nen Tag w?nscht: Benedict Stein -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From barry at list.org Fri Jul 15 21:33:04 2011 From: barry at list.org (Barry Warsaw) Date: Fri, 15 Jul 2011 15:33:04 -0400 Subject: [mmgsoc] Status - webUI In-Reply-To: <1310485377.8923.18.camel@vaio-fe31m> References: <20110705090654.4ab9b865@neurotica.wooz.org> <1309976466.2371.14.camel@vaio-fe31m> <20110712112109.38d6fdaa@resist.wooz.org> <1310485377.8923.18.camel@vaio-fe31m> Message-ID: <20110715153304.045d9186@resist.wooz.org> On Jul 12, 2011, at 05:42 PM, Benedict Stein wrote: >Please feel free to download the latest dev version or contact me on IRC. Thanks for the update. I'm hoping to spend some more time on Mailman in the next two weeks, so I'll take a look at the GSoC branches. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: