Hey Dushyant, I'm not always on IRC so I figured I'd reply so you have my email if you want to reach me. As Barry mentioned, I've planned to retire dumbBTree in favor of a Storm ORM object with all of the Article types mapped to various properties (with the same naming scheme) since we won't be using pickles anymore. Mapping a list of message-ids to build conversations seems like a good idea to me and could easily be added to the ORM class I've written to represent Articles. Let me know if you'd like to discuss anything further, even if it is as simple as adding an additional list property.<div>
<br></div><div>- Drew<br><br><div class="gmail_quote">On Fri, Jun 10, 2011 at 6:25 PM, Terri Oda <span dir="ltr"><<a href="mailto:terri@zone12.com">terri@zone12.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">Barry Warsaw wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2. Generation of conversations list pages:<br>
Suppose at one point, we have 100 pages containing conversations<br>
lists. When a new message arrives for archiving, we can just update one<br>
or two pages by inserting new entry corresponding to this conversation<br>
and removing old entry(only if it is present in those one or two<br>
pages). This will eventually leave static files of conversation lists in<br>
an inconsistent state. So, we have to rebuild more or all pages of<br>
conversations list as a cron job.<br>
</blockquote>
<br>
I think we should explore again the performance of generating these pages on<br>
the fly. As I mentioned to Andrew, if doing so would cause problems meeting<br>
the schedule, it shouldn't be the highest priority, but if you and he can<br>
somehow share the load, then I think it's worth exploring. You'll have to<br>
figure out whether dynamic generation of the pages will be acceptable<br>
performance wise, and you'll need some way to cache the results, but if it<br>
works out, I think it could be quite powerful.<br>
</blockquote>
<br></div>
As Barry says... basically do what allows you to meet your schedule.<br>
<br>
I'm wondering if it might not be easier to do this particular task dynamically, though: then you could just query the database and get the most up-to-date listing. Would it be easier to write the code for that and then for the static code, use that same script to regenerate the conversation when a new message comes in?<br>
<br>
Incidentally, how do you propose to do conversations in Mailman 3? Systers is using a modified codebase where conversations are separated using +conversation stuff stuck into the email address used for posting. (So for example, I might post a job interview to systers+jobs@... if I were posting a job to the mailing list). You won't have that logic, so will you be using References:/In-Reply-To: combined with the subject line to approximate that? It's considered a bit of a technical problem in general, but probably any approximation we use will be "good enough" to start. I can't remember if we've talked about that before...<br>
<br>
Terri<br>
<br>
_______________________________________________<br>
MMGSOC mailing list<br>
<a href="mailto:MMGSOC@python.org" target="_blank">MMGSOC@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/mmgsoc" target="_blank">http://mail.python.org/mailman/listinfo/mmgsoc</a><br>
</blockquote></div><br></div>