Question of speed - Flat file DBMS
parker at hiredatum.demon.co.uk
Sun Mar 6 14:36:57 CET 2005
In message <mailman.3365.1110014887.22381.python-list at python.org>, I.V.
Aprameya Rao <aprameya at students.iiit.net> writes
>OK, i forgot to mention this.
>The speed is a critical issue because there will be a competition and
>whosever solution is faster wins the prize.
>Hence will a python solution be as fast as a C++ solution??
>On 4 Mar 2005, John Machin wrote:
>> I.V. Aprameya Rao wrote:
>> > Hi
>> > I have to implement a flat file dbms. The basic condition is that
>> > relations will be given in files and i will have to run certain
>> > project join queries on those relations.
>> > Can someone tell me as to which language will be faster, python or
>> Faster to get a working app released: Python
>> Faster to drive you nuts: C++
>> Faster processing the files: My hunch is C++, but not by much. After
>> you've shipped your working app (in Python), you'll still have lots of
>> spare time to tweak up the speed -- IF it's slow, if anybody notices,
>> and if anybody cares -- and this newsgroup usually provides a lively
>> response to "how do I make this faster" questions.
The key to speed is disk caching. Unless you're doing a profound amount
of computation on your data, I doubt there'll be any significant
difference between using Python or C++, except due to how much disk i/o
is done by the different programs and language environments.
So cache as much as you can- read your entire database into memory if it
will fit. If not then try to ensure you can read your entire indices in
memory. Create an index for any field you'll be querying on to avoid
having to read the entire record. If you're dealing with massive data,
think about indices of indices. Work on the data, or at least the
indices. in memory.
Well, that's everything I ever learned about database design.
More information about the Python-list