[Tutor] overview of how data is handled

[Alan Gauld]

> First let me say that the books and all the online tutorials (that I
> have seen) do not address how to handle data retrieved from a SQL
> database. They explain how to get the data but not how the data is
to be
> handled after it is retrieved.

Thats because you can handle it any way you like!
Once its out its all yours.

> experiments that the data needs to be in a dict form.

Hmm, I wonder. The normal approach is to use a cursor and
retrieve the data piece by piece or in small chunks. Are
you trying to pull back all the data at once? That will
be more problematic (albeit faster if you have enough RAM).
But if you allow the database cursor to hold and traverse
the query results for you it shouldn't be too hard.

This is normal relational database practice, but not normal
Windows IDE type practice which disguises the cursor with
table views and the like. Python takes a more server like
approach (as normally used by COBOL, C, etc).

> (two loops).  If the program has to convert from list into dict for
> ten's of thousands of records it is going to be very  slow.

Yep, that's why you normally write specific queries and insert
the results in a cursor. You can then fetch the data from the
cursor. Alternatively write more specific SQL and execute on
demand - it depends on where the CPU time gets eaten...

> The handling of data is like nothing I have ever done.  It's nothing
> like Delphi, Java, .Net or VFP.

Thats true. I don;t know of any Python environment that mimics
the way VB and Delphi present data. But python is far from
alone in its approach.

> application - i.e data entry with grids, preparing reports for
viewing
> or printing etc....

Normally to use a grid with N rows you pull N rows outof the
cursor and populate the grid. How the population gets done
will depend on the grid you use!

Preparing reports is more normally done by writing as much as
possible in SQL and then iterating line by line over the
resultant cursor.

Having said all that I haven't used Python's database interface
in earnest, my experiences are with COBOL and C!

Alan G.