xml + mmap cross

castironpi castironpi at gmail.com
Thu Sep 4 21:57:51 CEST 2008


On Sep 4, 2:14 am, Stefan Behnel <stefan... at behnel.de> wrote:
> castironpi wrote:
> > Any interest in pursuing/developing/working together on a mmaped-xml
> > class?  Faster, not readable in text editor.
>
> Any hints on what you are talking about?
>
> Stefan

Nice to hear from you.  I assumed you were familiar with the problem;
you're not.

In an XML file, entries are stored in serial, sort of like this.

AAA BBB CCC DDD

Or more recognizably,

<A><B><C>something</C><D>something</D></B></A>

Point is, to change <C>something</C> to <C>something else</C>, you
have to recopy everything after that.

AAA BBB CCC DDD
AAA BBBb CCC DDD

requires 7 writes, 'b CCC DDD', not 1.

I want to use a simple tree structure to store:

0 A-> None, 1
1 B-> None, 2
2 C-> 3, None
3 D-> None, None

Each node maps to 'Next, Child', or more accurately, 'Next Sibling,
First Child'.

You get constant time updates to contents, and log-time searches.

There was a similar problem today in:

From: Gerhard Häring <g... at ghaering.de>
Date: Thu, 04 Sep 2008 13:08:59 +0200
Subject: Re: cPickle

The OP wanted to update the third element in a pickled tuple, but not
the first two.

I propose to write a tree structure to a memory-mapped file.  A
heavyweight string class, Rope, I wrote, exceeded native string speeds
at a file size of two megs.  You could use that, or store the tree
directly.

The obstacle is probably mmap 'alloc' and 'free' routines, which I
posted on Google Code.



More information about the Python-list mailing list