Python database of plain text editable by notepad or vi

[James Harris]

On 25 Mar, 22:56, Jon Clements <jon... at googlemail.com> wrote:
> On 25 Mar, 22:40, James Harris <james.harri... at googlemail.com> wrote:
>
> > I am looking to store named pieces of text in a form that can be
> > edited by a standard editor such as notepad (under Windows) or vi
> > (under Unix) and then pulled into Python as needed. The usual record
> > locking and transactions of databases are not required.
>
> > Another way to look at it is to treat the separate files as entries in
> > a dictionary. The file name would be the key and the lines of the file
> > the value.
>
> > Anyone know of a database (with a Python interface) which will allow
> > text files to be treated as database fields? If not I can just write
> > it but I thought it best to ask if there was an existing solution
> > first.
...
> I could be missing something here, but aren't you basically just
> talking about an OS's filesystem?

For storage, yes. The files would be marked-up text stored in the
filesystem. The "dbms" (I use the term loosely!) would provide access
to them by some full or partial key mechanism yet to be determined.
Read-only access would do - at least for now.

> glob or listdir somewhere, then create a dict using the file contents
> would meet your criteria, with very little lines of code -- but I
> would be interested to know what the use-case was for this... Is it
> read completely at start up time, or if each file contains a large
> amount of lines and aren't fixed width (or has no other indexing
> support without maintenance), then is a complete sequential-scan
> required each time, or do you just tell the user to not update when
> running (unless I s'pose something along the lines of a SIGHUP for
> config files is applicable).

All good questions. For now, at least, the files can be read-only and
I'd want those on disk to be the master copies at all times. If I was
writing it myself I'd probably 'cache' some files in memory and stat
them before use. If newer I would reread the file.

>
> Sorry, just don't understand why you'd want this.

I tried to avoid boring folks with the details. I'm toying with some
ideas for a way to help generate source code (in various languages,
not just Python). If it goes ahead the text files would be mainly
marked-up code snippets - with or without symbols that need to be
replaced.

Rather than write one single monolithic app I thought to split it into
reusable components. One part being data access could perhaps be an
existing database (and I'll take a look at jkn's suggestion).

Think of the database as similar to an associative array stored on
disk. The only difference is I may want to play fast and loose with
the keys in some ways - e.g. check for partial key matches or return a
list of part-matched keys. The language name could be part of the key
but I'd also need to store variants for specific language versions.
I'm not sure yet how it will all pan out. As I say, just throwing
around some ideas.

James