Storing a big amount of path names
Tim Chase
python.list at tim.thechases.com
Thu Feb 11 20:13:32 EST 2016
On 2016-02-12 00:31, Paulo da Silva wrote:
> What is the best (shortest memory usage) way to store lots of
> pathnames in memory where:
>
> 1. Path names are pathname=(dirname,filename)
> 2. There many different dirnames but much less than pathnames
> 3. dirnames have in general many chars
>
> The idea is to share the common dirnames.
Well, you can create a dict that has dirname->list(filenames) which
will reduce the dirname to a single instance. You could store that
dict in the class, shared by all of the instances, though that starts
to pick up a code-smell.
But unless you're talking about an obscenely large number of
dirnames & filenames, or a severely resource-limited machine, just
use the default built-ins. If you start to push the boundaries of
system resources, then I'd try the "anydbm" module or use the
"shelve" module to marshal them out to disk. Finally, you *could*
create an actual sqlite database on disk if size really does exceed
reasonable system specs.
-tkc
More information about the Python-list
mailing list