File IO question.
benn at cenix-bioscience.com
Fri Jul 16 00:07:15 CEST 2004
<snip from above!>
> with handling the size of the trace file.
> If a file's data could be piped to a compressed file
> as the trace was being taken, more comprehensive test
> cases could be run at a single session.
> E.g., a 7 day test would currently be out of the
> question because it generates a 25 gig trace. But if
> the trace file's contents was piped to a compressed file it would only
> end up taking up 2.5 gig of space
> once the test completed.
> Unfortunately my script was unable to process a trace
> file created by the application and the application
> wouldn't write to a file that my script created while
> the script was accessing the file.
<snip from below>
Looking at this you really need to get that data off the
computer - there's no other way around it as you need the data and the
computers are too low capacity to handle all the data.
If these computers are networked can you create a buffer of a few 100
megs and throw the trace data across to another computer which has the
capacity to handle this amount of data. You should be able to do that
by creating a new Handler in logger which keeps trying to pump data down
a socket (or whatever) while storing trace data on the hard drive in
'chucks' (that is assuming your data is being made faster than you can
throw it down a socket - probably isn't though). Think of this as just
a meaty version of normal networking and you may have your answer??
I did this once using a message bus and JBoss (open source J2EE app
server), worked pretty well - just make sure your message server can
handle the load or use federated servers.
Senior Automation Engineer
Tel : +49 (0)351 4173 154
e-mail : benn at cenix-bioscience.com
Cenix Website : http://www.cenix-bioscience.com
More information about the Python-list