Python IO performance?

[Jeff Epler]

On Thu, Jun 05, 2003 at 12:10:47PM +1000, Andrew MacIntyre wrote:
> On Thu, 4 Jun 2003, Bengt Richter wrote:
> 
> > >* Perl uses OS-specific FILE* hacks for speed; Python mostly sticks with
> > >simple, portable calls.
> > IMO if the latter does not result in max disk thoughput for simple IO, there is
> > room for improvement in the OS event/scheduling/dispatching and/or the interface(s) to it.
> > Not that that helps Python per se ;-/
> 
> It is a not uncommon misconception that OS library routines are highly
> optimised for performance.  OS suppliers, particularly those supporting
> multiple archictectures, frequently prefer portability over performance.
> Even within an architecture, say i386, maximising performance requires
> CPU specific tricks, and in a binary distribution some reasonable
> compromise must be reached to perform satisfactorily across a range of
> hardware.
> 
> So application developers who depend on maximising some aspect of system
> performance need to take matters into their own hands, as Perl does.

.. another reason is that the OS I/O library must perform correctly
under the range of circumstances described by whatever standard they
conform to (eg POSIX, SuS, win32) under various circumstances
(multithreading, binary-level structure compatibility),
but the program-specific I/O library generally has fewer such
requirements.

Jeff