On donderdag, nov 28, 2002, at 05:16 Europe/Amsterdam, Guido van Rossum wrote:

- test_largefile takes a *very* long time. Perhaps it actually creates a truly large file (the Mac OSX filesystem is case insensitive, so I suppose it may be so different that it doesn't support files with holes in them). Maybe the test should disable itself (or part of itself) unless a specific resource is requested?

Do you mean that on other systems it does *not* create these gigantic files??!? I've always wondered what the use was... Hmm, if this test test support for files with holes, how come that Windows then doesn't have the same problem as the Mac, it also doesn't support holes in files, or does it? -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Jack Jansen:

I've always wondered what the use was... Hmm, if this test test support for files with holes, how come that Windows then doesn't have the same problem as the Mac, it also doesn't support holes in files, or does it?

Martin v. Löwis: # Depends on the file system; NTFS surely does. # # However, on Windows, the test is not run unless the largefile resource # is given to regrtest.py. NTFS has only supported sparse files since Windows 2000 and then only if you set the sparse file attribute. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base /sparse_files.asp Neil

[Jack Jansen]

Do you mean that on other systems it does *not* create these gigantic files??!? I've always wondered what the use was... Hmm, if this test test support for files with holes,

No, it's testing support for large files period. If the OS supports large files, and you've got enough disk space, it should pass regardless of whether the filesystem sticks holes in the middle.

how come that Windows then doesn't have the same problem as the Mac, it also doesn't support holes in files, or does it?

Holes are irrelevant to correct functioning. On Win2K the test passes and does indeed take a very long time. On Win9X the test passes but takes very little time. The difference seems to be that Win2K actually fills a file with gigabytes of NUL characters, but on Win9X you effectively get a window onto whatever happened to be sitting on disk in the blocks allocated for the file. But in neither case are there holes. Here on Win98SE:

...

...

f = file('temp.dat', 'wb') f.seek(1000000) f.write('abc') f.close() f = file('temp.dat', 'rb') guts = f.read() len(guts) 1000003 guts[50000:50050] '|\x88\x08\x004\x0e6\x0e7\x0e8\x0e:\x0e;\x0eQ\x0e=\x0e>\x0e?\x0e@\x0eA\x0eB \x0eD\x0eE\x0eF\x0eH\x0eI\x0eJ\x0eQ\x0eL\x0eM\x0eQ\x0e'

You could very well find personal info in there! Win2K prevents that.

...

...

...

...

"GvR" == Guido van Rossum writes:

GvR> - test_re crashes unless I do ulimit -s 2000 -- haven't tried GvR> other values, but the default of 512 (KB) is insufficient. I GvR> know the README file explains this, but now that I've GvR> experienced this myself, I wonder if we shouldn't hack main() GvR> to increase the stack size to 2 MB, inside an #ifdef darwin GvR> or something like that. +1, but I'll defer to Jack. We discussed this a while back (for Py2.2.2 IIRC), which is when I updated the README bit. FWIW, Pipermail (in Mailman) suffers the same fate and contains the following bit of code. -Barry -------------------- snip snip -------------------- # MacOSX has a default stack size that is too small for deeply recursive # regular expressions. We see this as crashes in the Python test suite when # running test_re.py and test_sre.py. The fix is to set the stack limit to # 2048; the general recommendation is to do in the shell before running the # test suite. But that's inconvenient for a daemon like the qrunner. # # AFAIK, this problem only affects the archiver, so we're adding this work # around to this file (it'll get imported by the bundled pipermail or by the # bin/arch script. We also only do this on darwin, a.k.a. MacOSX. if sys.platform == 'darwin': try: import resource except ImportError: pass else: soft, hard = resource.getrlimit(resource.RLIMIT_STACK) newsoft = min(hard, max(soft, 1024*2048)) resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))

GvR> - test_re crashes unless I do ulimit -s 2000 -- haven't tried GvR> other values, but the default of 512 (KB) is insufficient. I GvR> know the README file explains this, but now that I've GvR> experienced this myself, I wonder if we shouldn't hack main() GvR> to increase the stack size to 2 MB, inside an #ifdef darwin GvR> or something like that.

[Barry]

+1, but I'll defer to Jack. We discussed this a while back (for Py2.2.2 IIRC), which is when I updated the README bit. FWIW, Pipermail (in Mailman) suffers the same fate and contains the following bit of code.

Thanks! I've added your code to regrtest.py. I'm reluctant to do this in PyMain(), because I don't know whether this adds to Python's resource usage; the re tests are pretty extreme. I'm also considering disabling test_largefile on Mac OSX unless a -u option is given. I'm still seeing errors in test_socket; today (on a different network) I got this: test test_socket failed -- Traceback (most recent call last): File "/Users/guido/python/src/Lib/test/test_socket.py", line 223, in testHostnameRes ip = socket.gethostbyname(hostname) gaierror: (7, 'No address associated with nodename') --Guido van Rossum (home page: http://www.python.org/~guido/)

On Monday, Dec 2, 2002, at 11:30 Europe/Amsterdam, Guido van Rossum wrote:

I vaguely recall that the number 50000 is somewhat historically significant, but I don't remember why. But it works on all other platforms I know.

Except all other platforms with small stacks (MacOS9, WinCE, probably more).

Do you know if setting the stack limit actually allocates that much memory for the stack in the process in Mac OSX, or does it only reserve VM space (like on traditional Unix)?

I don't know, but I can't imagine it does anything more than reserve VM space. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Guido> I'm still seeing errors in test_socket; today (on a different Guido> network) I got this: Guido> test test_socket failed -- Traceback (most recent call last): Guido> File "/Users/guido/python/src/Lib/test/test_socket.py", line 223, in testHostnameRes Guido> ip = socket.gethostbyname(hostname) Guido> gaierror: (7, 'No address associated with nodename') What does the hostname(1) command return? By default I think it generates some oddball thing like "Montanaro.local.". Apple calls it a Rendevous name. (I've no idea what that is.) The fix for me was recommended by another group: get a dyndns.org account and change the HOSTNAME line in /etc/hostconfig from HOSTNAME=-AUTOMATIC- to HOSTNAME=montanaro.dyndns.org Skip

[Skip Montanaro]

What does the hostname(1) command return? By default I think it generates some oddball thing like "Montanaro.local.". Apple calls it a Rendevous name. (I've no idea what that is.) The fix for me was recommended by

Rendevous is Apple's implementation of IEEE's zero-conf. The "oddball" name is what is used for the local DNS entries used by Rendevous since it stores info by name instead of IP. -Brett

...

...

...

...

"GvR" == Guido van Rossum writes:

GvR> Thanks, Skip, but I'm more interested in fixing test_socket GvR> so it doesn 't rely on this. Maybe testHostnameRes() should GvR> be disabled when gethostbyaddr() gives an error? It makes GvR> unwarranted assumptions IMO about how DNS is set up. BTW, test_socket.py passes just fine for me. hostname(1) returns "freewill.local." (yes, trailing dot). The difference may be that I used Netinfo Manager to add hostname entries for all my local machines, including the MacOSX machine I ran the tests on. My Netinfo-fu is pretty meager so I don't know if I did it completely right, but it seems to work for me. -Barry