how about adding ping's uuid module to the standard lib ?
see subject and http://python.org/sf/1368955 comments ? </F> (note sure how my previous attempt to post this ended up on comp.lang.python instead, but I'll blame it on gmane... ;-)
At 06:29 AM 3/7/2006 +0100, Fredrik Lundh wrote:
see subject and http://python.org/sf/1368955
comments ?
Why can't the UUIDs be immutable, so they can be dictionary keys? Also, it would be nice if you could just call UUID() to create a generic UUID in a platform-appropriate way. PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.: http://svn.eby-sarna.com/PEAK/src/peak/util/_uuidgen.c?view=markup and: http://svn.eby-sarna.com/PEAK/src/peak/util/uuid.py?view=markup for details. This implementation also has unit tests: http://svn.eby-sarna.com/PEAK/src/peak/util/tests/uuid.py?view=markup and has been in production use for several years now. On the other hand, Ping's implementation is a lot simpler than ours; ours does a lot of stuff to support e.g. /dev/urandom or fallback seeding, whereas Python 2.4 offers os.urandom. We also use different fallback algorithms for getting the MAC address, including a fallback to a randomly-generated address if a system one could not be obtained. Ping's version will fail if there is no way to get the address (e.g. on Windows 9X, which has no ipconfig.exe), and does not cache the address for repeated uses, making its generation of verion-1 UUIDs *very* expensive. I like the idea of having RFC-compliant UUIDs in the stdlib, but I really want immutable ones, preferably without {} in their standard string representation. And efficient use of platform-local UUID generation APIs would also be preferable. (Note that using the Windows C API for UUIDs would make it unnecessary to execute a separate program in order to get version 1 UUIDs, and the same is true for the BSD variants with a UUID API.) (In addition to the PEAK UUID implementation, OSAF's Chandler also has a compliant UUID implementation that's written entirely in C, but I haven't played with it much. I don't know whether it supports using platform APIs for generation, but it does implement hashable, immutable UUID objects.)
On 3/7/06, Phillip J. Eby <pje@telecommunity.com> wrote:
At 06:29 AM 3/7/2006 +0100, Fredrik Lundh wrote:
see subject and http://python.org/sf/1368955
comments ?
would be nice if you could just call UUID() to create a generic UUID in a platform-appropriate way. PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.:
Given that ctypes is going to be in Python 2.5, it might be reasonable to use it to access platform-specific APIs like this. It certainly makes some sense for Windows, where running an external process is slow, and the existence of suitable external utilities (and the format of their output!) is unreliable... (I haven't looked at the code, so I don't know if this applies to Ping's use of ipconfig). Paul
Paul Moore wrote:
On 3/7/06, Phillip J. Eby <pje@telecommunity.com> wrote:
see subject and http://python.org/sf/1368955
comments ? would be nice if you could just call UUID() to create a generic UUID in a
At 06:29 AM 3/7/2006 +0100, Fredrik Lundh wrote: platform-appropriate way. PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.:
Given that ctypes is going to be in Python 2.5, it might be reasonable to use it to access platform-specific APIs like this. It certainly makes some sense for Windows, where running an external process is slow, and the existence of suitable external utilities (and the format of their output!) is unreliable... (I haven't looked at the code, so I don't know if this applies to Ping's use of ipconfig).
IMO that is a perfect use-case for ctypes - access 2 or 3 platform-specific api functions. Too bad that ctypes whill be an optional module, so I'm not sure if it could be used in the Python library itself. Thomas
On Tue, Mar 07, 2006 at 10:19:03PM +0100, Thomas Heller wrote:
Too bad that ctypes whill be an optional module, so I'm not sure if it could be used in the Python library itself.
try: import ctypes except ImportError: def fallback(): ... else: def real_thing(): ... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
On Tue, Mar 07, 2006 at 10:19:03PM +0100, Thomas Heller wrote:
Too bad that ctypes whill be an optional module, so I'm not sure if it could be used in the Python library itself.
try: import ctypes except ImportError: def fallback(): ... else: def real_thing(): ...
Personally, I don't like standard modules to behave different if I install other stuff, so please change 'could' to 'should' in this sentence:
not sure if it could be used in the Python library itself.
Having UUID in the stdlib would be very helpful. Philip Eby writes:
I like the idea of having RFC-compliant UUIDs in the stdlib, but I really want immutable ones, preferably without {} in their standard string representation. And efficient use of platform-local UUID generation APIs would also be preferable.
I completely agree with Philip. Bill
Quite a few people have expressed interest in having UUID functionality in the standard library, and previously on this list some suggested possibly using the uuid.py module i wrote: http://zesty.ca/python/uuid.py This module provides a UUID class, functions for generating version 1, 3, 4, and 5 UUIDs, and can convert UUIDs to and from strings of 32 hex digits and strings of 16 bytes. The thread of messages posted on python-dev begins here: http://mail.python.org/pipermail/python-dev/2006-March/062119.html My reading of this is that there was a pretty good consensus on the issues with this module that need to be addressed: (a) UUIDs should be immutable (and usable as dict keys). (b) The str() of a UUID should not contain curly braces. (c) The uuid1() function, which generates a version-1 UUID, uses a very slow method of getting the MAC address. (d) The retrieved MAC address should be cached. (e) Tests need to be written. The big question seems to be item (c); all the other items are easy to take care of. Assuming (a), (b), (d), and (e) are done, i see a few options for how to proceed from there: 1. Remove the uuid1() function, then put uuid.py in the standard library so at least we'll have the rest of the UUID functionality in 2.5b1. Fill in uuid1() later. 2. Remove the MAC-address functionality from uuid.py; instead let the caller give the MAC address as an argument to uuid1(). Put that in the standard library for 2.5b1 and fill in the function for retrieving the MAC address later. 3. Add uuid.py to the standard library with its current slow method of finding the MAC address (parsing the output of ifconfig or ipconfig), but cache the output so it's only slow the first time. 4. Figure out how to use ctypes to retrieve the MAC address. This would only work on certain platforms, but we could probably cover the major ones. On the other hand, it seems unlikely that this would be fully hammered out before the code freeze. 5. Don't include any UUID functionality in 2.5b1; put it off until 2.6. What are your thoughts on this? Thanks! -- ?!ng
Ka-Ping Yee wrote:
Quite a few people have expressed interest in having UUID functionality in the standard library, and previously on this list some suggested possibly using the uuid.py module i wrote:
+1!
This module provides a UUID class, functions for generating version 1, 3, 4, and 5 UUIDs, and can convert UUIDs to and from strings of 32 hex digits and strings of 16 bytes.
The thread of messages posted on python-dev begins here:
http://mail.python.org/pipermail/python-dev/2006-March/062119.html
My reading of this is that there was a pretty good consensus on the issues with this module that need to be addressed:
(a) UUIDs should be immutable (and usable as dict keys).
(b) The str() of a UUID should not contain curly braces.
(c) The uuid1() function, which generates a version-1 UUID, uses a very slow method of getting the MAC address.
(d) The retrieved MAC address should be cached.
(e) Tests need to be written.
(f) "import uuid" or "import uuidlib" ?
The big question seems to be item (c); all the other items are easy to take care of. Assuming (a), (b), (d), and (e) are done, i see a few options for how to proceed from there:
1. Remove the uuid1() function, then put uuid.py in the standard library so at least we'll have the rest of the UUID functionality in 2.5b1. Fill in uuid1() later.
2. Remove the MAC-address functionality from uuid.py; instead let the caller give the MAC address as an argument to uuid1(). Put that in the standard library for 2.5b1 and fill in the function for retrieving the MAC address later.
3. Add uuid.py to the standard library with its current slow method of finding the MAC address (parsing the output of ifconfig or ipconfig), but cache the output so it's only slow the first time.
4. Figure out how to use ctypes to retrieve the MAC address. This would only work on certain platforms, but we could probably cover the major ones. On the other hand, it seems unlikely that this would be fully hammered out before the code freeze.
5. Don't include any UUID functionality in 2.5b1; put it off until 2.6.
6. Combine 2 and 3: require the user to pass in a MAC argument to uuid1, but provide a SlowGetMacAddress helper that wraps the existing code. </F>
Fredrik> 6. Combine 2 and 3: require the user to pass in a MAC argument Fredrik> to uuid1, but provide a SlowGetMacAddress helper that wraps Fredrik> the existing code. Or make the MAC address an optional arg to uuid1. If given, use it. If not, use the slow lookup (then cache the result). Skip
skip@pobox.com wrote:
Fredrik> 6. Combine 2 and 3: require the user to pass in a MAC argument Fredrik> to uuid1, but provide a SlowGetMacAddress helper that wraps Fredrik> the existing code.
Or make the MAC address an optional arg to uuid1. If given, use it. If not, use the slow lookup (then cache the result).
the reason for making it a required argument is to make it clear that the code is using a "suboptimal" way to get at the MAC value. explicit is better than implicit etc etc. </F>
Fredrik> the reason for making it a required argument is to make it Fredrik> clear that the code is using a "suboptimal" way to get at the Fredrik> MAC value. Fredrik> explicit is better than implicit etc etc. Can't we expect there will be a faster way to get at the MAC address at some point in the future, maybe via a _uuid extension module that does all the magic in C? Or is there something inherently slow in discovering a machine's MAC address (I realize such a task would probably be quite platform-depenedent). Skip
On 6/9/06, Fredrik Lundh <fredrik@pythonware.com> wrote:
6. Combine 2 and 3: require the user to pass in a MAC argument to uuid1, but provide a SlowGetMacAddress helper that wraps the existing code.
That sounds like the right thing to do, although I wouldn't call it "slow"; just let it be documented as 'might not always work and might be inefficient', so there's room to make it the perfect and preferred way to get a MAC address without having to rename it. (Not that I think that's a reliable prospect, but hey ;) -- Thomas Wouters <thomas@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
Okay. I have done as Fredrik suggested:
6. Combine 2 and 3: require the user to pass in a MAC argument to uuid1, but provide a SlowGetMacAddress helper that wraps the existing code.
I agree with Thomas Wouters:
That sounds like the right thing to do, although I wouldn't call it "slow"; just let it be documented as 'might not always work and might be inefficient',
The method for getting the node ID is called getnode() and is documented as possibly slow. If a hardware address cannot be obtained, we use a random 48-bit number with the highest bit set to 1, as recommended by RFC 4122. I have done as Skip proposed here:
Or make the MAC address an optional arg to uuid1. If given, use it. If not, use the slow lookup (then cache the result).
I've made the address an optional argument, not a required one, because i think "Explicit better than implicit" applies to changes in output. It makes sense to require an explicit argument if it's actually going to produce a different result, but i don't think it is necessary to require an explicit argument just because the routine might be a bit slow. Letting the address be an optional argument means that in future, we can change the implementation of getnode() to make it faster or more reliable, and users of the module will benefit without having to change any of their code. Finally, Phillip brought up PEAK:
PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK. Oh, and there is now a test suite that should cover all the code paths. This is all posted at http://zesty.ca/python/uuid.py now, documentation page at http://zesty.ca/python/uuid.html, tests at http://zesty.ca/python/test_uuid.py . I'll sleep now (6 am), commit tomorrow unless there are objections. Thanks for your input, everyone! -- ?!ng
On 6/10/06, Ka-Ping Yee <python-dev@zesty.ca> wrote:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK.
Instead of using pywin32, could you use ctypes, as that's part of core Python? It looks like the only Win32 API you use is CoCreateGUID, so wrapping it should be doable... Paul.
On 6/10/06, Paul Moore <p.f.moore@gmail.com> wrote:
On 6/10/06, Ka-Ping Yee <python-dev@zesty.ca> wrote:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK.
Instead of using pywin32, could you use ctypes, as that's part of core Python? It looks like the only Win32 API you use is CoCreateGUID, so wrapping it should be doable...
Here's some sample code (taken from Thomas Heller's comtypes module)
from ctypes import oledll, Structure, byref from ctypes.wintypes import BYTE, WORD, DWORD class GUID(Structure): ... _fields_ = [("Data1", DWORD), ... ("Data2", WORD), ... ("Data3", WORD), ... ("Data4", BYTE * 8)] ... guid = GUID() oledll.ole32.CoCreateGuid(byref(guid)) 0 guid <__main__.GUID object at 0x00978EE0> guid.Data1 3391869098L guid.Data2 51115 guid.Data3 20060 guid.Data4 <__main__.c_byte_Array_8 object at 0x00978E40>
I'm not sure what the int(...[-13,-1], 16) does (as it stands, it fails for me - I think you need to put str() round the CreateGuid call) but I *think* it's the equivalent of guid.Data1 above. So, you'd have: def win32_getnode(): """Get the hardware address on Windows using Win32 extensions.""" from ctypes import oledll, Structure, byref from ctypes.wintypes import BYTE, WORD, DWORD class GUID(Structure): _fields_ = [("Data1", DWORD), ("Data2", WORD), ("Data3", WORD), ("Data4", BYTE * 8)] guid = GUID() oledll.ole32.CoCreateGuid(byref(guid)) return guid.Data1 Hope this is of use. Paul.
Paul Moore wrote:
On 6/10/06, Ka-Ping Yee <python-dev@zesty.ca> wrote:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK.
Instead of using pywin32, could you use ctypes, as that's part of core Python? It looks like the only Win32 API you use is CoCreateGUID, so wrapping it should be doable...
http://docs.python.org/dev/lib/module-msilib.html#l2h-5633 Regards, Martin
Ka-Ping Yee wrote:
Finally, Phillip brought up PEAK:
PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK.
Oh, and there is now a test suite that should cover all the code paths.
This is all posted at http://zesty.ca/python/uuid.py now, documentation page at http://zesty.ca/python/uuid.html, tests at http://zesty.ca/python/test_uuid.py .
(From http://zesty.ca/python/uuid.py:) def win32_getnode(): """Get the hardware address on Windows using Win32 extensions.""" try: import pywintypes return int(pywintypes.CreateGuid()[-13:-1], 16) except: pass try: import pythoncom return int(pythoncom.CreateGuid()[-13:-1], 16) except: pass This does not work, for several reasons. 1. (pythoncom|pywintypes).CreateGuid() return a PyIID instance, which you cannot slice:
import pywintypes pywintypes.CreateGuid() IID('{4589E547-4CB5-4BCC-A7C3-6E88FAFB4301}') type(pywintypes.CreateGuid()) <type 'PyIID'> pywintypes.CreateGuid()[-13:-1] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: unsubscriptable object
So, you would have to change your code to 'str(pythoncom.CreateGuid())[-13:-1]'. (BTW: Why does it first try pywintypes, the pythoncom?) 2. These functions call to win32 CoCreateGuid function, which create a new uuid. As documented on MSDN, this calls the UuidCreate function which: "The UuidCreate function generates a UUID that cannot be traced to the ethernet/token ring address of the computer on which it was generated." In other words, the last 12 characters do *not* identify the mac address, in fact your function, if repaired, returns a different result each time it is called. See this link for further info: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/rpc/rpc/uui... A possible solution, if you really need the mac address, is to call the UuidCreateSequential function where available (W2k, XP, ...), and UuidCreate on older systems (W98, ME). Link to MSDN: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/rpc/rpc/uui... 3. Now that ctypes is part of Python, why not use it? This snippet of code displays the mac address of one of the ethernet adapters I have in my machine: from ctypes import * import binascii class UUID(Structure): _fields_ = [("Data1", c_ulong), ("Data1", c_ushort), ("Data1", c_ushort), ("Data1", c_ubyte * 8)] def CreateGuid(): uuid = UUID() if 0 == windll.rpcrt4.UuidCreateSequential(byref(uuid)): return str(buffer(uuid)) print binascii.hexlify(CreateGuid()[-6:]) It should be extended to also work on w98 or me, probably. Thomas
Hi Thomas,
This does not work, for several reasons.
1. (pythoncom|pywintypes).CreateGuid() return a PyIID instance, which you cannot slice:
You're right. The PEAK code must have been based on an earlier version of the CreateGuid() routine. I've fixed this, and added detection of the UUID version so that the MAC address will only be picked up if the UUID version is 1.
(BTW: Why does it first try pywintypes, the pythoncom?)
Because PEAK does this, and i see the CreateGuid routine imported from both modules in Google searches for code, i assumed that it used to live in one module and moved to the other. I've also figured out how to get the MAC address using NetBIOS calls, and added that to the repertoire of methods. I've tested this and it works fast. I think this may be better than UuidCreateSequential, because it should work on both Win98 and XP. The updated version is posted at http://zesty.ca/python/uuid.py . -- ?!ng
Ka-Ping Yee wrote:
Hi Thomas,
This does not work, for several reasons.
1. (pythoncom|pywintypes).CreateGuid() return a PyIID instance, which you cannot slice:
You're right. The PEAK code must have been based on an earlier version of the CreateGuid() routine.
I've fixed this, and added detection of the UUID version so that the MAC address will only be picked up if the UUID version is 1.
(BTW: Why does it first try pywintypes, the pythoncom?)
Because PEAK does this, and i see the CreateGuid routine imported from both modules in Google searches for code, i assumed that it used to live in one module and moved to the other.
I've also figured out how to get the MAC address using NetBIOS calls, and added that to the repertoire of methods. I've tested this and it works fast. I think this may be better than UuidCreateSequential, because it should work on both Win98 and XP.
I have not tested the speed, but extending my snippet to also work on 98 should be nearly trivial: try: _func = windll.rpcrt4.UuidCreateSequential except AttributeError: _func = windll.rpcrt4.UuidCreate def CreateGuid(): uuid = UUID() if 0 == _func(byref(uuid)): return str(buffer(uuid)) Thomas
On Sat, 10 Jun 2006, Thomas Heller wrote:
I have not tested the speed, but extending my snippet to also work on 98 should be nearly trivial:
try: _func = windll.rpcrt4.UuidCreateSequential except AttributeError: _func = windll.rpcrt4.UuidCreate
def CreateGuid(): uuid = UUID() if 0 == _func(byref(uuid)): return str(buffer(uuid))
Okay. Thanks for this. I'll give it a shot. -- ?!ng
On Sat, 10 Jun 2006, Thomas Heller wrote:
[some nice ctypes code]
Done. Works like a charm. Thanks for providing the code! On Sat, 10 Jun 2006, Phillip J. Eby wrote:
Also, for Python 2.5, these imports could probably be replaced with a ctypes call, though I'm not experienced enough w/ctypes to figure out what the call should be.
Happily, we have *the* ctypes guru here, and he's solved the problem for Windows at least.
Similarly, for the _uuidgen module, you've not included the C source for that module or the setup.py incantations to build it.
Yes, the idea was that even though _uuidgen isn't included with core Python, users would magically benefit if they installed it (or if they happen to be using Python in a distribution that includes it); it's the same idea with the stuff that refers to Win32 extensions. Is the presence of _uuidgen sufficiently rare that i should leave out uuidgen_getnode() for now, then? -- ?!ng
At 11:16 AM 6/10/2006 -0500, Ka-Ping Yee wrote:
On Sat, 10 Jun 2006, Thomas Heller wrote:
[some nice ctypes code]
Done. Works like a charm. Thanks for providing the code!
On Sat, 10 Jun 2006, Phillip J. Eby wrote:
Also, for Python 2.5, these imports could probably be replaced with a ctypes call, though I'm not experienced enough w/ctypes to figure out what the call should be.
Happily, we have *the* ctypes guru here, and he's solved the problem for Windows at least.
Similarly, for the _uuidgen module, you've not included the C source for that module or the setup.py incantations to build it.
Yes, the idea was that even though _uuidgen isn't included with core Python, users would magically benefit if they installed it (or if they happen to be using Python in a distribution that includes it);
_uuidgen is actually peak.util._uuidgen; as far as I know, that's the only place you can get it.
it's the same idea with the stuff that refers to Win32 extensions. Is the presence of _uuidgen sufficiently rare that i should leave out uuidgen_getnode() for now, then?
Either that, or we could add the code in to build it. PEAK's setup.py does some relatively simple platform checks to determine whether you're on a BSD that has it. The other alternative is to ask the guru nicely if he'll provide another ctypes snippet to call the uuidgen(2) system call if present. :) By the way, I'd love to see a uuid.uuid() constructor that simply calls the platform-specific default UUID constructor (CoCreateGuid or uuidgen(2)), if available, before falling back to one of the Python implementations. Most of my UUID-using application code doesn't really care what type of UUID it gets, and if the platform has an efficient mechanism, it'd be convenient to use it.
Thomas Heller wrote:
I don't know if this is the uuidgen you're talking about, but on linux there is libuuid:
Thanks! Okay, that's in there now. Have a look at http://zesty.ca/python/uuid.py . Phillip J. Eby wrote:
By the way, I'd love to see a uuid.uuid() constructor that simply calls the platform-specific default UUID constructor (CoCreateGuid or uuidgen(2)),
I've added code to make uuid1() use uuid_generate_time() if available and uuid4() use uuid_generate_random() if available. These functions are provided on Mac OS X (in libc) and on Linux (in libuuid). Does that work for you? I'm using the Windows UUID generation calls (UuidCreate and UuidCreateSequential in rpcrt4) only to get the hardware address, not to make UUIDs, because they yield results that aren't compliant with RFC 4122. Even worse, they actually have the variant bits set to say that they are RFC 4122, but they can have an illegal version number. If there are better alternatives on Windows, i'm happy to use them. -- ?!ng
At 07:24 PM 6/11/2006 -0500, Ka-Ping Yee wrote:
Thomas Heller wrote:
I don't know if this is the uuidgen you're talking about, but on linux there is libuuid:
Thanks!
Okay, that's in there now. Have a look at http://zesty.ca/python/uuid.py .
Phillip J. Eby wrote:
By the way, I'd love to see a uuid.uuid() constructor that simply calls the platform-specific default UUID constructor (CoCreateGuid or uuidgen(2)),
I've added code to make uuid1() use uuid_generate_time() if available and uuid4() use uuid_generate_random() if available. These functions are provided on Mac OS X (in libc) and on Linux (in libuuid). Does that work for you?
Sure - but actually my main point was to have a uuid() call you could use to just get whatever the platform's preferred form of GUID is, without having to pick what *type* you want. The idea being that there should be some call you can make that will always give you something reasonably unique, without being overspecified as to the type of uuid. That way, people can be told to use uuid.uuid() to get unique IDs for use in their programs, without having to get into what types of UUIDs do what. Perhaps that isn't feasible, or is a bad idea for some other reason, but my main point was to have a call that means "get me a good unique ID". :)
On Sun, 11 Jun 2006, Phillip J. Eby wrote:
At 07:24 PM 6/11/2006 -0500, Ka-Ping Yee wrote:
I've added code to make uuid1() use uuid_generate_time() if available and uuid4() use uuid_generate_random() if available. These functions are provided on Mac OS X (in libc) and on Linux (in libuuid). Does that work for you?
Sure - but actually my main point was to have a uuid() call you could use to just get whatever the platform's preferred form of GUID is, without having to pick what *type* you want.
I'm reluctant to do that, because there's a privacy question here. I think the person using the module should have control over whether the UUID is going to leak the host ID or not (rather than leaving it up to whatever the platform prefers or which call the implementor of uuid.py happened to choose for a given platform).
Perhaps that isn't feasible, or is a bad idea for some other reason, but my main point was to have a call that means "get me a good unique ID". :)
Couldn't we just recommend uuid.uuid4() for that? -- ?!ng
At 08:22 AM 6/10/2006 -0500, Ka-Ping Yee wrote:
Finally, Phillip brought up PEAK:
PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK.
There appears to be a transcription error, there; the second win32 import isn't covered by a try/except and the ImportError seems to have disappeared as well. Also, for Python 2.5, these imports could probably be replaced with a ctypes call, though I'm not experienced enough w/ctypes to figure out what the call should be. Similarly, for the _uuidgen module, you've not included the C source for that module or the setup.py incantations to build it. But again, it could probably be replaced by ctypes calls to uuidgen(2) on BSD-ish platforms. I'll take a whack at addressing these once the code is in, though, unless there's a ctypes guru or two available...?
Phillip J. Eby wrote:
At 08:22 AM 6/10/2006 -0500, Ka-Ping Yee wrote:
Finally, Phillip brought up PEAK:
PEAK's uuid module does this such that if win32all is present, you get a Windows GUID, or if you have a FreeBSD 5+ or NetBSD 2+ kernel you use the local platform uuidgen API. See e.g.:
...so i looked at PEAK's getnodeid48() routine and borrowed the Win32 calls from there, with a comment giving attribution to PEAK.
There appears to be a transcription error, there; the second win32 import isn't covered by a try/except and the ImportError seems to have disappeared as well.
Also, for Python 2.5, these imports could probably be replaced with a ctypes call, though I'm not experienced enough w/ctypes to figure out what the call should be.
Similarly, for the _uuidgen module, you've not included the C source for that module or the setup.py incantations to build it. But again, it could probably be replaced by ctypes calls to uuidgen(2) on BSD-ish platforms.
I'll take a whack at addressing these once the code is in, though, unless there's a ctypes guru or two available...?
I don't know if this is the uuidgen you're talking about, but on linux there is libuuid:
from ctypes import * lib = CDLL("libuuid.so.1") uuid = create_string_buffer(16) lib.uuid_generate(byref(uuid)) 2131088494 from binascii import hexlify hexlify(buffer(uuid)) '0c77b6d7e5f940b18e29a749057f6ed4'
At 06:39 PM 6/10/2006 +0200, Thomas Heller wrote:
I don't know if this is the uuidgen you're talking about, but on linux there is libuuid:
from ctypes import * lib = CDLL("libuuid.so.1") uuid = create_string_buffer(16) lib.uuid_generate(byref(uuid)) 2131088494 from binascii import hexlify hexlify(buffer(uuid)) '0c77b6d7e5f940b18e29a749057f6ed4'
Nice. :) But no, this one's a uuidgen() system call on FreeBSD>=5.0 and NetBSD>=2.0; it may be in other BSD variants as well... perhaps OS X? NAME uuidgen -- generate universally unique identifiers LIBRARY Standard C Library (libc, -lc) SYNOPSIS #include <sys/types.h> #include <sys/uuid.h> int uuidgen(struct uuid *store, int count); DESCRIPTION The uuidgen() system call generates count universally unique identifiers (UUIDs) and writes them to the buffer pointed to by store. The identi- fiers are generated according to the syntax and semantics of the DCE ver- sion 1 variant of universally unique identifiers. See below for a more in-depth description of the identifiers. When no IEEE 802 address is available for the node field, a random multi-cast address is generated for each invocation of the system call. According to the algorithm of generating time-based UUIDs, this will also force a new random clock sequence, thereby increasing the likelihood for the identifier to be unique. When multiple identifiers are to be generated, the uuidgen() system call will generate a set of identifiers that is dense in such a way that there is no identifier that is larger than the smallest identifier in the set and smaller than the largest identifier in the set and that is not already in the set. Universally unique identifiers, also known as globally unique identifiers (GUIDs), have a binary representation of 128-bits. The grouping and meaning of these bits is described by the following structure and its description of the fields that follow it: struct uuid { uint32_t time_low; uint16_t time_mid; uint16_t time_hi_and_version; uint8_t clock_seq_hi_and_reserved; uint8_t clock_seq_low; uint8_t node[_UUID_NODE_LEN]; }; time_low The least significant 32 bits of a 60-bit timestamp. This field is stored in the native byte-order. time_mid The least significant 16 bits of the most sig- nificant 28 bits of the 60-bit timestamp. This field is stored in the native byte-order. time_hi_and_version The most significant 12 bits of the 60-bit timestamp multiplexed with a 4-bit version number. The version number is stored in the most significant 4 bits of the 16-bit field. This field is stored in the native byte-order. clock_seq_hi_and_reserved The most significant 6 bits of a 14-bit sequence number multiplexed with a 2-bit vari- ant value. Note that the width of the variant value is determined by the variant itself. Identifiers generated by the uuidgen() system call have variant value 10b. the variant value is stored in the most significant bits of the field. clock_seq_low The least significant 8 bits of a 14-bit sequence number. node The 6-byte IEEE 802 (MAC) address of one of the interfaces of the node. If no such inter- face exists, a random multi-cast address is used instead. The binary representation is sensitive to byte ordering. Any multi-byte field is to be stored in the local or native byte-order and identifiers must be converted when transmitted to hosts that do not agree on the byte-order. The specification does not however document what this means in concrete terms and is otherwise beyond the scope of this system call. RETURN VALUES Upon successful completion, the value 0 is returned; otherwise the value -1 is returned and the global variable errno is set to indicate the error. ERRORS The uuidgen() system call can fail with: [EFAULT] The buffer pointed to by store could not be written to for any or all identifiers. [EINVAL] The count argument is less than 1 or larger than the hard upper limit of 2048. SEE ALSO uuidgen(1), uuid(3) STANDARDS The identifiers are represented and generated in conformance with the DCE 1.1 RPC specification. The uuidgen() system call is itself not part of the specification. HISTORY The uuidgen() system call first appeared in FreeBSD 5.0 and was subse- quently added to NetBSD 2.0. time_low The least significant 32 bits of a 60-bit timestamp. This field is stored in the native byte-order. time_mid The least significant 16 bits of the most sig- nificant 28 bits of the 60-bit timestamp. This field is stored in the native byte-order. time_hi_and_version The most significant 12 bits of the 60-bit timestamp multiplexed with a 4-bit version number. The version number is stored in the most significant 4 bits of the 16-bit field. This field is stored in the native byte-order. clock_seq_hi_and_reserved The most significant 6 bits of a 14-bit sequence number multiplexed with a 2-bit vari- ant value. Note that the width of the variant value is determined by the variant itself. Identifiers generated by the uuidgen() system call have variant value 10b. the variant value is stored in the most significant bits of the field. clock_seq_low The least significant 8 bits of a 14-bit sequence number. node The 6-byte IEEE 802 (MAC) address of one of the interfaces of the node. If no such inter- face exists, a random multi-cast address is used instead. The binary representation is sensitive to byte ordering. Any multi-byte field is to be stored in the local or native byte-order and identifiers must be converted when transmitted to hosts that do not agree on the byte-order. The specification does not however document what this means in concrete terms and is otherwise beyond the scope of this system call. RETURN VALUES Upon successful completion, the value 0 is returned; otherwise the value -1 is returned and the global variable errno is set to indicate the error. ERRORS The uuidgen() system call can fail with: [EFAULT] The buffer pointed to by store could not be written to for any or all identifiers. [EINVAL] The count argument is less than 1 or larger than the hard upper limit of 2048.
Phillip J. Eby wrote:
At 06:39 PM 6/10/2006 +0200, Thomas Heller wrote:
I don't know if this is the uuidgen you're talking about, but on linux there is libuuid:
from ctypes import * lib = CDLL("libuuid.so.1") uuid = create_string_buffer(16) lib.uuid_generate(byref(uuid)) 2131088494 from binascii import hexlify hexlify(buffer(uuid)) '0c77b6d7e5f940b18e29a749057f6ed4'
Nice. :) But no, this one's a uuidgen() system call on FreeBSD>=5.0 and NetBSD>=2.0; it may be in other BSD variants as well... perhaps OS X?
For completeness :-), although its probably getting off-topic: $ uname -a FreeBSD freebsd 6.0-RELEASE FreeBSD 6.0-RELEASE #0: Thu Nov 3 09:36:13 UTC 2005 root@x64.samsco.home:/usr/obj/usr/src/sys/GENERIC i386 $ python Python 2.4.1 (#2, Oct 12 2005, 01:36:32) [GCC 3.4.4 [FreeBSD] 20050518] on freebsd6 Type "help", "copyright", "credits" or "license" for more information.
from ctypes.util import find_library find_library("c") 'libc.so.6' from ctypes import * libc = CDLL("libc.so.6") uuid = create_string_buffer(16) libc.uuidgen(uuid, 1) 0 uuid[:] '\xd2\xff\x8e\xe3\xa3\xf8\xda\x11\xb0\x04\x00\x0c)\xd1\x18\x06'
$
On OS X, this call is not available, but /usr/lib/libc.dylib has a uuid_generate function which is apparently compatible to the linux example I posted above. Thomas
Thomas> On OS X, this call is not available, but /usr/lib/libc.dylib has Thomas> a uuid_generate function which is apparently compatible to the Thomas> linux example I posted above. Confirmed: % python Python 2.5a2 (trunk:46644M, Jun 4 2006, 10:58:16) [GCC 4.0.0 (Apple Computer, Inc. build 5026)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from ctypes.util import find_library >>> print find_library("c") /usr/lib/libc.dylib >>> from ctypes import * >>> libc = CDLL("libc.dylib") >>> uuid = create_string_buffer(16) >>> libc.uuid_generate(uuid, 1) -1073747536 >>> print repr(uuid[:]) '\x03K\xab\x0e\x96\xba@3\xa5\x9f\x04\x1d\x9b\x91\x08\x1b' Skip
Fredrik Lundh wrote:
Ka-Ping Yee wrote:
Quite a few people have expressed interest in having UUID functionality in the standard library, and previously on this list some suggested possibly using the uuid.py module i wrote:
+1!
+1 as well. I have a couple of suggestions for improving that implementation: 1. You're currently using os.urandom, which can raise a NotImplementedError. You should be prepared to fall back on a different PRNG... which leads to the 2nd suggestion: 2. random.randrange is a method on a default random.Random instance that, although seeded by urandom (if available), may not be the user's preferred PRNG. I recommend making it possible for the user to supply their own random.Random instance for use by the module. That's all. :)
On Sat, 10 Jun 2006, Mike Brown wrote:
I have a couple of suggestions for improving that implementation:
1. You're currently using os.urandom, which can raise a NotImplementedError. You should be prepared to fall back on a different PRNG...
The latest version (http://zesty.ca/python/uuid.py) does this.
2. random.randrange is a method on a default random.Random instance that, although seeded by urandom (if available), may not be the user's preferred PRNG. I recommend making it possible for the user to supply their own random.Random instance for use by the module.
I decided not to add more code to do this, because the UUID constructor is now designed in such a way that it's very simple to convert your own randomness into a UUID. If you want to use another source of randomness, you'd just get 16 random bytes and then call UUID(bytes=random_stuff, version=4). That seems simpler to me than adding an extra argument and requiring a randrange() method on it. -- ?!ng
Ka-Ping Yee <python-dev@zesty.ca> wrote:
Quite a few people have expressed interest in having UUID functionality in the standard library, and previously on this list some suggested possibly using the uuid.py module i wrote:
Some comments on the code:
for dir in ['', r'c:\windows\system32', r'c:\winnt\system32']:
Can we get rid of these absolute paths? Something like this should suffice:
from ctypes import * buf = create_string_buffer(4096) windll.kernel32.GetSystemDirectoryA(buf, 4096) 17 buf.value.decode("mbcs") u'C:\\WINNT\\system32'
for function in functions: try: _node = function() except: continue
This also hides typos and whatnot. I guess it's better if each function catches its own exceptions, and either return None or raise a common exception (like a class _GetNodeError(RuntimeError)) which is then caught. Giovanni Bajo
On Sun, 11 Jun 2006, Giovanni Bajo wrote:
Some comments on the code:
for dir in ['', r'c:\windows\system32', r'c:\winnt\system32']:
Can we get rid of these absolute paths? Something like this should suffice:
from ctypes import * buf = create_string_buffer(4096) windll.kernel32.GetSystemDirectoryA(buf, 4096) 17 buf.value.decode("mbcs") u'C:\\WINNT\\system32'
I'd like to, but i don't want to use a method for finding the system directory that depends on ctypes. Is there a more general way?
for function in functions: try: _node = function() except: continue
This also hides typos and whatnot.
The intended semantics of getnode() are that it cannot fail. The individual *_getnode() functions do throw exceptions if something goes wrong, and so they can be tested individually on platforms where they are expected to work. -- ?!ng
Ka-Ping Yee <python-dev@zesty.ca> wrote:
for dir in ['', r'c:\windows\system32', r'c:\winnt\system32']:
Can we get rid of these absolute paths? Something like this should suffice:
from ctypes import * buf = create_string_buffer(4096) windll.kernel32.GetSystemDirectoryA(buf, 4096) 17 buf.value.decode("mbcs") u'C:\\WINNT\\system32'
I'd like to, but i don't want to use a method for finding the system directory that depends on ctypes.
Why?
Is there a more general way?
GetSystemDirectory() is the official way to find the system directory. You can either access it through ctypes or through pywin32, but I guess we're moving to ctypes for this kind of stuff, since it's bundled in 2.5. I don't know any Python sys/os method to get a pointer to that directory. Another thing that you might do is to drop those absolute system directories altogether. After all, ipconfig should always be in the path. As a last note, you are parsing ipconfig output assuming an English Windows installation. My Italian Windows 2000 has localized output. Giovanni Bajo
On Mon, 12 Jun 2006, Giovanni Bajo wrote:
GetSystemDirectory() is the official way to find the system directory.
You're right. I've added a call to GetSystemDirectory, with a fallback to the usual locations if it doesn't work.
Another thing that you might do is to drop those absolute system directories altogether. After all, ipconfig should always be in the path.
Yup, that's why '' is the first item in the list of directories to try.
As a last note, you are parsing ipconfig output assuming an English Windows installation. My Italian Windows 2000 has localized output.
Thanks for catching this. I've fixed it in the latest version, which is now checked in to the trunk. -- ?!ng
On 6/12/06, Ka-Ping Yee <python-dev@zesty.ca> wrote:
On Sun, 11 Jun 2006, Giovanni Bajo wrote:
Some comments on the code:
for dir in ['', r'c:\windows\system32', r'c:\winnt\system32']:
Can we get rid of these absolute paths? Something like this should suffice:
from ctypes import * buf = create_string_buffer(4096) windll.kernel32.GetSystemDirectoryA(buf, 4096) 17 buf.value.decode("mbcs") u'C:\\WINNT\\system32'
I'd like to, but i don't want to use a method for finding the system directory that depends on ctypes. Is there a more general way?
Why not use ctypes? This is precisely the situation it was designed for. There's nothing more general (this is totally Windows-specific after all). The alternative is to depend on pywin32, which is not part of the core, or to write a custom C wrapper, which seems to me to be precisely what we're trying to move away from... Paul.
Ka-Ping Yee wrote:
I'd like to, but i don't want to use a method for finding the system directory that depends on ctypes. Is there a more general way?
I think os.path.join(os.environ["SystemRoot"], "system32") should be fairly reliable. If people are worried that the directory might not be system32, then HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Windows\SystemDirectory gives you the value (although I could not find out how to expand REG_EXPAND_SZ keys with _winreg). Regards, Martin
participants (12)
-
"Martin v. Löwis" -
Bill Janssen -
Fredrik Lundh -
Giovanni Bajo -
Ka-Ping Yee -
Mike Brown -
Oleg Broytmann -
Paul Moore -
Phillip J. Eby -
skip＠pobox.com -
Thomas Heller -
Thomas Wouters