On 1/3/07, Fred L. Drake, Jr. wrote:

On Wednesday 03 January 2007 11:06, Martin v. Löwis wrote:

...

In #1626545, Anton Tropashko requests that object.h should be renamed, because it causes conflicts with other software.

I would like to comply with this requests for 2.6, assuming there shouldn't be many problems with existing software as object.h shouldn't be included directly, anyway.

+1

Maybe this should be done in a more systematic fashion? E.g. by giving all "internal" header files a "py_" prefix? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Wednesday 03 January 2007 12:38, Guido van Rossum wrote:

Maybe this should be done in a more systematic fashion? E.g. by giving all "internal" header files a "py_" prefix?

Even better. +42 -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

Guido van Rossum schrieb:

Maybe this should be done in a more systematic fashion? E.g. by giving all "internal" header files a "py_" prefix?

Yet another alternative would be to move all such header files into a py/ directory, so you would refer to them as #include "py/object.h" Any preferences? Regards, Martin

On 1/3/07, "Martin v. Löwis" wrote:

Guido van Rossum schrieb:

...

Maybe this should be done in a more systematic fashion? E.g. by giving all "internal" header files a "py_" prefix?

Yet another alternative would be to move all such header files into a py/ directory, so you would refer to them as

#include "py/object.h"

Any preferences?

Whatever is easier to make this happen. +1 for either solution from me. -Brett

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 3, 2007, at 2:29 PM, Martin v. Löwis wrote:

Guido van Rossum schrieb:

...

Maybe this should be done in a more systematic fashion? E.g. by giving all "internal" header files a "py_" prefix?

Yet another alternative would be to move all such header files into a py/ directory, so you would refer to them as

#include "py/object.h"

Any preferences?

I think I prefer this, although I'd choose "python/object.h" just for explicitness. But if you go with a header prefix, then the shorter "py_" is fine. FWIW, I tried to do a quick grep around some of our code and I found that the only "internal" header we include is structmember.h. Why is that not part of Python.h again? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRZwJ+nEjvBPtnXfVAQL55wP/SxjN9/ncb86KnhRMUf24U6fp+u5JrpN3 irJfi/3tf9iXFtHXPkvkc4hEM9DkF8pa+jYDICG1pZ2J0YQD/AcSuB52WoWDwBtn BIKt1QmJvxgWZLW+dAekWhgSD95laPw+72iCwBvFlIP3+IXtF/Fw9AtuxiGwQzE3 R71j5tZAde4= =nA9B -----END PGP SIGNATURE-----

On 3-Jan-2007, at 23:17 , Gregory P. Smith wrote:

+1 on using the python/*.h subdirectory.

I'm a bit concerned about the "python/*.h": could it cause trouble in combination with Apple's framework naming convention (#include magically gets the header out of the quicktime framework) and the case-insensitiveness of the Mac filesystem? Will #include "python/blabla.h" always find that file along the -I paths, and not somehow accidentally start looking for /Library/Framework/Python.framework/ Headers/blabla.h? -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

On 1/3/07, Thomas Wouters wrote:

On 1/3/07, Guido van Rossum wrote:

...

On 1/3/07, Fred L. Drake, Jr. wrote:

...

On Wednesday 03 January 2007 11:06, Martin v. Löwis wrote:

...

In #1626545, Anton Tropashko requests that object.h should be renamed, because it causes conflicts with other software.

I would like to comply with this requests for 2.6, assuming there shouldn't be many problems with existing software as object.h shouldn't be included directly, anyway.

+1

Maybe this should be done in a more systematic fashion? E.g. by giving all "internal" header files a "py_" prefix?

I was thinking the same, and I'm sure Neal Norwitz is/was too (he suggested this a few times in the past, at least outside of python-dev.)

Wow, I didn't realize I was that much of a broken record. :-) I don't even remember talking to Thomas about it, only Guido. I definitely would like to see all private header files clearly denoted by their name or directory. I saw Jack's comment about Apple's naming scheme, but I'm ignoring that for the moment. I have bad memories from the Motif days of including everything with one file. I prefer to see includes with the directory. This provides a sort of namespace: #include "python/foo.h" Though, if the user only has to include a single Python.h like currently, this wouldn't make as much sense. I don't recall the same problems in Python that existed when using Motif. Here are some options (python/ can be omitted too): #include "python/public.h" #include "python/internal/foo.h" #include "python/private/foo.h" #include "python/_private.h" I don't really like prefixing with py_ because from a user's perspective I interpret py_ to be a public header that gives me a namespace. I prefer a directory that indicates its intent because it can't be misunderstood. IIRC Guido didn't want to introduce a new directory. In that case my second choice is to prefix the filename with an underscore as a leading underscore often is interpreted as private. Going along with this change I would like to see all identifiers in public header files prefixed with [_]Py. And public header files shouldn't be able to include private header files (by convention). Or we should have some other way of denoting which files must prefix all their identifiers and which can use anything because they are only used in Python code. For example, right now node (struct _node) is exported in node.h in 2.4. I think it moved in 2.5, but it's still exported IIRC. n

On 1/3/07, "Martin v. Löwis" wrote:

Neal Norwitz schrieb:

...

Wow, I didn't realize I was that much of a broken record. :-) I don't even remember talking to Thomas about it, only Guido. I definitely would like to see all private header files clearly denoted by their name or directory.

What is a private header file, and does Python have any?

I can see why Modules/sre.h is "private": it won't get installed at all, so users can't include them. For everything in Include, I think users can, and will, include them directly, unless they get them through Python.h.

By private, I mean internal only to python and don't need to prefix their identifiers with Py and are subject to change without backwards compatibility. Include/graminit.h is one example of what I mean. Some others are: bitset.h, grammar.h, opcode.h, metagrammar.h, errcode.h Others are kinda questionable (they have some things that are definitely public, others I'm not so sure about): code.h, parsetok.h, pyarena.h, longintrepr.h, osdefs.h, pgen.h, node.h These are just some examples of what I mean. These may be in include because they are used between two top level directories, but not meant to be exported. There could be other reasons too. n

On 1/3/07, "Martin v. Löwis" wrote:

Neal Norwitz schrieb:

...

By private, I mean internal only to python and don't need to prefix their identifiers with Py and are subject to change without backwards compatibility. Include/graminit.h is one example of what I mean. Some others are: bitset.h, grammar.h, opcode.h, metagrammar.h, errcode.h

Ah. This seems to be a requirement completely different from the one I'm talking about. By this definition, object.h is *not* an internal header file, yet I want it to be renamed.

Agreed. I was mixing two things that aren't necessarily related because I see the same possible solution. I'm also using this one example as a good opportunity to clean up more things. Let me try to explain a bit more below.

As for this issue: how about moving all such private header files out of Include entirely? The parser-related ones should go into Parser, for example (grammar.h, bitset.h, graminit.h, metagrammar.h, errcode.h). This would leave us with opcode.h only.

...

Others are kinda questionable (they have some things that are definitely public, others I'm not so sure about): code.h, parsetok.h, pyarena.h, longintrepr.h, osdefs.h, pgen.h, node.h

Thomas said that at least code.h must stay where it is.

What is the reason that you want them to be renamed?

Sorry, I wasn't trying to imply that these should necessarily be renamed, only that the internal portions be moved elsewhere. I guess I should explain my mental model first which might make things clearer. Then again, I'm tired, so who knows if it will explain anything. :-) I'm a Python embedder and I want to know what's available to me. I look in Include and see a ton of header files. Do I need all these? What do I *need* and what can I *use*? I only want to see the public stuff that is available to me. Thus I want anything that has internal/implementation details/etc out of my sight to reduce my learning curve. I don't ever want to learn about something I won't need nor include anything I won't need. That's one part. Another part of my mental model is that I'm a Python developer and I'm modifying a header file that is implementation specific. I need to share it among different subdiretories (say Python and Objects). So I really need to stick the header file in a common place, Include/ is it. I don't want to export anything, but I don't know if other third party developers will use the header or not. Or maybe I need to include some implementation details in another public header. I'll probably be lazy and just make a single header which has some internal and some public stuff. I want clear rules on when identifiers need to be prefixed. If it's internal (e.g., in an internal directory or prefixed with _), it can have any name and can't be included from any non-internal header. I can also change it in a point release. If anyone uses anything from here, they are on their own. If I see any identifier in a non-internal header, it must be public and therefore prefixed with Py or _Py. The Python headers are pretty good about prefixing most things. But they could be better. I think it gets harder to maintain without the rules though. Finally, by putting everything in directories and always including a directory in the header file, like: #include "python/python.h" #include "python/internal/foo.h" There can never be an include file name collision as what started this thread. It also provides a simple way of demonstrating what's public and what is not. It addresses all my complaints. There are only a few rules and they are simple. But I am addressing several points that are only loosely related which I what I think generated some confusion. Adding the directory also makes clear were the header file comes from. If you see: #include "node.h" you don't know if that's a python node.h, from some other part of the code or a third party library. Not to try to confuse things even more, but I will point out something Google does that is only indirectly related. Google requires importing modules. You aren't supposed to import classes. You wouldn't do: from foo.bar import Message # ... msg = Message(...) You would do: from foo import bar # ... msg = bar.Message(...) This makes it clear where Message comes from, just like adding a python prefix to all header file names makes it clear where the header file lives. Both are good for traceability, though in different ways. This technique makes it easier to manage larger code bases, particularly when there are multiple libraries used. n

Martin v. Löwis wrote:

Neal Norwitz schrieb:

...

Wow, I didn't realize I was that much of a broken record. :-) I don't even remember talking to Thomas about it, only Guido. I definitely would like to see all private header files clearly denoted by their name or directory.

What is a private header file, and does Python have any?

I've a partially related question... why isn't the module structure in an include file .h and is instead in Objects/moduleobject.c ? For the cached lookup optimization I copied the definition but that's surely a bad way to do it.... I however wondered if there were good reasons for module objects for not being published. Andrea

Barry Warsaw schrieb:

I agree. I don't mind if Python.h is just a wrapper around #includes from python/*.h. I think we should add structmember.h and structseq.h to Python.h and perhaps move everything else into a 'python' subdirectory.

For the python subdirectory, there is the issue that the framework includes in OSX magically look for python.framework when searching for python/foo.h, which they find, so that may get us the wrong version. Somebody would have to study the details here, first. Regards, Martin

On 4 Jan, 2007, at 17:56, Fred L. Drake, Jr. wrote:

On Thursday 04 January 2007 11:33, Martin v. Löwis wrote:

...

For the python subdirectory, there is the issue that the framework includes in OSX magically look for python.framework when searching for python/foo.h, which they find, so that may get us the wrong version. Somebody would have to study the details here, first.

If everything public gets included from Python.h, perhaps python/ object.h and friends could become pythonX.Y/object.h; I'm not sure this will solve the Mac OS framework magic issue, though, not being a Mac OS developer.

That would solve the problem, however I don't think there is one. An experiment seems to indicate that the include path is prefered over the magic inclusion of framework headers (see the trace below). I haven't checked yet if the behaviour shown below is intentional, but I'd be surprised if it isn't. $ mkdir include $ mkdir include/Python $ cat > include/Python/object.h < demo.c <<-EOF #include EOF $ $ cc -c demo.c In file included from demo.c:1: /System/Library/Frameworks/Python.framework/Headers/object.h:227: error: parse error before ‘FILE’ /System/Library/Frameworks/Python.framework/Headers/object.h:353: error: parse error before ‘PyType_IsSubtype’ ... more errors removed, this clearly includes a header from the python framework $ $ cc -I include -c t.c In file included from demo.c:1: include/Python/object.h:1:2: error: #error "my error" Therefore moving all headers into a directory named Python would cause no problems users that use the normal way of linking with python and would even allow users to use the python framework as a normal framework. Ronald

-Fred

-- Fred L. Drake, Jr. <fdrake at acm.org> _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ ronaldoussoren%40mac.com

Andrea Griffini schrieb:

I've a partially related question... why isn't the module structure in an include file .h and is instead in Objects/moduleobject.c ? For the cached lookup optimization I copied the definition but that's surely a bad way to do it.... I however wondered if there were good reasons for module objects for not being published.

I guess it's included in the C file because that's the easiest way to implement it. AFAICT, it has been that way from the beginning. Regards, Martin

On 2/25/07, Jeremy Hylton wrote:

On 1/3/07, "Martin v. Löwis" wrote:

...

In #1626545, Anton Tropashko requests that object.h should be renamed, because it causes conflicts with other software.

I would like to comply with this requests for 2.6, assuming there shouldn't be many problems with existing software as object.h shouldn't be included directly, anyway.

I like the idea of renaming the header files, but I expect there is a lot more opportunity for breaking code that you estimate. I did a search on google.com/codesearch and found seven external packages that include Python.h and object.h (before I gave up).

So rather than a simple rename, we should probably rename, change all references in the core to use the new name, and make a simple object.h that only does: #include "new_object.h"

I assume we expect people won't include it, because it is also included in object.h. But they do it. Is there documentation that says you shouldn't import it?

I thought somewhere (embedding/extending doc?) it mentions that only Python.h should be included, but if we have a naming convention/directory structure, this becomes more obvious. Doc/ext/extending.tex: To support extensions, the Python API (Application Programmers Interface) defines a set of functions, macros and variables that provide access to most aspects of the Python run-time system. The Python API is incorporated in a C source file by including the header \code{"Python.h"}. (But it may not say nothing else should be included.) n

Collin Winter schrieb:

Was any course of action ever decided on for this issue, or was the consensus that it would break too much code? If the latter, what about making the change for Python 3000?

Neither, nor. If a grand renaming is not feasible, I'd atleast do something about object.h for Python 2.6. Regards, Martin