Mailman 3 Re: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c - Python-Dev

newer
Re: [Python-Dev] [Python-checkins]...

Re: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

Barry Warsaw

2 Jan 2006 2 Jan '06

11:03 p.m.

I think we have a fundamental problem with Python-ast.c and Python-ast.h. These files should not be both auto-generated and checked into Subversion. The problem is that if you do a "make distclean", these files will get removed and "svn stat" will give you a ! flag. Of course, you can "svn up" to get them back, but that just masks the problem. Worse, if you distclean, then configure, then make, it is possible these files will change. Say you're working on a patch that touches a lot of files and you do an "svn commit". It's certainly possible that you'll accidentally commit new versions of the Python-ast.* files, possibly even broken ones. The general rule should be that no file that is ever generated can be checked into Subversion. Probably the right approach is to check in a template file that will not get removed by a distclean, and modify the build process to generate Python-ast.* from those template files. -Barry

Attachments:

signature.asc (application/pgp-signature — 307 bytes)

Show replies by date

Neal Norwitz

2 Jan 2 Jan

11:16 p.m.

New subject: [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

On 1/2/06, Barry Warsaw wrote:

...

I think we have a fundamental problem with Python-ast.c and Python-ast.h. These files should not be both auto-generated and checked into Subversion.

I agree with the problem statement.

...

The general rule should be that no file that is ever generated can be checked into Subversion. Probably the right approach is to check in a template file that will not get removed by a distclean, and modify the build process to generate Python-ast.* from those template files.

I'm not sure about your proposed solution, though. There's a bootstrapping issue. Python-ast.[ch] are generated by a python 2.2+ script. /f created a bug report if only 2.1 is available. The Python-ast.[ch] should probably not be removed by distclean. This is similar to configure. Would that make you happy? What else would improve the current situation? If you go the template route, you would just copy the files. That doesn't seem to gain anything. n

Barry Warsaw

11:22 p.m.

New subject: [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

On Mon, 2006-01-02 at 15:16 -0800, Neal Norwitz wrote:

...

The Python-ast.[ch] should probably not be removed by distclean. This is similar to configure. Would that make you happy? What else would improve the current situation?

I think it would, without causing bootstrapping issues. -Barry

Jeremy Hylton

3 Jan 3 Jan

1:23 a.m.

New subject: [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

On 1/2/06, Neal Norwitz wrote:

...

On 1/2/06, Barry Warsaw wrote:

...
I think we have a fundamental problem with Python-ast.c and Python-ast.h. These files should not be both auto-generated and checked into Subversion.

I agree with the problem statement.

...
The general rule should be that no file that is ever generated can be checked into Subversion. Probably the right approach is to check in a template file that will not get removed by a distclean, and modify the build process to generate Python-ast.* from those template files.

I'm not sure about your proposed solution, though.

There's a bootstrapping issue. Python-ast.[ch] are generated by a python 2.2+ script. /f created a bug report if only 2.1 is available.

The Python-ast.[ch] should probably not be removed by distclean. This is similar to configure. Would that make you happy? What else would improve the current situation?

I think this solution is better. It's relatively rare for people to change the ast definition, so for most purposes these should be static files. Jeremy

"Martin v. Löwis"

8:38 a.m.

New subject: [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

Jeremy Hylton wrote:

...

I think this solution is better. It's relatively rare for people to change the ast definition, so for most purposes these should be static files.

Interestingly enough, I found yesterday that Python-ast.c did change for me, even though I had not touched the AST grammar at all. Assuming somebody forgot to commit this file, I just did. I now investigated where this change originated from, and it was this commit r41812 | tim.peters | 2005-12-26 00:18:31 +0100 (Mo, 26 Dez 2005) | 2 lines Whitespace normalization. This indicates two problems: apparently, the whitespace normalization also normalizes string literals; not sure whether this is a good idea or not. Furthermore, people should be made aware that something deserves their attention when they find that Python-ast.c has changed for them (but I guess Tim never saw Python-ast.c change, because this build step isn't executed on Windows). Regards, Martin

Tim Peters

6:58 p.m.

New subject: [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

[Jeremy Hylton]

...

...
I think this solution is better. It's relatively rare for people to change the ast definition, so for most purposes these should be static files.

[Martin v. Löwis]

...

Interestingly enough, I found yesterday that Python-ast.c did change for me, even though I had not touched the AST grammar at all. Assuming somebody forgot to commit this file, I just did.

I now investigated where this change originated from, and it was this commit

r41812 | tim.peters | 2005-12-26 00:18:31 +0100 (Mo, 26 Dez 2005) | 2 lines

Whitespace normalization.

This indicates two problems: apparently, the whitespace normalization also normalizes string literals; not sure whether this is a good idea or not.

reindent.py never leaves a hard tab character behind, or a line with trailing whitespace. Those are features, because they're invisible, and _someone's_ editor is going to destroy either or both of those sooner or later anyway. No core code should rely on them (inside string literals or not). I don't believe that asdl_c.py requires either hard tab characters or trailing whitespace on lines (both of which reindent.py replaced/removed), and reindent.py's output is a fixed-point for reindent.py, so those shouldn't happen again (unless/until someone sticks hard tabs or trailing whitespace into asdl_c.py again). If someone is really keen to have hard tab characters inside the CODE strings written by this file, then "the right" way to do that is to use \t escapes inside the triple-quoted strings (instead of using embedded hard tab characters).

...

Furthermore, people should be made aware that something deserves their attention when they find that Python-ast.c has changed for them (but I guess Tim never saw Python-ast.c change, because this build step isn't executed on Windows).

I can do better than guess about that ;-)

Donovan Baarda

12:13 p.m.

New subject: [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

On Mon, 2006-01-02 at 15:16 -0800, Neal Norwitz wrote:

...

On 1/2/06, Barry Warsaw wrote:

...
I think we have a fundamental problem with Python-ast.c and Python-ast.h. These files should not be both auto-generated and checked into Subversion.

I agree with the problem statement.

...
The general rule should be that no file that is ever generated can be checked into Subversion. Probably the right approach is to check in a template file that will not get removed by a distclean, and modify the build process to generate Python-ast.* from those template files.

I'm not sure about your proposed solution, though.

There's a bootstrapping issue. Python-ast.[ch] are generated by a python 2.2+ script. /f created a bug report if only 2.1 is available.

The Python-ast.[ch] should probably not be removed by distclean. This is similar to configure. Would that make you happy? What else would improve the current situation?

If you go the template route, you would just copy the files. That doesn't seem to gain anything.

The solution I use is to never have anything auto-generated in CVS/SVN, but have "make dist" generate and include anything needed for bootstrapping in the distribution tarball (or whatever). Doing "make distclean" should delete enough to bring you back to a freshly extracted distribution tarball, and "make maintainer-clean" should delete all auto-generated files to bring you back to a clean CVS/SVN checkout. I tend to include quite a few generated files in the distribution tarball that are not in CVS/RCS. Things like ChangeList (generated by cvs2cl), all the autotools autogen'ed files, generated datafiles, etc. This way your source distributions don't have any bootstrap problems, but you also don't have any auto-generated files in CVS/SVN and the problems they create. It does however mean that a fresh CVS/SVN checkout does have additional build requirements above and beyond building from a source distribution. -- Donovan Baarda http://minkirri.apana.org.au/~abo/

6687

Age (days ago)

6688

Last active (days ago)

List overview

Download

6 comments

6 participants

participants (6)

"Martin v. Löwis"
Barry Warsaw
Donovan Baarda
Jeremy Hylton
Neal Norwitz
Tim Peters

Re: [Python-Dev] [Python-checkins] commit of r41880 - python/trunk/Python/Python-ast.c

Barry Warsaw

Neal Norwitz

Barry Warsaw

Jeremy Hylton

"Martin v. Löwis"

Tim Peters

Donovan Baarda

tags

participants (6)