[python-win32] adodbapi and .hgeol

[Mark Hammond]

On 14/06/2013 7:00 PM, Vernon D. Cole wrote:
> I found the following on [pywin32-checkins] this morning:
>
>     diff -r 4ded0a838f8c -r 1594b8c27383 .hgeol
>     --- a/.hgeol    Mon Apr 29 10:01:40 2013 -0400
>     +++ b/.hgeol    Thu Jun 13 18:29:09 2013 +1000
>     @@ -2,4 +2,8 @@
>     **.dsp = CRLF
>     **.dsw = CRLF
>     **.vbp = CRLF
>     +# adodbapi appears to have been checked in with windows line endings
>     +# and without the 'eol' extension enabled, so we say it's binary to
>     stop
>     +# hg continually thinking it has changed.
>     +path:adodbapi = bin
>     ** = native
>     diff -r 4ded0a838f8c -r 1594b8c27383 CHANGES.txt
>     --- a/CHANGES.txt    Mon Apr 29 10:01:40 2013 -0400
>     +++ b/CHANGES.txt    Thu Jun 13 18:29:09 2013 +1000
>
>
> No matter how old you get (I just turned 63) you still have new things
> to learn. Until today, I have been completely unaware of an .hgeol file.
>
> Since adodbapi now also runs on Linux, I had better make sure the line
> endings stay right. Python does not care, nor does the Pythonwin IDE
> editor, but I have not been consistant about what editor I use, which
> may be the cause of this problem.
>
> My quick reading of the documentation for .hgeol leads me to believe
> that setting the entire adodbapi path to =bin is not the correct way to
> handle this.  I will adapt my other mercurial repository to match this
> -- but I want to do it right this time.  What line ending style are my
> Python and .txt files supposed to have?  I will adapt my source.

It's all quite confusing and a PITA.  What I *think* the story is...

* Using the "eol" extension means that it's possible to have the 
repository use \n as line endings, and, depending on the platform being 
used, have either \n or \r\n line-endings in checkouts of the 
repository.  This has obvious benefits for cross-platform projects - but 
in the case of pywin32 it's not any practical benefit (ie, for this 
project there would be almost no downsides to having the repo itself 
always have \r\n endings)

But we are where we are - in pywin32, the repository has \n line 
endings.  If the eol extension is enabled, a checkout on Windows will 
have \r\n line endings.  A checkout on Linux etc would have \n line 
endings (except adodbapi - see below)

Unfortunately, enabling the eol extension isn't enough to "fix" the 
current checkout - so you either want to have enabled eol before 
checking out, or clobber the existing tree ('hg up -r null'), then do a 
normal update.

To demonstrate:

# first clone the pywin32 directory, but do *not* update the working tree.
% hg clone -U pywin32 pywin32-clone
% cd pywin32-clone # this dir is empty, apart from .hg/
# Just in-case eol is enabled globally, we'll disable it for this clone:
% echo "[extensions]" >> .hg/hgrc
% echo "eol=!" >> .hg/hgrc # this disables it locally.
# now update the working copy:
% hg up

# .. now view the EOL chars of, eg, setup.py - use some editor
# like Scite or Pythonwin which can show these.
# setup.py has \n line endings.
# Do the same for, eg, adodbapi/is64bit.py - note it has \r\n!
# [FWIW - license.txt in adpdbapi has \n...]

# Now, enable EOL for this repo - edit .hg/hgrc and remove the
# "eol=!" line...
% hg up
# hg says it did nothing - setup.py has not changed!
# So - nuke the working tree
% hg up -r null
# verify all files gone.
% hg up

# Now re-check setup.py - it now has \r\n locally.  adodbapi.py's files
# still all have \r\n.

So - it appears that adodbapi was initially (or most recently - I 
haven't checked the history) checked into hg *without* the eol extension 
being enabled and while the local tree had \r\n line endings.

This means the *repository* copy of adodbapi has \r\n line endings. 
With the eol extension installed, hg gets very upset (even though it 
probably shouldn't) - it keeps thinking the adodbapi files have always 
changed (a diff shows every line changed).  I can't find a way to avoid 
that - except, to tell hg that the adodbapi directory consists of 
"binary" files.  This simply tells hg to not attempt line ending 
conversions on these files.  So we end up with adodbapi having \r\n line 
endings regardless of the platform on which the repo is checked out on.

The "fix" to this would, IIUC (and untested), involve:
* ensuring eol is disabled for the clone
* adjust all files to have \n line endings
* commit the changes
[and to get things sane again]
* nuke the tree (ie, 'hg up -r null')
* enable the eol extension
* Remove the 'binary' annotation in .hgeol
* 'hg up'
* verify it all worked as expected.
* commit .hgeol
* probably mail the checkins list explaining that you just did this, and 
for others to be aware of hg being dumb, not realizing current checkouts 
need updating etc, so to consider doing, eg, "hg up -C" or similar to 
stop it complaining :)

The other downside of this is this checkin would touch every line of 
every such file - so things like 'hg blame' etc would become useless for 
these files - it would be impossible to see what change touched a single 
line - this checkin would get the blame for every single line.

OTOH though, the problem is purely theoretical - I doubt people are 
using this repo anywhere but Windows, so no one will notice any 
difference.  So the status quo is probably fine in practice, and it 
avoids the 'blame' problem mentioned above.  Which is exactly why I just 
changed .hgeol instead of coordinating a line-end-change commit with you 
(but I do apologize for not firing you a mail at the time anyway).

However, if you have an existing hg repo which you hope people also use 
on Linux, you probably *do* want to take action in that repo - the files 
should be in the repo with \n line endings, and rely on the eol 
extension to get \r\n files locally on Windows.  (If your current repo 
is the same as pywin32, I'd expect linux users which checkout your repo 
will end up with \r\n line endings on Linux).  If you do fix that repo, 
then you could consider fixing pywin32 too - but that's your call and is 
unlikely to impact anyone using pywin32.

Just to make matters worse, the 'eol' extension really isn't a panacea - 
I do a huge amount of work with the huge Mozilla hg repositories, and 
'eol' just impacts performance too much.  So I have the eol extension 
disabled globally (so my mozilla checkouts all have \n line endings 
locally), and just enable it locally for a few repos such as pywin32.

I hope that makes sense (and is accurate ;)

Mark