[IronPython] Feedback needed for bug fix:Import pre-compiledmodules

Tue Jan 23 21:00:10 CET 2007

This should be solvable in a similar but slightly different way.  We wouldn't want to require an extra argument (in this case the versioned IronPython engine) to be required for each call.  Also I'm going to pick a slightly more generic example "DoOperation" instead of Add (because a bug fix to the semantics of Add would just change the body of Add and everything would be ok.  A signature change is really more interesting).

Static object DoOperation(object arg) { DoOperation_1_0_1(arg, null); }
Static object DoOperation_1_0_1(object arg1, object arg2) { ... }

In this case we could preserve the old DoOperation method and add the new DoOperation_1_0_1 method that anything in 1.0.1 or later would call.  All code compiled against 1.0 would continue to call the old version and pass in a default value for the new argument that we added.

The interesting thing here isn't can we come up with a scheme that enables us to fix bugs in the runtime from version to version.  The real challenge is ensuring that we actually follow this pattern from version to version and verifying that old assemblies continue to run on newer versions of IronPython.  That becomes an increasingly large test burden over time with an ever expanding test matrix as we ship each new version.

Certainly within minor versions this is something we might be able to pull off with only a little difficulty but for example ensuring that v1.0 binaries work against v1.1 (or worse yet, v2.0) well be quite the challenge.

One interesting question is why are people interested in this feature?  Is it primarily for the improved performance that loading the cached DLL gives or is it for enabling the shipping of binary-only EXEs that run against arbitrary versions of IronPython.dll?  It'd be interesting to see what most people want out of this feature to gauge how we should evolve this and the level of compatibility we should maintain between versions.

-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Keith J. Farmer
Sent: Tuesday, January 23, 2007 11:01 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Feedback needed for bug fix:Import pre-compiledmodules

I could be wrong (I certainly have been in the past), but the current scheme seems to pre-empt the built-in mechanisms.

There are several ways you can get a reference to an assembly -- file name, name space without strong name, name space and version, name space and public key, etc.  This works well enough for most (not all) purposes.  If you want .NET to load the most recent version of an assembly, you just need to ask for it.  If you require a certain version, or culture, you can ask for those as well.

Consider an addin approach, where the IronPython.dll is the addin rather than a compile-time reference.  If you did that, then the exe could have a bootstrapper that merely asks for the runtime without specifying strong names.  It loads IronPython, potentially obeying any of the standard redirection declarations in exe.config. The exe then casts the IP runtime to IIronPython_1_0_1, and starts calling methods.

In versioning IronPython, if a breaking change needs to happen, we can use explicit interfaces:

public int IIronPython_1_0_0.Add(int x, int y) { return x + y + 0.1; }
public int IIronPython_1_0_1.Add(int x, int y) { return x + y; // bug fix }

.. I'm just doing this off the top of my head at the moment, but I think it'd at least alleviate some of the problem.

-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of J. Merrill
Sent: Tuesday, January 23, 2007 7:22 AM
To: Discussion of IronPython
Subject: Re: [IronPython] Feedback needed for bug fix:Import pre-compiledmodules

I'm not arguing with you -- just playing devil's advocate.  Isn't "everyone has to use the same centrally maintained copy of a DLL" the recipe for "DLL hell" that .Net is supposed to let us avoid?  In the specific scenario you provide -- you update a DLL used by an existing EXE -- .Net is designed to keep using the old version of the DLL unless you either re-compile to re-build the EXE, or add an entry to the EXE's .config file that tells it that it's ok to use the newer one.  (That only applies for DLLs in the GAC, as I understand it.)

.Net isn't supposed to load app X that references DLL Y unless the "identity" of Y is the identity listed in the manifest for app X.  Changing the identity of a DLL can be done by changing its version number; unfortunately, unless the DLL is installed in the GAC, you can't have two copies of the same DLL differing only in their version and have "the right one" (the one referenced by the EXE, or pointed to by the EXE's config file) load.

Sigh.

At 12:38 AM 1/23/2007, Keith J. Farmer wrote
>Why do you assume the deployment will involve dropping IronPython in the application directory?  Sure, you *could*, but it's unreasonable, I think, to force the end user to have Yet Another Copy of a dll when it could just reference the latest-and-greatest at a central location.
>
>The situation I see is:
>
>Install IronPython.
>Install a binaries-only IP app.
>Update IronPython to change a spelling error in a resource -- suddenly the app doesn't even load.
>
>So, for binaries-only, the situation's just plain broken.  Granted, I'd wager that most Python is distributed with source (if not as source).
>
>Another alternative?  Use explicit interfaces in the IronPython runtime to allow side-by-side versioning of the API.  The exe's bootstrapper can load IronPython.dll without using the strong name, grab the runtime, cast to that interface and deal with versioning issues for some period of time before obsoletion.  This would also allow developers to switch compatibility levels when testing their programs.
>
>-----Original Message-----
>From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of J. Merrill
>Sent: Monday, January 22, 2007 8:03 PM
>To: Discussion of IronPython
>Subject: Re: [IronPython] Feedback needed for bug fix:Import pre-compiledmodules
>
>Could IP just ignore the timestamp on ironpython.dll and let the .Net runtime figure out if there are any references to no-longer-present mechanisms within the binary?
>
>Keith, is it really the case that your clients without Python source are going to download new versions of IP and (this is important) put them in the directory with your software?  That is, assuming that you put IP.DLL in the directory with the EXEs/DLLs you built, even if they're doing their own IP development elsewhere on the machine and updating it regularly, won't your executables use the old IP.DLL until you give them the new one (and presumably matching recompiled EXEs/DLLs)?
>
>At 09:24 PM 1/22/2007, Keith J. Farmer wrote
>>the upgraded-ironpython scenario
>>
>>>>> I do not think this is supported. The pre-compiled module has much dependency on IronPython.dll. Some emitted calls in those modules could be changed (or removed) in the next version of IronPython.dll.
>>
>>
>>That makes me itch...  I understand runtime dependencies, but the binaries-only deployment scenario just dropped in value if they are completely invalidated because the runtime undergoes a minor rev (or is otherwise touched).
>>
>>Would it be possible for the runtime to query the assembly to determine if it's compatible or not?  Some sort of poor man's static analysis (eg, a manifest of API dependencies could be generated when the assembly is stored to disk, and if the runtime doesn't find any in the list that match any breaking change from the producing version, it accepts it).
>>
>>
>>________________________________
>>
>>From: users-bounces at lists.ironpython.com on behalf of Haibo Luo
>>Sent: Mon 1/22/2007 4:44 PM
>>To: Discussion of IronPython
>>Subject: Re: [IronPython] Feedback needed for bug fix:Import pre-compiledmodules
>>
>>
>>
>>If ironpython.dll is newer than lib.exe, and lib.py does not exist, we should expect an exception?
>>
>>>>> Yes
>>
>>[snip]
>
>
>J. Merrill / Analytical Software Corp
>
>
>_______________________________________________
>users mailing list
>users at lists.ironpython.com
>http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>_______________________________________________
>users mailing list
>users at lists.ironpython.com
>http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

J. Merrill / Analytical Software Corp

_______________________________________________
users mailing list
users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

_______________________________________________
users mailing list
users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com