Can boost_python dll be delay loaded?
Hi I am working on a Windows application that uses boost_python, linked with shared libraries. I need to store the boost_python dll in a folder other than one in the usual search path (exe folder, System32, search path). I propose to do this by 'delay loading' the boost_python dll and calling SetDllDirectory() from the application. So far I haven't got this to work. I have read that some DLLs can't be loaded in this way. Does anyone have experience of delay loading boost_python? Best regards David
Wouldn't using an embedded manifest to specify a non-default DLL location be much easier? Niall On 29 Jul 2010 at 11:26, David Aldrich wrote:
Hi
I am working on a Windows application that uses boost_python, linked with shared libraries.
I need to store the boost_python dll in a folder other than one in the usual search path (exe folder, System32, search path). I propose to do this by 'delay loading' the boost_python dll and calling SetDllDirectory() from the application.
So far I haven't got this to work. I have read that some DLLs can't be loaded in this way. Does anyone have experience of delay loading boost_python?
Best regards
David _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
-- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909.
On 29 Jul 2010 at 12:09, David Aldrich wrote:
Wouldn't using an embedded manifest to specify a non-default DLL location be much easier?
Thanks. I am not familiar with how that works. Please can you give me any tips on how to do this?
Searching MSDN or Google would give you all the information you need - basically you edit an XML file with the appropriate info. I do remember seeing somewhere though that DLL-via-manifest support has become deprecated in Windows 7 and onwards due to it causing more problems than it solves, and off the top of my head I don't know what they're intending to replace it with. That said, there is no reason why it would ever cease to function as too much of Microsoft's own stuff is heavily dependent on it. HTH, Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909.
Hi Niall
Wouldn't using an embedded manifest to specify a non-default DLL location be much easier?
Thanks for your reply. I have now succeeded in creating a manifest file for an assembly that includes the boost_python dll's. But I don't understand how to specify a non-default location for the assembly using that manifest. I believe this is done using 'probing' in Windows 7 but I think that is unsupported in XP. So maybe I shall have to use delayed loading after all. Which brings me back to my original question, has anyone succeeded in delay loading boost_python under Visual C++? Best regards David
On 30 Jul 2010 at 8:22, David Aldrich wrote:
Thanks for your reply. I have now succeeded in creating a manifest file for an assembly that includes the boost_python dll's. But I don't understand how to specify a non-default location for the assembly using that manifest. I believe this is done using 'probing' in Windows 7 but I think that is unsupported in XP.
I thought that the application installer registered the DLL manifests at their locations? It's like registering COM or .NET objects, in fact I think it's the same mechanism nowadays.
So maybe I shall have to use delayed loading after all. Which brings me back to my original question, has anyone succeeded in delay loading boost_python under Visual C++?
Well, the method you outlined ought to be prevented by the system and certainly by any competent anti-virus. It would be ripe for exploitation. As for why it doesn't work when others do, I would guess it's due to how the BPL machinery sets itself up on process init. During static init the BPL library sets up all sorts of runtime information which is later used by BPL clients, and during all of this it is extremely important that everything be kept in precisely the right order. Put another way I would be extremely doubtful that you can delay load the BPL library. Your best bet it would seem now is to have PATH modified on app install, but that is a very lazy way out and is indeed banned in lots of corporate environments. Another alternative is that you could hive off all BPL using code into a separate DLL and have that delay loaded instead. That ought to work, though you really got to wonder at the cost/benefit ratio. HTH, Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909.
Hi Niall
I thought that the application installer registered the DLL manifests at their locations? It's like registering COM or .NET objects, in fact I think it's the same mechanism nowadays.
Well, I don't plan to use an application installer. Most of our users build and run the application from within a svn working copy.
As for why it doesn't work when others do, I would guess it's due to how the BPL machinery sets itself up on process init. During static init the BPL library sets up all sorts of runtime information which is later used by BPL clients, and during all of this it is extremely important that everything be kept in precisely the right order.
I don't understand the intricacies of Boost's initialisation, nor do I even know when it happens. I naively thought that nothing would happen until I explicitly call a boost_python API method. Does the fact that I have included <boost/python.hpp> mean that some boost initialisation will occur before main() is reached?
Your best bet it would seem now is to have PATH modified on app install, but that is a very lazy way out and is indeed banned in lots of corporate environments.
Sadly, I think that is the only solution. Thanks again for your help. Best regards David
On 30 Jul 2010 at 10:56, David Aldrich wrote:
I thought that the application installer registered the DLL manifests at their locations? It's like registering COM or .NET objects, in fact I think it's the same mechanism nowadays.
Well, I don't plan to use an application installer. Most of our users build and run the application from within a svn working copy.
It's usually easy enough to rig up a COM object registration into a build system. Surely it would similar for manifest stuff.
As for why it doesn't work when others do, I would guess it's due to how the BPL machinery sets itself up on process init. During static init the BPL library sets up all sorts of runtime information which is later used by BPL clients, and during all of this it is extremely important that everything be kept in precisely the right order.
I don't understand the intricacies of Boost's initialisation, nor do I even know when it happens. I naively thought that nothing would happen until I explicitly call a boost_python API method. Does the fact that I have included <boost/python.hpp> mean that some boost initialisation will occur before main() is reached?
Rather it is when the BPL DLL is loaded into the process that it performs its static init which sets up the internal environment ready for your client code to call BPL APIs. The BPL DLL, like any complex metaprogrammed C++ DLL nowadays, will expect to be initialised before main() gets called and would be unlikely to work if main() gets called first.
Your best bet it would seem now is to have PATH modified on app install, but that is a very lazy way out and is indeed banned in lots of corporate environments.
Sadly, I think that is the only solution. Thanks again for your help.
The only other I can think of is putting a symbolic link to the BPL DLL in the same directory as the executable. Symbolic links are Vista or later only sadly, though you could install Microsoft's Unix subsystem on XP and get symbolic links that way :) HTH, Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909.
Hi Niall Thanks again for your answers and help. Best regards David
-----Original Message----- From: cplusplus-sig-bounces+david.aldrich=eu.nec.com@python.org [mailto:cplusplus-sig-bounces+david.aldrich=eu.nec.com@python.org] On Behalf Of Niall Douglas Sent: 30 July 2010 16:48 To: Development of Python/C++ integration Subject: Re: [C++-sig] Can boost_python dll be delay loaded?
On 30 Jul 2010 at 10:56, David Aldrich wrote:
I thought that the application installer registered the DLL manifests at their locations? It's like registering COM or .NET objects, in fact I think it's the same mechanism nowadays.
Well, I don't plan to use an application installer. Most of our users build and run the application from within a svn working copy.
It's usually easy enough to rig up a COM object registration into a build system. Surely it would similar for manifest stuff.
As for why it doesn't work when others do, I would guess it's due to how the BPL machinery sets itself up on process init. During static init the BPL library sets up all sorts of runtime information which is later used by BPL clients, and during all of this it is extremely important that everything be kept in precisely the right order.
I don't understand the intricacies of Boost's initialisation, nor do I even know when it happens. I naively thought that nothing would happen until I explicitly call a boost_python API method. Does the fact that I have included <boost/python.hpp> mean that some boost initialisation will occur before main() is reached?
Rather it is when the BPL DLL is loaded into the process that it performs its static init which sets up the internal environment ready for your client code to call BPL APIs. The BPL DLL, like any complex metaprogrammed C++ DLL nowadays, will expect to be initialised before main() gets called and would be unlikely to work if main() gets called first.
Your best bet it would seem now is to have PATH modified on app install, but that is a very lazy way out and is indeed banned in lots of corporate environments.
Sadly, I think that is the only solution. Thanks again for your help.
The only other I can think of is putting a symbolic link to the BPL DLL in the same directory as the executable. Symbolic links are Vista or later only sadly, though you could install Microsoft's Unix subsystem on XP and get symbolic links that way :)
HTH, Niall
-- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909.
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Click https://www.mailcontrol.com/sr/QPy1GWmUWovTndxI!oX7Uqfv98hA5qioKLUaEvLrmgO mhsjStaar8uC2P7BlaqMtj6B2UVGAGYT63Umxp4EWgA== to report this email as spam.
Your best bet it would seem now is to have PATH
modified on app install, but that is a very lazy way out and is indeed banned in lots of corporate environments.
I solved this problem via a small launcher executable. It is just like a small .bat file which sets PATH and PYTHONPATH before launching the actual application. Only cmd.exe makes a mess passing through command-line arguments with spaces, therefore I resorted to this: http://cctbx.svn.sourceforge.net/viewvc/cctbx/trunk/libtbx/windows_dispatcher.c?revision=11155&view=markup This is compiled once (windows_dispatcher.exe, in the same svn). A Python script makes copies and replaces the placeholders for the path names with the actual path names. Ralf
On 30 Jul 2010 at 11:38, Ralf W. Grosse-Kunstleve wrote:
Your best bet it would seem now is to have PATH modified on app install, but that is a very lazy way out and is indeed banned in lots of corporate environments.
I solved this problem via a small launcher executable. It is just like a small .bat file which sets PATH and PYTHONPATH before launching the actual application. Only cmd.exe makes a mess passing through command-line arguments with spaces, therefore I resorted to this:
Also, now you make me think about it, Microsoft Detours can inject an arbitrary DLL at any given location into a process. HTH, Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909.
participants (3)
-
David Aldrich -
Niall Douglas -
Ralf W. Grosse-Kunstleve