[Python-Dev] Location of CI for Windows Embeddable Distro

[Gerald Wiltse]

I have it working on linux now.  Is now a good time to open a ticket on
Issue Tracker?

Gerald R. Wiltse
jerrywiltse at gmail.com



On Fri, May 3, 2019 at 11:55 AM Gerald Wiltse <jerrywiltse at gmail.com> wrote:

> Ok, disregard previous message.  I fixed the PYTHONPATH in my command
> and the .pyc files and .  I also saw that python36._pth is specifically
> only supported by windows.
>
> Now i'm moved on to the following error:
>
> zipimport.ZipImportError: can't decompress data; zlib not available
>
> I'm on Ubuntu.  I've definitely got zlib1g-dev package installed, and I've
> recompiled a number of times trying to get it to work.  I've tried Ubuntu
> 16 and 18.
>
> Is this just a limitation on linux?
>
> Gerald R. Wiltse
> jerrywiltse at gmail.com
>
>
>
> On Thu, May 2, 2019 at 4:29 PM Gerald Wiltse <jerrywiltse at gmail.com>
> wrote:
>
>> Just an update based on testing.  If I zip the modules up into
>> python36.zip (whether in .py or .pyc form) the linux binary does not find
>> them.  If I copy them into the embedded root, alongside the python
>> executable, it finds them as expected.
>>
>> This just makes it seem more like an issue where Linux binary does not
>> respect .zip file as a source of modules.
>>
>> Gerald R. Wiltse
>> jerrywiltse at gmail.com
>>
>>
>>
>> On Thu, May 2, 2019 at 2:56 PM Gerald Wiltse <jerrywiltse at gmail.com>
>> wrote:
>>
>>> With minimal patching of tools/msi/make_zip.py, I was able to create an
>>> equivalent zip package on Linux.  It doesn't work out of the box of
>>> course.  I had to do two commands
>>>
>>> My biggest concern is that even if the layout is "correct" on linux, the
>>> python runtime might not actually support being run like this on Linux.
>>> Perhaps the code for using python36.zip as a landmark and source of modules
>>> is only implemented on Windows.  Do you know if that is the case?
>>>
>>>     tmp$ export LD_LIBRARY_PATH=$(pwd)
>>>     tmp$ export PYTHONHOME=$(pwd)
>>>     tmp$ export PYTHONPATH=$(pwd)
>>>     tmp$ ./python
>>>     Fatal Python error: Py_Initialize: Unable to get the locale encoding
>>>     ModuleNotFoundError: No module named 'encodings'
>>>
>>>     Current thread 0x00007fdde9eb1680 (most recent call first):
>>>     Aborted (core dumped)
>>>
>>> Here's the full directory structure, error, and patch I created. Any
>>> additional feedback would be great.
>>>
>>> https://gist.github.com/solvingj/87305e0f70e14d738d6989ba2f41352f
>>>
>>> Gerald R. Wiltse
>>> jerrywiltse at gmail.com
>>>
>>>
>>>
>>> On Thu, May 2, 2019 at 11:24 AM Gerald Wiltse <jerrywiltse at gmail.com>
>>> wrote:
>>>
>>>> I've successfully automated the build of the embedded zip on Windows
>>>> for 3.6 on my internal CI using tools/msi script.
>>>>
>>>> Moving onto linux, can you please summarize the primary
>>>> challenges/differences for creating the embeddable zip for linux?
>>>>
>>>> Gerald R. Wiltse
>>>> jerrywiltse at gmail.com
>>>>
>>>>
>>>>
>>>> On Wed, May 1, 2019 at 2:47 PM Gerald Wiltse <jerrywiltse at gmail.com>
>>>> wrote:
>>>>
>>>>> Yes, I'll help with this, and I try to get signed up with an account
>>>>> for the issue tracker ASAP.
>>>>>
>>>>> I've just started trying to build create the embeddable package for
>>>>> windows locally (all from source).  Once I have that automated, I'll take a
>>>>> naive stab at the Linux layout.  I'm sure i'll soon hit the additional
>>>>> complications you're talking about and come back to you with questions.
>>>>>
>>>>> Gerald R. Wiltse
>>>>> jerrywiltse at gmail.com
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 1, 2019 at 1:57 PM Steve Dower <steve.dower at python.org>
>>>>> wrote:
>>>>>
>>>>>> On 01May2019 1305, Gerald Wiltse wrote:
>>>>>> > That does help, thanks so much for the fast response!  I now see
>>>>>> how
>>>>>> > it's executed on Azure pipelines also, so that should be enough for
>>>>>> me
>>>>>> > to reverse engineer the rest.
>>>>>> >
>>>>>> > Followup questions if you have time:
>>>>>> > 1. It seems that directory doesn't exist in the 3.6 branches/tags,
>>>>>> so it
>>>>>> > must have been done a different way back then.  Do you think it's
>>>>>> > back-portable to 3.6?
>>>>>>
>>>>>> Yep, it should work with 3.6. There were scripts under Tools/msi back
>>>>>> then that did the same thing, but with the Store app I rewrote it to
>>>>>> be
>>>>>> more flexible.
>>>>>>
>>>>>> > 2. One of our objectives is to effectively create a nearly
>>>>>> identical
>>>>>> > embeddable package for linux. Do you think the layout code is
>>>>>> generic
>>>>>> > enough to be modified to create a linux equivalent?
>>>>>>
>>>>>> Probably not, it's under the "PC" directory because it is entirely
>>>>>> CPython specific.
>>>>>>
>>>>>> I'd love to be able to have a similar embeddable distro for Linux
>>>>>> (and
>>>>>> macOS), and I'll be investing some of my time in working towards
>>>>>> this,
>>>>>> but it's far more complicated than simply laying out the right files.
>>>>>> If
>>>>>> you're interested in helping with this, then we're in the right place
>>>>>> to
>>>>>> discuss it, but we'll likely have to move to the issue tracker at
>>>>>> some
>>>>>> point once we figure out who else wants/needs to be involved.
>>>>>>
>>>>>> > Obviously, there are many alternative methods and documents for
>>>>>> creating
>>>>>> > "embeddable" pythons on linux, but the windows package has all the
>>>>>> > characteristics we want, and of course we'd like our Windows and
>>>>>> Linux
>>>>>> > process build and packaging processes to be as similar as possible,
>>>>>> so
>>>>>> > it seems attractive to try to port this layout script to linux.
>>>>>>
>>>>>> "make install" already does the same thing, but the install location
>>>>>> matters at build time, so it's a little more complex to make it work
>>>>>> as
>>>>>> an embedded runtime. I'm sure there are people who know how to make
>>>>>> it
>>>>>> work though.
>>>>>>
>>>>>> Cheers,
>>>>>> Steve
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190503/419e5e8c/attachment.html>