[Python-Dev] AIX to stable, what does that take?

[Michael Felt]

On 10/4/2018 10:30 AM, INADA Naoki wrote:
> Hello,
>
> First of all, congratulations on passing all test on AIX.
>
>> My assumption is that it needs to (at least) pass all tests - and that
>> is why I keep asking for attention. All the PRs to fix individual tests
>> mean less if they are not merged, for whatever reason.
>>
>> However, maybe there is another way, or even something additional
>> needed. Maybe something I cannot provide and then I can adjust my
>> expectations and goals.
> As a one of core developer, I don't know anything about AIX.
> If my change breaks AIX build, I can't investigate what's happened.
>
> So I think we need following in devguide:
>
> * Brief description about AIX, from developer's point of view.
This I might be able to do. Bullet form:
* Committed to POSIX standard (valid when release came out, so AIX 5.3
confirms to a different standard than AIX 7.2)
* While Linux affinity is recognized - GNU (or GNP - GNU not POSIX)
integration is not guaranteed. - GNU rte is not provided under support.
There is a so-called Toolbox, GNU an other OSS utilities supplied by
many packaged as RPMs. Unfortunately, different RPM providers (Michael
Perlz, BULL Freeware, IBM, and others) have different numbering (the
part after the package version, e.g., python-2.7.10-XXX so they do not
mix well). Headache for both admins and developers trying to develop in
a GNU-like environment.
* As a consultant, fedup with what is called the "RPM hell" by many AIX
admins - I do not use any RPMs. I build everything myself, using xlc
(gcc introduces the need for a GNU RTE, e.g., glibc). I package using
installp (the "native") AIX package manager, and strive to make the
packages independent (these days). When there are dependencies I try to
build them as static libraries so that they do not become an additional
install dependency.
* finally, a bit deeper: while the AIX linker loader supports svr4
shared libraries (it is the data, not the file name) it also supports
having multiple shared libraries in a classic archive. So, rather that
.../lib/libxxx.so and .../lib64/libxxx.so AIX prefers .../lib/libxxx.a
with two so-called members, with same or different names. The one found
is not it's name, but the symbol name and size of the ABI (32-bit or 64-bit)
* Hope that is enough of the key issues for now.
** In general, GNU autotools (autoreconf -f -v) works well, as does
configure.ac et al. for createing OSS Makefiles.
> * How to run AIX on (VirtualBox, AWS EC2, Azure, GCP) easily.
Easily! ? :) - well, on a POWER server it was easy enough for me to
follow the step by step instructions for creating a buildbot. If I had a
POWER server with more resources I would examine using the AIX internal
WPAR - however, a POWER server configured to use PowerVC uses "EC2" aka
cloud-init for creating a virtual machine. With that environment it
should be "easy" to provide additional instructions to cloud-init-ec2.

Or, I provide you a login on my personal server that I run the buildbot
on. etc. etc. - Where there is a will, there is a way.
> * How to set up a development environment for Python.
Again, follow the instructions for setting up a buildbot.
> * How to build Python.
git clone ...
autoreconf -v -f (as needed)
./configure --with-pydebug  #gcc compiler
./configure --with-pydebug --without-computed-gotos # xlc compiler
make
make test
> * How to debug C code.
I learned, 40 years ago, using adb (a debugger) - I do a lot of
single-stepping. gdb is not the default debugger. If I were a developer
I would probably dig into the AIX debuggers (there are at least two, kdb
(kernel debugger, which I do use occaisionally for performance issues)
and another I try to avoid. I add fprintf statements and am looking at
learning how to use probevue.

In short, you probably have many much better ideas on how to debug C
than I do :)
>
> And even though there is a developer guide, it will take more long time
> than fixing issues on AIX, compared Linux, macOS, and Windows.
>
> But without this guide, it feels almost impossible to maintain AIX build to me.
IMHO: The AIX build is stable, but this is unrecognized because it does
have differences that cause tests to fail. I can think of one test that
PASSes, but should fail. And another test that passes, but should have
failed (in test_uuid) I have submitted a PR.

I tried to fix "all" in one PR, which confused people - so redid it as
two (got _uuid working in Python 3.7 ! yes!!) but the "original" to fix
uuid.py and test_uuid.py is still "awaiting change review".

My gut feeling to maintaining AIX is: a) all test pass so a potential
regression is flagged; b) someone such as myself who knows the platform
and can establish a "root cause" on why it is failing with AIX so that
c) a developer becomes aware and can decide to ignore or adjust; d)
alternatives - such as work around an implementation limitation (as I
try to do, e.g., for test_time and the related _datetime.c) is yet
another path.

In other words - it needs to be a shared responsibility - some
volunteers with a passion for platform stability (in this specific case
AIX and me as "passionate-person" and perhaps someone as youself who
wants to focus on the language itself - ideally without deep (or any!)
concern for platform differences.
>
> Regards,
My 6 bits!