[pypy-dev] pypy -mime rfc2045_file? -- Was/is?: Re: Python FFI

[Bengt Richter]

Hi all,
I hope I did the right thing changing the subject line, though
it was the ffi C header and CPython mixed source issues that got me started...

On 05/15/2012 11:36 PM Armin Rigo wrote:
> Hi Stefan,
>
> On Wed, May 16, 2012 at 7:01 AM, Stefan Behnel<stefan_ml at behnel.de>  wrote:
>> I'm with Jean-Paul here.
>
> ...for a reason that might be unrelated to Jean-Paul's original
> complain: it's not like Cython gives first-class object manipulation
> on the result of parsing its .pyx files...  well, too bad but Python
> is not Lisp :-)
>
>> I don't think there's anything wrong with requiring declarations for
>> external C code to live in external modules.
>
> I'm not against the idea of recognizing also Cython syntax extensions
> in the declarations, and indeed external files are a better idea than
> big inlined strings then.  But I would also like to try out different
> paths.  In one of them, these inline declarations would contain mostly
> just "#include<foo.h>" and the real header would be processed with
> the help of the C compiler.
>
>
> A bientôt,
>
> Armin.

I see in this a general method of passing arbitrary sources to a translation system
-- in this case with pypy as the invoking shell command. with now a possibly
open-ended selection of file types. After C headers, CPython, ... ?? ;-)

rfc2045[1] and related rfcs define the MIME way of specifying multipart data
sets, with open-ended ways of specifying types, character encoding, transmission encoding,
etc., and includes a simple delimiter system that permits nested quoting/inclusion
implying tree structure possibilities (unlikely, but if needed) without too much judo.
[1] http://tools.ietf.org/html/rfc2045

So I'm wondering if you could see using a new command line option something like
     pypy  -mime mime_container_file  -mime optionally_another  python_source

You could even optionally package everything including python sources in the container file,
specifying what's what in the mime headers. Also wouldn't need encoding cookies in the source bodies.
Or you could just use the mime file as a top level type and encoding specification for the rest of
your sources, like a typed manifest for what your pypy invocation is going to use.

I am not totally in love with MIME, but I like the abstraction behind it. And it is a pretty well
defined standard in common use (which solves a big part of the documentation problem ;-)

I would think that using a general-purpose multipart container file type as (optional) source
would help with IDEs also, making it easy to make a folded top level MIME representation that can
be expanded by clicking a [+] line to invoke plugin specialized editors selected on the basis
of the metadata headers in the MIME. (I guess you could do the same with any container file, .e.g. .tgz,
but I suspect you'd wind up re-inventing conventions already defined in rfc2045).

Further, there's no reason you couldn't define MIME parts that were just a line or two of headers,
a blank separator, and some url lines pointing to files anywhere. You could include sha1 hashes
or gpg signatures etc in the header section for automated security and integrity checks, specify
caching and/or automatic retrieval or update, etc. etc. But you could also use MIME very minimally ;-)

This would be a way to include read-only legacy files with no modifications, while specifying
encoding etc in a mixed type set that would otherwise require encoding cookie mods.

You could also include or refer to patch or test stuff, but you could also use single-part MIME
files like typed symbolic links with only a single line of metadata, a blank line, and an url
in the file. I'm sure you are already ahead of me into other possibilities.

Regards,
Bengt Richter