From davidanderson2374 at outlook.com  Wed Jun  5 13:34:39 2013
From: davidanderson2374 at outlook.com (David Anderson)
Date: Wed, 5 Jun 2013 16:04:39 +0430
Subject: [capi-sig] Itutinga - Nomes dos aprovados
Message-ID: <13704482210b6e958e969a345cb124e7ac40645ea4@outlook.com>

Itutinga  ALINE MOURA FERREIRA, LARESSA RAMOS FERREIRA, FERNANDA LEONTSINIS CARVALHO BRANCO, NATIELLY ALEXANDRE CARNEIRO, JO?O CARLOS MOREIRA DE CARVALHO, CLAUDIO LUIS GOMES PEREIRA, MARIA CECILIA ALBUQUERQUE CORDEIRO, ISABELLE PEREIRA DA SILVA. RUI ANDRADE DA SILVA, ELIANA DANTAS RIBEIRO, MATHEUS FERNANDES CAMPOS, JOSE VALDIR TEIXEIRA BRAGA FILHO, VINICIUS CANDIDO DOS REIS.  Perol?ndia.

Piquet Carneiro ALINE PINHO BARROS, LARISSA FORTE VASCONCELOS, FERNANDO DE MORAES RODRIGUES, NEWTON BEVILAQUA DIAS NETO, JO?O CARLOS MOREIRA DE CARVALHO, CLAYTON DANTAS DE OLIVEIRA, MARIA DA CONCEI??O SIM?ES LOPES, IVANILDO NUNES DA SILVA. SABRINA ISABEL DE OLIVEIRA PAIVA, ELIELSON LEANDRO DE LIMA, MAURICIO CARLOS DE SOUZA FILHO, JOSE WILSON DE CARVALHO FILHO, VITOR GOMES DOS SANTOS. Cassil?ndia.

Catas Altas ANA CAROLINA DE FIGUEIREDO COSTA, LIGEOV?NIA FABR?CIA HAC?RIO SILVA, FRANCISCO EMANUEL TIMBO FERREIRA JUNIOR, PAULO MATHEUS MOTA MARANHAO, JO?O CARLOS MOREIRA DE CARVALHO, DAMI?O FRANCISCO BARBOSA FILHO, MARIA GISELDA DE OLIVEIRA ARA?JO, JANAINA SALES DE BRITO. SHERLANE CRISTINA DA SILVA, ANTONIO LEONARDO FREITAS SIQUEIRA, LUCAS MEDEIROS CAVALCANTE COSTA, GILDENE M? DE MACEDO RODRIGUES, RAYSA PINHEIRO LEMOS. Cassil?ndia.



From hniksic at xemacs.org  Fri Jun  7 22:36:01 2013
From: hniksic at xemacs.org (Hrvoje Niksic)
Date: Fri, 07 Jun 2013 22:36:01 +0200
Subject: [capi-sig] Embedded python,
	adding a path to sys.path before import, strange observations
In-Reply-To: <CAFG21oge6V=nJDg7YYzUy5_M7CEwD69CZUECJ_gxHWAB9=gtKQ@mail.gmail.com>
	(Todd Lyons's message of "Tue, 14 May 2013 08:37:53 -0700")
References: <CAFG21oge6V=nJDg7YYzUy5_M7CEwD69CZUECJ_gxHWAB9=gtKQ@mail.gmail.com>
Message-ID: <87d2rxk6a6.fsf@xemacs.org>

Todd Lyons <tlyons at ivenue.com> writes:

> I was testing by comparing the output of Py_GetPath() before I
> adjusted the path, and then after I adjusted the path.  It always
> shows the same.
[...]
> Question A: Is it normal that Py_GetPath() doesn't show the adjusted path?

The documentation is clear:

char* Py_GetPath()

    Return the *default* [emphasis mine] module search path; this is
    computed from the program name (set by Py_SetProgramName() above)
    and some environment variables. The returned string consists of a
    series of directory names separated by a platform dependent
    delimiter character. The delimiter character is ':' on Unix and Mac
    OS X, ';' on Windows. The returned string points into static
    storage; the caller should not modify its value. The list sys.path
    is initialized with this value on interpreter startup; it can be
    (and usually is) modified later to change the search path for
    loading modules.

The last sentence makes it clear that sys.path is initialized from the
value returned by Py_GetPath(), and not the other way around.

> Question B: All three seem to work equally well, is there a reason to
> choose one over the other?

All are fine (I guess I'd use method 1 myself), but the code that
manipulates sys.path is almost always fishy.  In many cases there are
better ways to achieve the same result.

From hniksic at xemacs.org  Fri Jun  7 22:44:04 2013
From: hniksic at xemacs.org (Hrvoje Niksic)
Date: Fri, 07 Jun 2013 22:44:04 +0200
Subject: [capi-sig] Constructing variable arg tuples
In-Reply-To: <CAFG21og0fmj9XnNx+T6ZQ3fZnwEoaJNne2SpNLZO7NV2xoyNQg@mail.gmail.com>
	(Todd Lyons's message of "Tue, 21 May 2013 07:27:09 -0700")
References: <CAFG21og0fmj9XnNx+T6ZQ3fZnwEoaJNne2SpNLZO7NV2xoyNQg@mail.gmail.com>
Message-ID: <878v2lk5wr.fsf@xemacs.org>

Todd Lyons <tlyons at ivenue.com> writes:

> Hi all, I'm embedding python into a c app (exim, the MTA).  I've
> gotten code in to initialize python, read in the embedded script, call
> various functions within that script, and appropriately return data
> from those functions back to the exim call.
>
> One thing that does not make sense yet though, is how to assemble the
> data to be passed to the python functions.  Imagine the following
> simple function:
>
> def simple_math(a,b):
>      return a+b
>
> In this case, a and b are integers or long integers.  How can I format
> that incoming data without knowing in advance that there are two
> arguments to this function and that are both are integers?

Obviously the scripts and your embedding module need to agree on an API;
for example, the script could look like this:

import exim

exim.modify_configuration(...)
x = exim.query_something(...)
exim.add_callback(my_function)
...

The "exim" module is provided by you as the embedder and serves for the
script to access exim's functionality.  (Otherwise the script would be
executed as just another Python script, which would not be too useful
from the embedding standpoint.)

In other words, you're not calling the functions from the script: the
script is calling your functions.  The only time that you are calling a
script's function is when the script passes a function to one of your
functions that accepts a callback argument -- but even then the
interface of the function is regulated by you, and it is you who
prescribe how many and what kind of arguments it will accept.

> If the reason for this question is not clear, imagine that the
> sysadmin goes and edits the python script so that it now is:
>
> def simple_math(name,a,b):
>      // do something to log "name" to a logfile
>      return a+b

What would be the point of such a function, before or after the edit?
You cannot just call any random function from a script: you need to
execute the script, providing a way for the script to call *you*.

From tlyons at ivenue.com  Sun Jun  9 06:07:16 2013
From: tlyons at ivenue.com (Todd Lyons)
Date: Sat, 8 Jun 2013 21:07:16 -0700
Subject: [capi-sig] Constructing variable arg tuples
In-Reply-To: <878v2lk5wr.fsf@xemacs.org>
References: <CAFG21og0fmj9XnNx+T6ZQ3fZnwEoaJNne2SpNLZO7NV2xoyNQg@mail.gmail.com>
	<878v2lk5wr.fsf@xemacs.org>
Message-ID: <CAFG21ojfQx6YKV0sCnps-GFZABqv3591jNRsKV1Nwcs=cAR-TQ@mail.gmail.com>

On Fri, Jun 7, 2013 at 1:44 PM, Hrvoje Niksic <hniksic at xemacs.org> wrote:
> Todd Lyons <tlyons at ivenue.com> writes:
>
>> Hi all, I'm embedding python into a c app (exim, the MTA).  I've
>> gotten code in to initialize python, read in the embedded script, call
>> various functions within that script, and appropriately return data
>> from those functions back to the exim call.

I've gotten quite a bit further now that when I sent in that email.
My code can be seen at:

http://git.exim.org/users/tlyons/exim.git/shortlog/refs/heads/master_volatile_python

>> One thing that does not make sense yet though, is how to assemble the
>> data to be passed to the python functions.  Imagine the following
>> simple function:
>>
>> def simple_math(a,b):
>>      return a+b
>>
>> In this case, a and b are integers or long integers.  How can I format
>> that incoming data without knowing in advance that there are two
>> arguments to this function and that are both are integers?
>
> Obviously the scripts and your embedding module need to agree on an API;

Yes.  I've set the following limitations:
1. All args passed to a python function are converted to strings.  It
is up to the function to include an implicit cast
2. No dicts, tuples, etc are passed.  Just simple strings or numbers
(which are converted to strings).
3. Return from a function is always cast back to a string, which is
what exim prefers.

This is subject to changes and additions.

> The "exim" module is provided by you as the embedder and serves for the

I plan to make two basic functions:
1. exim.log - allows functions in python to log info to the exim logging system.
2. exim.expand - allows functions in python to access read-only exim
expansion variables (i.e. headers, body, recipients, senders,
configuration directives, etc)

> script to access exim's functionality.  (Otherwise the script would be
> executed as just another Python script, which would not be too useful
> from the embedding standpoint.)

Correct, it's not executed as a standalone script.

> In other words, you're not calling the functions from the script: the
> script is calling your functions.  The only time that you are calling a
> script's function is when the script passes a function to one of your
> functions that accepts a callback argument -- but even then the

Rather it's "imported" and any of the functions may be called from any
part of the exim ACL's at any time.

> interface of the function is regulated by you, and it is you who
> prescribe how many and what kind of arguments it will accept.

I had to do a little work to figure out how many arguments the
function is written to accept, and then double check that the call
from the exim ACL's is using the correct number, and only then convert
the variables and make the function call.

Here is a more real-world example of the way I or an exim admin would
use it.  This is how I use the embedded perl to count the number of a
specifc header and set a variable which will later in the SMTP session
be used to reject this email:

  warn    !sender_domains = lsearch;/etc/exim/quarantine_skip_domains
          set acl_m_h_count = ${perl{count_headers}{from}}
          condition     = ${if >{$acl_m_h_count}{1}}
          control       = fakereject/Quarantined: RFC 5322 limits an
email to one From header, yours had $acl_m_h_count.\nYour email is
being kept for evaluation.  If it was a legitimate\nmessage, it may
still be delivered to the target recipient(s).\nRejected sender
domain: $sender_address_domain
          set acl_m_quarantine = 1

The perl function count_headers() gets passed one variable: "from",
stored in a variable named $header.  The perl function is written to
load the headers from the message (a big text blob), convert it to an
array, then loop through the array looking for /^$header:/, counting
how many it finds.  Then it returns the number.

The "condition" line simply compares the returned value to 1, and if
it's greater than one, it will set the fakereject and quarantine
settings.

My goal with this explanation is to show that the intended usage of
python is very simple.  It will not be used for threading.  It will
not be used for large amounts of the processing of the email.  It is
intended to do small calculations (reputation querying, reputation
scoring, detecting anomalies in the email, etc).  The goal is not to
write a massive python system.  The goal is to make parts of the exim
ACL configuration easier by implementing some of the more
complicated-to-do-in-exim parts in a language you are familiar with.
I use the embedded perl a lot, and I am adding embedded python since a
few people have asked for the same python functionality.

My code currently does not handle exceptions that occur in the python
functions.  I'm still working on that.

> What would be the point of such a function, before or after the edit?
> You cannot just call any random function from a script: you need to
> execute the script, providing a way for the script to call *you*.

The random function would be called from exim's ACL as it's processing
a message.

...Todd
--
The total budget at all receivers for solving senders' problems is $0.
 If you want them to accept your mail and manage it the way you want,
send it the way the spec says to. --John Levine

From hniksic at xemacs.org  Mon Jun 10 23:09:34 2013
From: hniksic at xemacs.org (Hrvoje Niksic)
Date: Mon, 10 Jun 2013 23:09:34 +0200
Subject: [capi-sig] Constructing variable arg tuples
In-Reply-To: <CAFG21ojfQx6YKV0sCnps-GFZABqv3591jNRsKV1Nwcs=cAR-TQ@mail.gmail.com>
	(Todd Lyons's message of "Sat, 8 Jun 2013 21:07:16 -0700")
References: <CAFG21og0fmj9XnNx+T6ZQ3fZnwEoaJNne2SpNLZO7NV2xoyNQg@mail.gmail.com>
	<878v2lk5wr.fsf@xemacs.org>
	<CAFG21ojfQx6YKV0sCnps-GFZABqv3591jNRsKV1Nwcs=cAR-TQ@mail.gmail.com>
Message-ID: <87a9mxbrld.fsf@xemacs.org>

Todd Lyons <tlyons at ivenue.com> writes:

>> Obviously the scripts and your embedding module need to agree on an API;
>
> Yes.  I've set the following limitations:
> 1. All args passed to a python function are converted to strings.  It
> is up to the function to include an implicit cast
> 2. No dicts, tuples, etc are passed.  Just simple strings or numbers
> (which are converted to strings).
> 3. Return from a function is always cast back to a string, which is
> what exim prefers.
>
> This is subject to changes and additions.

Such an interface makes sense in Perl, where you can always treat
numbers and strings in vice-versa, but might be somewhat inconvenient in
Python, which is much more "strongly" typed (you can query for an
object's type, and things like 1+"2" fail).

If my understanding is correct, you want exim configuration to define
which Python functions will be called by exim, without an API defined
upfront.

>> interface of the function is regulated by you, and it is you who
>> prescribe how many and what kind of arguments it will accept.
>
> I had to do a little work to figure out how many arguments the
> function is written to accept, and then double check that the call
> from the exim ACL's is using the correct number, and only then convert
> the variables and make the function call.

Why would you need to do that -- can't you just call the function and
leave it to raise an exception if called with an incorrect argument
count?  Not only for EAFP, but because inspecting how many argument a
function can fail.  After all, the function could be imported from a
module written in C or Boost.Python, or accept *args, which will likely
confuse count detection.

> The perl function count_headers() gets passed one variable: "from",
> stored in a variable named $header.  The perl function is written to
> load the headers from the message (a big text blob), convert it to an
> array, then loop through the array looking for /^$header:/, counting
> how many it finds.  Then it returns the number.
>
> The "condition" line simply compares the returned value to 1, and if
> it's greater than one, it will set the fakereject and quarantine
> settings.

This will work and is easy to implement, but it does feel a bit more
Perlish than Pythonic.  If you want to go for a solution with a more
native Python accent, I'd recommend passing the Python function some
sort of context argument.  Instead of a string, this would be an object
with some methods that retrieve the message information, such as
headers, and others to affect the response.  This needn't be a complex
framework -- if designed carefully, excellent functionality can be
achieved with a very small number of methods.

From tlyons at ivenue.com  Tue Jun 11 14:46:26 2013
From: tlyons at ivenue.com (Todd Lyons)
Date: Tue, 11 Jun 2013 05:46:26 -0700
Subject: [capi-sig] Constructing variable arg tuples
In-Reply-To: <87a9mxbrld.fsf@xemacs.org>
References: <CAFG21og0fmj9XnNx+T6ZQ3fZnwEoaJNne2SpNLZO7NV2xoyNQg@mail.gmail.com>
	<878v2lk5wr.fsf@xemacs.org>
	<CAFG21ojfQx6YKV0sCnps-GFZABqv3591jNRsKV1Nwcs=cAR-TQ@mail.gmail.com>
	<87a9mxbrld.fsf@xemacs.org>
Message-ID: <CAFG21oiquJA0Ez=aWUHnwKPqDBSupEgjTBbyEvF+y7VWNUSZkA@mail.gmail.com>

On Mon, Jun 10, 2013 at 2:09 PM, Hrvoje Niksic <hniksic at xemacs.org> wrote:
>
> If my understanding is correct, you want exim configuration to define
> which Python functions will be called by exim, without an API defined
> upfront.

Thank you for phrasing it properly, I was doing such a poor job.  That
is exactly what I am doing.

>> I had to do a little work to figure out how many arguments the
>> function is written to accept, and then double check that the call
>> from the exim ACL's is using the correct number, and only then convert
>> the variables and make the function call.
> Why would you need to do that -- can't you just call the function and
> leave it to raise an exception if called with an incorrect argument count?

 Because it segfaulted instead of raising an exception.  If you remove
lines 151-158 in python.c
(http://git.exim.org/users/tlyons/exim.git/blob/4469a42e017cb6e44646d453b86e4f987fec790f:/src/src/python.c),
then it will segfault when the function is called with the incorrect
number of args.

> This will work and is easy to implement, but it does feel a bit more
> Perlish than Pythonic.  If you want to go for a solution with a more
> native Python accent, I'd recommend passing the Python function some
> sort of context argument.  Instead of a string, this would be an object
> with some methods that retrieve the message information, such as
> headers, and others to affect the response.  This needn't be a complex
> framework -- if designed carefully, excellent functionality can be
> achieved with a very small number of methods.

Noted, thanks for the input.  That's good stuff.

...Todd
--
The total budget at all receivers for solving senders' problems is $0.
 If you want them to accept your mail and manage it the way you want,
send it the way the spec says to. --John Levine