[BangPypers] eval (was: BangPypers Digest, Vol 86, Issue 16)

[Noufal Ibrahim KV]

Please ignore my earlier reply to your email. I hit send by accident.

Please edit the subject line when you're replying to a digest.

On Mon, Oct 27 2014, Okan bhan wrote:


[...]

>> 2. A mapping object is similar to a dictionary. It should
>>    return the actual object if you try to access it. Not a boolean True
>>    or False.
>>
> Agree with you. Ideally it should have returned object and I can test
> by `if obj` or `if not obj`. Lets assume for now I want to continue
> with this.

I didn't understand what you mean by this. What are you attempting to
test with the `if` statements? And what do you want to continue with?

>> The mapping objects given are specifications of the locals and globals
>> in which you will evaluate your string. Consult help("eval") for more
>> information.
>>
>>     eval(source[, globals[, locals]]) -> value
>>
> Makes sense. So essentially I'm trying to evaluate and expression
> ('foo' or 'foo and bar') against a local mapping. Not fully confident
> but I can see the use of locals & globals here.

Every statement needs an evaluation context. Suppose I ask what the
result of the statement `x+2` is, you need a context which provides the
value of `x` to be able to answer meaningfully.

Python has two such contexts. First is the local context which is inside
the function and the second is the global context which is at the module
level.

Without these, the source referred to above cannot be `eval`uated.

[...]

> That is strange, I'm still getting 5.6 as result. I tried on different
> python versions as well.

What version of Python are you using?


[...]

>> 5.6 is not an attribute access. 5.6 is is a floating point number. aa.a
>> is an attribute access. It will first return False for `aa` since there
>> is no such key and your code returns False and then try to access `a` in
>> the Boolean which, predictably, fails.
>>
> Big thanks for this. Let me explain what, I understood here.  Lets say
> statement we want to evaluate is `eval(expression)`. Python starts
> checking 'expression' against each datatype starting from int -> float
> -> str. For all native datatypes, it will return corresponding
> result. Finally if it doesn't matches any of native types, tries to
> access against mapping(or any other object in local or global scope).

This is a theory that explains the observations but I can't fully
endorse your wording. Statements like "Python starts *checking*
'expression' against each *datatype* *starting* from int -> float ->
str" is too slipshod for my tastes. 

Expressions (and I use this to mean the expr_stmt in the grammar[1]),
have to be evaluated. A variable is an expression and it is evaluated by
a symbol lookup. It is looked up first in the local and on failure, in
the global context.

> Which is the reason,
>>> type(eval('5.6'))   is float
>>> type(eval("'5.6'") is str  (expression is under double quotes).
>
> For the case of eval('aa.a'), it first tries to match 'aa'. This is not any
> native type so goes to check in mapping and finds it True. Then tries to
> find attribute 'a' of returned type (a boolean). Hence boolean object has
> no attribute 'a'.

More or less but it doesn't try to "match" `aa`. The expression first
tokenised. You can see this. 

1,0-1,2:        NAME    'aa'
1,2-1,3:        OP      '.'
1,3-1,4:        NAME    'a'
2,0-2,0:        ENDMARKER       ''

This will get reduced into an expression and then be evaluated. The `aa`
is looked up and then the semantics indicate that an attribute lookup
should take place. 

[...]

> I came across blog posts suggesting not to use eval mostly as it can lead
> to un-secure code. So wanted to check should I put efforts in solving this
> with 'eval' or not. Not an option now. I will explain my problem in next
> question.

evaling code which you have complete control over and which you generate
is as safe as general code (although trickier to debug). If there's some
tainted data in the string that you eval, the effects might be
unpredictable.


[...]

> yes. If my above observation is correct, this is clear to me. I will try to
> explain my problem:
>
> I have a list of strings ( say ['aaa', 'bbb', '4.5-'] ) and I want to check
> if an expression (say 'aa') is a substring of any of this item.
>
> Simply:   all([ item for item in ['aaa', 'bbb', '4.5-'] if 'aa' in item])
>   ==> gives True as 'aa' is a substring of 'aaa'.

Fair enough. That makes sense.

> But my expression can be a logical expression like 'aa or bb',  'aa and
> bbb', 'aaa and 4.5' etc.

I'd do this by creating a predicate function
e.g.

  def test(x):
    'aa' in x or 'bb' in x

and then

  all(item for item in ['aaa', 'bbb', '4.5-'] if test(item))


> This works in my above implementation. If we assume mapping object from
> above. Writing it again here for ease.
>
>> class SimpleMapping:
>>   def __init__(self, items):
>>     self._items = items
>>
>>   def __getitem__(self, subitem):
>>     for item in self._items:
>>       if subitem in item:
>>         return True
>>     return False
>
>>>> mapping = SimpleMapping(set(['aaa', 'bbb', '4.5-']))
>
>>>>   eval('aa and bb', {}, mapping)
> True                                                          # since both
> are substring
>>>>  eval('aa and bbb', {}, mapping)
> True                                                          # again since
> both are substring
>>>>  eval('aa and bbbb', {}, mapping)
> False                                                         # 'bbbb'
> fails.

Okay. I think I see what you're trying to do. You're creating something
like a mapping which contains these strings and then evaling a boolean
expression which just does a lookup. 

This works but it contorted and hard to read/maintain. I wouldn't do it
this way. I'd use a predicate.

> So till now it works fine for my case. But in case if I want to check for
> '4.5' in this.
>
>>>> eval('aaaa or bbbb or 4.5')
> 4.5                                                           # Fails for
> 'aaaa', fails for 'bbbb' but evaluates '4.5' as 4.5. Something I don't
> want. It should be
>              checking 4.5 against mapping and I want a boolean True here.
>
> If I am unclear, I will repeat my problem again.

Eval is not suited for the problem. I suppose you could do some quoting
magic to take care of this but, like I said, it just makes it more and
more contorted.

When you eval something, it's treated as a piece of python code. You
however want to check for strings.


> Problem:   How can I evaluate a logical expression, in python where
> expression contains a dot ('.').

This is not a complete statement. 

   file1.opened and file2.opened

is a logical expression which has



[...]

> Sorry for long reply and redundant answer but I wanted to be clear. If I
> was able to explain properly can you suggest anything here?

[...]

I'd use the predicate approach.

Footnotes: 
[1]  https://docs.python.org/2/reference/grammar.html

-- 
Cordially,
Noufal
http://nibrahim.net.in