simplified Python parsing question

[Eric S. Johansson]

On 7/30/2012 10:59 AM, Laszlo Nagy wrote:
>
>>
>> yeah the problem is also little more complicated than simple parsing of 
>> Python code. For example, one example (from the white paper)
>>
>> *meat space blowback = Friends and family [well-meaning attempt]
>>
>> *could that be parsed by the tools you mention?
>
> It is not valid Python code. Pygments is able to tokenize code that is not 
> valid Python code. Because it is not parsing, it is just tokenizing. But if 
> you put a bunch of random tokens into a file, then of course you will never be 
> able to split that into statements.

If you have been reading the papers, you would understand what I'm doing. I'm 
trying to take Python code with speech recognition friendly symbols and 
translate the symbols into a code friendly form. My conjecture is that you can 
change your perspective on the code and look for the edge that would normally be 
used to define start of a symbol, you should be able to define the name string. 
Another possibility is looking at the region which just contains letters numbers 
and spaces and outside and use that as your definition of a name string. It 
would probably help to verify that each word is found in a dictionary although 
that adds extra complexity if you are trying to increase the dictionary at the 
same time as the translation table.

I'm beginning to think for the first generation I should just use regular 
expressions looking forwards and backwards and try to enumerate the possible cases.
>
> Probably, you will need to process ident/dedent tokens, identify the "level" 
> of the satement. And then you can tell what file, class, inner class, method 
> you are staying in. Inside one "level" or code block, you could try to divide 
> the code into statements.

I was starting in that direction so that is good confirmation

>
> Otherwise, I have no idea how a blind person could navigate in a Python 
> source. In fact I have no idea how they use regular programs. So I'm affraid I 
> cannot help too much with this. :-(

I'm sorry, I am, and I'm trying to help, hand disabled programmers. There are 
more disability than blindness and after almost 20 years of encountering this 
shortsightedness, I do get a little cranky at times. :-)
>
>