[Tutor] Python 3: string to decimal conversion

[cs at zip.com.au]

On 22May2016 08:19, Saidov <usaidov at gmail.com> wrote:
>Thank you all for the useful feedback. I am new to programming so bear
>with me while I learn the rules...
>
>I have run Cameron's code to print the values and have the traceback
>results. please see below.
[...]
>>            for row in records:
>[...]
>>                try:
>>                    expenses[ts.Date(row[0]).month] +=
>decimal.Decimal(row[4])
>>                except ValueError:
>>                    pass
[...]
>I suggest that you print out the value of row[4] before the "try"
>statement:
>  print("row[4] =", repr(row[4]))
[...]
>+ decimal <module 'decimal' from 'C:\mypath\Anaconda3\\lib\\decimal.py'> 
>module
>+ expenses {1: Decimal('0'), 2: Decimal('0'), 3: Decimal('0'), 4:
>Decimal('0'), 5: Decimal('0'), 6: Decimal('0'), 7: Decimal('0'), 8:
>Decimal('0'), 9: Decimal('0'), 10: Decimal('0'), 11: Decimal('0'), 12:
>Decimal('0')} dict
>row[0] '"1/1/2016"' str
>row[4] '""' str
>+ ts <module 'timestring' from
>'C:\mypath\Anaconda3\\lib\\site-packages\\timestring\\__init__.py'>
>module
>
>ipython traceback:
>
>row[4]= ' "" '
>
>Traceback (most recent call last):
>  File "C:\mypath\visual studio
>2015\Projects\Budget\Budget\Budget.py", line 28, in <module>
>    expenses[ts.Date(row[0]).month] += decimal.Decimal(row[4])
>decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
>
>I think the problem may be caused by an empty string value that is
>passed to decimal.Decimal function. The csv file contains some empty
>cells and I wanted the code to ignore them. That's why I had the
>ValueError exception.

I have two observations here:

Your strings in row[4] are not empty. If that output is repr(row[4]) then 
row[4] contains the text:

  ""

That is a two character string consisting of two quote characters.

Normally the csv.reader module will handle that for you, but I see that you 
passed the paramater:

  quoting=csv.QUOTE_NONE

to it when setting it up. It looks to me like your CSV file is a conventional 
one with quotes around string values. By passing csv.QUOTE_NONE you prevent the 
csv module from handling those for you and you get the "raw" column values. So 
a column with an empty string is probably written as "" in the file.

Could you show is the first line or so from your CSV file? That should tell us 
and you whether the file is "bare" comma separate values or the far more common 
quoted format.

If it is the quoted format, just remove the "quoting=csv.QUOTE_NONE" parameter 
altogether - the default for the csv module is quoted and it is _usually_ what 
is wanted. Of course you need to know one way or the other, so examine the CSV 
file itself.

The other observation is that you're trying to catch ValueError, when plainly 
the Decimal module is raising decimal.InvalidOperation. So you should catch 
that instead.

HOWEVER, just catching it and ignoring that row will _silently_ discard good 
input if you program is incorrect. You should almost always emit an error 
message or perform some other very specific action when you catch an exception.

Alan has suggested that you test specificly for an empty string.

I agree: you should act on exactly what is expected. By blindly catching 
ValueError or decimal.InvalidOperation and not reporting the string that caused 
it to happen you will silently ignore all kinds of unexpected input.

So I would advocate some code like this, similar to Alan's:

  if not row[4]:
    # empty column - we expect this and ignore it
    pass
  else:
    try:
      expenses[ts.Date(row[0]).month] += decimal.Decimal(row[4])
    except decimal.InvalidOperation as e:
      print("unexpected expenses value: %r" % (row[4],))

which will report the offending values, and presumably you expect either an 
empty column or a _valid_ expense value.

Cheers,
Cameron Simpson <cs at zip.com.au>