[New-bugs-announce] [issue10281] Exception raised when an NNTP overview field is absent

[Julien ÉLIE]

New submission from Julien ÉLIE <julien at trigofacile.com>:

Following the first example of the documentation:

import nntplib

s = nntplib.NNTP('news.trigofacile.com')
resp, count, first, last, name = s.group('fr.comp.lang.python')
print('Group', name, 'has', count, 'articles, range', first, 'to', last)
resp, overviews = s.over((last - 9, last))
for id, over in overviews:
    print(id, nntplib.decode_header(over['subject']))
s.quit()


An exception is raised:
"OVER/XOVER response doesn't include names of additional headers"


I believe the issue comes from the fact that the source code does not
handle the case described in Section 8.3.2 of RFC 3977:

   For all fields, the value is processed by first removing all CRLF
   pairs (that is, undoing any folding and removing the terminating
   CRLF) and then replacing each TAB with a single space.  If there is
   no such header in the article, no such metadata item, or no header or
   item stored in the database for that article, the corresponding field
   MUST be empty.

   Example of a successful retrieval of overview information for a range
   of articles:

      [C] GROUP misc.test
      [S] 211 1234 3000234 3002322 misc.test
      [C] OVER 3000234-3000240
      [S] 224 Overview information follows
      [S] 3000234|I am just a test article|"Demo User"
          <nobody at example.com>|6 Oct 1998 04:38:40 -0500|
          <45223423 at example.com>|<45454 at example.net>|1234|
          17|Xref: news.example.com misc.test:3000363
      [S] 3000235|Another test article|nobody at nowhere.to
          (Demo User)|6 Oct 1998 04:38:45 -0500|<45223425 at to.to>||
          4818|37||Distribution: fi
      [S] 3000238|Re: I am just a test article|somebody at elsewhere.to|
          7 Oct 1998 11:38:40 +1200|<kfwer3v at elsewhere.to>|
          <45223423 at to.to>|9234|51
      [S] .

   Note the missing "References" and Xref headers in the second line,
   the missing trailing fields in the first and last lines, and that
   there are only results for those articles that still exist.






Also please note that nntplib should also work in case the database is not consistent.  Some news servers might be broken and do not follow the MUST NOT...

   The LIST OVERVIEW.FMT command SHOULD list all the fields for which
   the database is consistent at that moment.  It MAY omit such fields
   (for example, if it is not known whether the database is consistent
   or inconsistent).  It MUST NOT include fields for which the database
   is inconsistent or that are not stored in the database.  Therefore,
   if a header appears in the LIST OVERVIEW.FMT output but not in the
   OVER output for a given article, that header does not appear in the
   article (similarly for metadata items).

----------
components: Extension Modules
messages: 120158
nosy: jelie
priority: normal
severity: normal
status: open
title: Exception raised when an NNTP overview field is absent
type: behavior
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10281>
_______________________________________