[Python-Dev] [PATCH] Add an authorization header to the initial request.

[Matěj Cepl]

Many websites (e.g. GitHub API) on the Internet are intentionally not
following RFC with regards to the Basic Authorization and require
Authorization header in the initial request and they never return 401
error. Therefore it is not possible to authorize with such websites just
using urllib2.py HTTPBasicAuthHandler as described in documentation.

However, RFC 2617, end of section 2 allows pre-authorization in the
initial request:

> A client MAY preemptively send the corresponding Authorization
> header with requests for resources in that space without
> receipt of another challenge from the server.

(RFC also suggests preauthorization of proxy requests, but that is not
part of this patch, however it could be trivially added)

Also, generating HTTP BasicAuth header has been refactored into special
method of AbstractBasicAuthHandler.

Suggested fix for bug# 19494

This is my first attempt to contribute to Python itself, so please be
gentle with me. Yes, I know that I miss unit tests and port to other
branches of Python (this is against 2.7), but I would like first some
feedback to see what I am missing (aside from the mentioned).

Matěj Cepl

---
 Lib/urllib2.py | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/Lib/urllib2.py b/Lib/urllib2.py
index aadeb73..a5feb03 100644
--- a/Lib/urllib2.py
+++ b/Lib/urllib2.py
@@ -848,6 +848,18 @@ class AbstractBasicAuthHandler:
     def reset_retry_count(self):
         self.retried = 0
 
+    def generate_auth_header(self, host, req, realm):
+        user, pw = self.passwd.find_user_password(realm, host)
+        if pw is not None:
+            raw = "%s:%s" % (user, pw)
+            auth = 'Basic %s' % base64.b64encode(raw).strip()
+            if req.headers.get(self.auth_header, None) == auth:
+                return None
+            req.add_unredirected_header(self.auth_header, auth)
+            return req
+        else:
+            return None
+
     def http_error_auth_reqed(self, authreq, host, req, headers):
         # host may be an authority (without userinfo) or a URL with an
         # authority
@@ -875,14 +887,10 @@ class AbstractBasicAuthHandler:
                     return response
 
     def retry_http_basic_auth(self, host, req, realm):
-        user, pw = self.passwd.find_user_password(realm, host)
-        if pw is not None:
-            raw = "%s:%s" % (user, pw)
-            auth = 'Basic %s' % base64.b64encode(raw).strip()
-            if req.headers.get(self.auth_header, None) == auth:
-                return None
-            req.add_unredirected_header(self.auth_header, auth)
-            return self.parent.open(req, timeout=req.timeout)
+        req = self.generate_auth_header(host, req, realm)
+
+        if req is not None:
+            self.parent.open(req, timeout=req.timeout)
         else:
             return None
 
@@ -898,6 +906,17 @@ class HTTPBasicAuthHandler(AbstractBasicAuthHandler, BaseHandler):
         self.reset_retry_count()
         return response
 
+    def http_request(self, req):
+        host = req.get_host()
+
+        new_req = self.generate_auth_header(host, req, None)
+        if new_req is not None:
+            req = new_req
+
+        return req
+
+    https_request = http_request
+
 
 class ProxyBasicAuthHandler(AbstractBasicAuthHandler, BaseHandler):
 
-- 
1.8.5.2.192.g7794a68