[Cryptography-dev] Parsing DER from PE File

Robert Simmons rsimmons0 at gmail.com
Sat Dec 29 01:19:59 EST 2018 

I also am having trouble parsing the extensions of the attached cert using
the code above:

Traceback (most recent call last):
  File "./extract_sigs.py", line 65, in <module>
    for extension in cert.extensions:
  File "/root/sigs/lib/python3.7/site-packages/cryptography/utils.py", line
162, in inner
    result = func(instance)
  File
"/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/x509.py",
line 137, in extensions
    self._backend, self._x509
  File
"/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py",
line 252, in parse
    value = handler(backend, ext_data)
  File
"/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py",
line 342, in _decode_basic_constraints
    return x509.BasicConstraints(ca, path_length)
  File
"/root/sigs/lib/python3.7/site-packages/cryptography/x509/extensions.py",
line 345, in __init__
    raise ValueError("path_length must be None when ca is False")
ValueError: path_length must be None when ca is False

On Sat, Dec 29, 2018 at 12:32 AM Robert Simmons <rsimmons0 at gmail.com> wrote:

> I do need all three certs. I do see what PKCS7_get0_signers does now.
>
> On Wed, Dec 26, 2018 at 11:27 AM Paul Kehrer <paul.l.kehrer at gmail.com>
> wrote:
>
>> We haven't had anyone request support for those legacy extension types,
>> but if you think you need it feel free to file an issue and we can discuss
>> adding it. The data can be parsed out of the UnknownExtension type right
>> now of course.
>>
>> So you need all 3 certs? Only one of them is used for signing which is
>> why the PKCS7_get0_signers call only returns that one. To obtain the rest
>> we'll need to de-opaque two PKCS7 structs in cryptography: PKCS7_ENVELOPE
>> and PKCS7_SIGN_ENVELOPE. Your use case should only require SIGN_ENVELOPE
>> de-opaqued but might as well get them both.
>>
>> -Paul
>>
>>
>>
>> On December 25, 2018 at 9:15:10 PM, Robert Simmons (rsimmons0 at gmail.com)
>> wrote:
>>
>> On a side note: there is one oid in the extensions of this cert that is
>> listed as unknown, but openssl parses it as:
>> Netscape Cert Type:
>>     Object Signing
>>
>> Is this something to submit a bug for?
>>
>> Also, happy holidays!
>>
>> On Tue, Dec 25, 2018 at 9:41 PM Robert Simmons <rsimmons0 at gmail.com>
>> wrote:
>>
>>> Thanks for the help above. However, I think I'm still missing something.
>>> When piping the DER binary data to openssl on the command line, the output
>>> appears to have three certificates in the example DER early in this thread.
>>> The code above has a list for certs, but it appears to only contain one
>>> cert at the end of the for loop. Is there a way to view the data from the
>>> other two? I've attached the output from openssl command line.
>>>
>>> On Mon, Dec 24, 2018 at 11:51 AM Paul Kehrer <paul.l.kehrer at gmail.com>
>>> wrote:
>>>
>>>> Great! I have an idea of how to implement an API for this limited
>>>> subset of pkcs7 as a utility function like the pkcs12 support we recently
>>>> merged. Hopefully I or someone else can get to it soon.
>>>>
>>>> -Paul
>>>>
>>>> On Dec 23, 2018, at 6:32 PM, Robert Simmons <rsimmons0 at gmail.com>
>>>> wrote:
>>>>
>>>> This works great! Thanks!
>>>>
>>>> On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer <paul.l.kehrer at gmail.com>
>>>> wrote:
>>>>
>>>>> One day I will learn to run the code I write before I ask people to
>>>>> use it. The missing signers variable should go after the pkcs7 assignment.
>>>>> It looks like this:
>>>>>
>>>>> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0)
>>>>>
>>>>> With that in place and using the extracted.der you previously provided
>>>>> I can parse a cert, which has the following subject/issuer data:
>>>>>
>>>>>         Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA
>>>>> Limited, CN=COMODO RSA Code Signing CA
>>>>>         Validity
>>>>>             Not Before: Oct 19 00:00:00 2018 GMT
>>>>>             Not After : Sep 25 23:59:59 2019 GMT
>>>>>         Subject: C=GB/postalCode=WA1 1RG, ST=UK,
>>>>> L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK,
>>>>> LIMITED, CN=TATIANA PUK, LIMITED
>>>>>
>>>>> I've also attached the cert. If this is what you're looking for then
>>>>> your use case is covered by the existing issue, although I still need to
>>>>> decide on an API for this.
>>>>>
>>>>> -Paul
>>>>>
>>>>>
>>>>>
>>>>> On December 23, 2018 at 2:17:54 AM, Robert Simmons (
>>>>> rsimmons0 at gmail.com) wrote:
>>>>>
>>>>> import os
>>>>> import pathlib
>>>>> import pefile
>>>>>
>>>>> target =
>>>>> pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe')
>>>>> fname = str(target)
>>>>> totsize = os.path.getsize(target)
>>>>> pe = pefile.PE(fname)
>>>>>
>>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']])
>>>>> sigoff = 0
>>>>> siglen = 0
>>>>> for s in pe.__structures__:
>>>>>     if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY':
>>>>>         sigoff = s.VirtualAddress
>>>>>         siglen = s.Size
>>>>> pe.close()
>>>>> with open(fname, 'rb') as fh:
>>>>>     fh.seek(sigoff)
>>>>>     thesig = fh.read(siglen)
>>>>>
>>>>> from cryptography.hazmat.backends.openssl.backend import backend
>>>>> from cryptography.hazmat.backends.openssl import x509
>>>>>
>>>>> bio = backend._bytes_to_bio(thesig[8:])
>>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL)
>>>>> certs = []
>>>>> for i in range(backend._lib.sk_X509_num(signers)):
>>>>>     x509_ptr = backend._lib.sk_X509_value(signers, i)
>>>>>     certs.append(x509._Certificate(backend, x509_ptr))
>>>>>
>>>>> That's the exact code I'm trying to run with the provided code snippet
>>>>> at the end. If you want to follow along with the exact file I'm working
>>>>> with:
>>>>> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8
>>>>>
>>>>> The password to that file is "infected" and btw: it is live malware,
>>>>> so please treat it accordingly. Run code on it in a safe environment for
>>>>> handling malware.
>>>>>
>>>>> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons <rsimmons0 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I've added the use case to the issue as requested. I tried the code
>>>>>> snippet, but the contents of signers is missing. What should that be?
>>>>>>
>>>>>> NameError: name 'signers' is not defined
>>>>>>
>>>>>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer <paul.l.kehrer at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Out of curiosity, does the following code load the cert you expect?
>>>>>>> der should be the bytes of extracted.der:
>>>>>>>
>>>>>>> from cryptography.hazmat.backends.openssl.backend import backend
>>>>>>> from cryptography.hazmat.backends.openssl import x509
>>>>>>>
>>>>>>> bio = backend._bytes_to_bio(der)
>>>>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL)
>>>>>>> certs = []
>>>>>>> for i in range(backend._lib.sk_X509_num(signers)):
>>>>>>>     x509_ptr = backend._lib.sk_X509_value(signers, i)
>>>>>>>     certs.append(x509._Certificate(backend, x509_ptr))
>>>>>>>
>>>>>>> Certs will be a list of signer certificates -- in this case, just
>>>>>>> one cert in the list. Please note that this code does not manage memory
>>>>>>> correctly so it should strictly be used to test if the cert you need is
>>>>>>> being properly extracted :)
>>>>>>>
>>>>>>> -Paul (reaperhulk)
>>>>>>>
>>>>>>>
>>>>>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer (
>>>>>>> paul.l.kehrer at gmail.com) wrote:
>>>>>>>
>>>>>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7
>>>>>>> envelope holding multiple certificates and at the moment cryptography
>>>>>>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind
>>>>>>> sharing your use case directly on
>>>>>>> https://github.com/pyca/cryptography/issues/3983 then it will help
>>>>>>> me when I'm prioritizing features for upcoming releases.
>>>>>>>
>>>>>>> -Paul
>>>>>>>
>>>>>>>
>>>>>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons (
>>>>>>> rsimmons0 at gmail.com) wrote:
>>>>>>>
>>>>>>> Definitely. I've attached the DER data as extracted from the PE file
>>>>>>> using the following code:
>>>>>>>
>>>>>>> pe = pefile.PE(fname)
>>>>>>>
>>>>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']])
>>>>>>> sigoff = 0
>>>>>>> siglen = 0
>>>>>>> for s in pe.__structures__:
>>>>>>>     if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY':
>>>>>>>         sigoff = s.VirtualAddress
>>>>>>>         siglen = s.Size
>>>>>>> pe.close()
>>>>>>> with open(fname, 'rb') as fh:
>>>>>>>     fh.seek(sigoff)
>>>>>>>     thesig = fh.read(siglen)
>>>>>>> with open('extracted.der', 'wb') as fh:
>>>>>>>     fh.write(thesig[8:])
>>>>>>>
>>>>>>> I've attached extracted.der as a zip file to maintain integrity as
>>>>>>> an attachment.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer <
>>>>>>> paul.l.kehrer at gmail.com> wrote:
>>>>>>>
>>>>>>>> Could you give us an example (in hex or b64 or something) so we can
>>>>>>>> easily reproduce? Make sure any certs you're giving us don't contain
>>>>>>>> sensitive data of course.
>>>>>>>>
>>>>>>>> -Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons (
>>>>>>>> rsimmons0 at gmail.com) wrote:
>>>>>>>>
>>>>>>>> I've asked this question on Stack Overflow here:
>>>>>>>> https://stackoverflow.com/q/53862702/1033217
>>>>>>>>
>>>>>>>> I have compared my code to Dider Stevens's disitool here (examine
>>>>>>>> the function ExtractDigitalSignature):
>>>>>>>>
>>>>>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py
>>>>>>>>
>>>>>>>> When I load that extracted file into a variable and try to parse it
>>>>>>>> with cryptography, it fails. If I pipe the same file to openssl on the
>>>>>>>> command line, it works.
>>>>>>>>
>>>>>>>> I am thinking this has to do with the number of certificates in the
>>>>>>>> directory in the PE file. There can be three (cert, intermediate CA, and
>>>>>>>> CA, etc).
>>>>>>>>
>>>>>>>> Any idea what's going on?
>>>>>>>> _______________________________________________
>>>>>>>> Cryptography-dev mailing list
>>>>>>>> Cryptography-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Cryptography-dev mailing list
>>>>>>>> Cryptography-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Cryptography-dev mailing list
>>>>>>> Cryptography-dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Cryptography-dev mailing list
>>>>>>> Cryptography-dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>> Cryptography-dev mailing list
>>>>> Cryptography-dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>>
>>>>> _______________________________________________
>>>>> Cryptography-dev mailing list
>>>>> Cryptography-dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>>
>>>> _______________________________________________
>>>> Cryptography-dev mailing list
>>>> Cryptography-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>
>>>> _______________________________________________
>>>> Cryptography-dev mailing list
>>>> Cryptography-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>>>
>>> _______________________________________________
>> Cryptography-dev mailing list
>> Cryptography-dev at python.org
>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>
>> _______________________________________________
>> Cryptography-dev mailing list
>> Cryptography-dev at python.org
>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20181229/4dcd57ec/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: e226bb734a0781b62f2f5a27216aba2b.der
Type: application/x-x509-ca-cert
Size: 1584 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20181229/4dcd57ec/attachment-0001.crt>

More information about the Cryptography-dev mailing list