0.17.0: pytest is failing

[kloczek]

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-ecdsa-0.17.0-5.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-ecdsa-0.17.0-5.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/python-ecdsa-python-ecdsa-0.17.0
plugins: hypothesis-6.36.0
collected 1464 items / 1 error / 1463 selected

================================================================================== ERRORS ==================================================================================
____________________________________________________________ ERROR collecting src/ecdsa/test_malformed_sigs.py _____________________________________________________________
/usr/lib64/python3.8/hashlib.py:157: in __hash_new
    return _hashlib.new(name, data)
E   ValueError: [digital envelope routines] initialization error

During handling of the above exception, another exception occurred:
src/ecdsa/test_malformed_sigs.py:41: in <module>
    hash_and_size = [
src/ecdsa/test_malformed_sigs.py:42: in <listcomp>
    (name, hashlib.new(name).digest_size) for name in algorithms_available
/usr/lib64/python3.8/hashlib.py:163: in __hash_new
    return __get_builtin_constructor(name)(data)
/usr/lib64/python3.8/hashlib.py:120: in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
E   ValueError: unsupported hash type whirlpool
========================================================================= short test summary info ==========================================================================
ERROR src/ecdsa/test_malformed_sigs.py - ValueError: unsupported hash type whirlpool
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================= 1 error in 1.17s =============================================================================

[tomato42]

That seems like a misconfigured environment.

The list of tested algorithms is taken from algorithms_available list:

python-ecdsa/src/ecdsa/test_malformed_sigs.py

Line 43 in 3b49fbe

(name, hashlib.new(name).digest_size) for name in algorithms_available

And that list is imported from hashlib:

python-ecdsa/src/ecdsa/test_malformed_sigs.py

Lines 5 to 15 in 3b49fbe

	try:
	from hashlib import algorithms_available
	except ImportError: # pragma: no cover
	algorithms_available = [
	"md5",
	"sha1",
	"sha224",
	"sha256",
	"sha384",
	"sha512",
	]

the fallback doesn't include whirlpool...

So it's the hashlib module that provides incorrect information

[tomato42]

Also, can't reproduce it on a clean Fedora 35 machine:

[root@fedora ~]# python3.8
Python 3.8.12 (default, Aug 30 2021, 00:00:00) 
[GCC 11.2.1 20210728 (Red Hat 11.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from hashlib import algorithms_available 
>>> algorithms_available
{'sha512_224', 'sha3_224', 'sha3_512', 'sm3', 'sha3_384', 'ripemd160', 'md5-sha1', 'sha512', 'sha256', 'sha224', 'sha512_256', 'sha1', 'blake2s', 'sha384', 'md4', 'whirlpool', 'md5', 'sha3_256', 'shake_128', 'blake2b', 'shake_256'}
>>> import hashlib
>>> a = {(name, hashlib.new(name).digest_size) for name in algorithms_available}
>>> a
{('blake2b', 64), ('sm3', 32), ('sha3_384', 48), ('blake2s', 32), ('md5-sha1', 36), ('md5', 16), ('shake_128', 0), ('sha512_256', 32), ('sha3_224', 28), ('sha256', 32), ('sha512_224', 28), ('sha512', 64), ('md4', 16), ('sha224', 28), ('sha3_256', 32), ('ripemd160', 20), ('sha384', 48), ('whirlpool', 64), ('sha1', 20), ('sha3_512', 64), ('shake_256', 0)}

[kloczek]

Just done the same test

[tkloczko@devel-g2v SPECS]$ python3
Python 3.8.12 (default, Jan 16 2022, 10:34:40)
[GCC 11.2.1 20211203 (Red Hat 11.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> algorithms_available
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'algorithms_available' is not defined
>>> from hashlib import algorithms_available
>>> algorithms_available
{'blake2b', 'sha224', 'ripemd160', 'sha3_384', 'sha512_224', 'sha512_256', 'sm3', 'blake2s', 'md4', 'sha384', 'shake_256', 'sha3_256', 'sha3_224', 'sha512', 'whirlpool', 'sha3_512', 'sha256', 'md5', 'md5-sha1', 'shake_128', 'sha1'}
>>> import hashlib
>>> a = {(name, hashlib.new(name).digest_size) for name in algorithms_available}
Traceback (most recent call last):
  File "/usr/lib64/python3.8/hashlib.py", line 157, in __hash_new
    return _hashlib.new(name, data)
ValueError: [digital envelope routines] initialization error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <setcomp>
  File "/usr/lib64/python3.8/hashlib.py", line 163, in __hash_new
    return __get_builtin_constructor(name)(data)
  File "/usr/lib64/python3.8/hashlib.py", line 120, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type ripemd160
>>>

[tomato42]

The compilation date is different... are you sure that you are using the distribution provided python?
Not some local version, compiled against openssl 3.0 instead of the 1.1.1 version?

[kloczek]

Yes I'm sure that I'm usiong python from my owen distribution 😄

That python 3.8.12 indeed is compiled against openssl 3,0.0 and up to now everything was fine.
If that is the cause it mean that it is some issue with python compiled against openssl 3.x.

[tomato42]

The issue is that algorithms like ripemd160 and whirlpool have been relegated to legacy provider in OpenSSL. That means that unless the legacy provider is explicitly loaded, they're not available. But Python should detect that and not list them as available.

I know that there have been quite a few changes around OpenSSL 3.0.0 compatibility: python/cpython#30455 so it's likely that not everything was backported to 3.8 branch.

In general, the upstream Python bug is still open: https://bugs.python.org/issue40479

[panlinux]

This is the python3 bug: python/cpython#91257

It has a few PRs listed in the comments, for different branches.

[tomato42]

For now I've filtered the problematic hashes in 82da89c, but we'll want to re-enable them in the future, when python fixes its bug.

[kloczek]

Which one bug? 🤔

[tomato42]

python/cpython#91257

[kloczek]

Just tested 0.18.0

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-ecdsa-0.18.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-ecdsa-0.18.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/python-ecdsa-python-ecdsa-0.18.0
plugins: hypothesis-6.41.0
collected 1768 items

src/ecdsa/test_curves.py .......................................................................................................                                     [  5%]
src/ecdsa/test_der.py ............................................................................                                                                   [ 10%]
src/ecdsa/test_ecdh.py .................ss.........................s.....sssssssssssss                                                                               [ 13%]
src/ecdsa/test_ecdsa.py ........................................................                                                                                     [ 16%]
src/ecdsa/test_eddsa.py ....................................................................................                                                         [ 21%]
src/ecdsa/test_ellipticcurve.py ........................                                                                                                             [ 22%]
src/ecdsa/test_jacobi.py ...................................................                                                                                         [ 25%]
src/ecdsa/test_keys.py ............................................................................................................................................. [ 33%]
.......................                                                                                                                                              [ 35%]
src/ecdsa/test_malformed_sigs.py ................................................................................................................................... [ 42%]
...............                                                                                                                                                      [ 43%]
src/ecdsa/test_numbertheory.py ..................................................................................................................................... [ 50%]
..........................................................................................................................                                           [ 57%]
src/ecdsa/test_pyecdsa.py .......................................................................................................................................... [ 65%]
.................................................................................................................................................................... [ 74%]
.................................................................................................................................................................... [ 84%]
.................................................................................................................................................................... [ 93%]
.............sssssss..ss......ssss.sssssss..ss......ssss.ss..................................                                                                        [ 98%]
src/ecdsa/test_rw_lock.py ....                                                                                                                                       [ 98%]
src/ecdsa/test_sha3.py ......s............                                                                                                                           [100%]

============================================================================= warnings summary =============================================================================
src/ecdsa/test_der.py::TestEncodeBitstring::test_implicit_unused_bits
src/ecdsa/test_der.py::TestEncodeBitstring::test_new_call_convention
src/ecdsa/test_der.py::TestRemoveBitstring::test_implicit_unexpected_unused
src/ecdsa/test_der.py::TestRemoveBitstring::test_new_call_convention
  /usr/lib64/python3.8/unittest/case.py:633: PytestRemovedIn8Warning: Passing None has been deprecated.
  See https://docs.pytest.org/en/latest/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests for alternatives in common use cases.
    method()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [2] src/ecdsa/test_ecdh.py:39: ECDH is not supported for Edwards curves
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP160r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP192r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP224r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP256r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP320r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP384r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support brainpoolP512r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support secp112r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support secp112r2
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support secp128r1
SKIPPED [1] src/ecdsa/test_ecdh.py:381: system openssl does not support secp160r1
SKIPPED [2] src/ecdsa/test_ecdh.py:376: Edwards curves are not supported for ECDH
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1033: system openssl does not support brainpoolP160r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1040: system openssl does not support brainpoolP192r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1047: system openssl does not support brainpoolP224r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1054: system openssl does not support brainpoolP256r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1061: system openssl does not support brainpoolP320r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1068: system openssl does not support brainpoolP384r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1075: system openssl does not support brainpoolP512r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:970: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:977: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:942: system openssl does not support secp112r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:949: system openssl does not support secp112r2
SKIPPED [1] src/ecdsa/test_pyecdsa.py:956: system openssl does not support secp128r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:963: system openssl does not support secp160r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1225: system openssl does not support brainpoolP160r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1232: system openssl does not support brainpoolP192r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1239: system openssl does not support brainpoolP224r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1246: system openssl does not support brainpoolP256r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1253: system openssl does not support brainpoolP320r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1260: system openssl does not support brainpoolP384r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1267: system openssl does not support brainpoolP512r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1162: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1169: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1134: system openssl does not support secp112r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1141: system openssl does not support secp112r2
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1148: system openssl does not support secp128r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1155: system openssl does not support secp160r1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1480: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_pyecdsa.py:1495: system openssl does not support prime192v1
SKIPPED [1] src/ecdsa/test_sha3.py:46: requites gmpy or gmpy2
============================================================== 1723 passed, 45 skipped, 4 warnings in 22.99s ===============================================================