HS512256 OpenSSL Exception: First num too large

[RootTJNII]

When trying to use the HS512256 algorithm I'm seeing Unsupported digest algorithm (sha512256).: first num too large. I've narrowed this down to the OpenSSL::Digest.new("sha512256") call in https://github.com/jwt/ruby-jwt/blob/master/lib/jwt/algos/hmac.rb#L14

irb(main):001:0> require 'openssl'
=> true
irb(main):002:0> "HS512256".sub('HS', 'sha')
=> "sha512256"
irb(main):003:0> OpenSSL::Digest.new("sha512256")
Traceback (most recent call last):
        6: from /usr/local/bin/irb:23:in `<main>'
        5: from /usr/local/bin/irb:23:in `load'
        4: from /usr/local/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        3: from (irb):3
        2: from (irb):3:in `new'
        1: from (irb):3:in `initialize'
RuntimeError (Unsupported digest algorithm (sha512256).: first num too large)

The ruby:2.6 container has the latest released OpenSSL gem per https://github.com/ruby/openssl/releases

$ docker run --rm -ti ruby:2.6 gem list openssl

*** LOCAL GEMS ***

openssl (default: 2.1.2)

I also don't see this algorithm in https://ruby-doc.org/stdlib-2.6/libdoc/openssl/rdoc/OpenSSL/Digest.html. Is SHA512256 correct?

[anakinj]

I think the issue is that to be able to use the SHA-512-256 algorithm you need to have the rbnacl gem and libsodium installed.

This is probably stated somewhere in the docs, but the code is not that good in showing what algorithms are supported natively and what require some extra things.

[RootTJNII]

Aah, yea, that's the problem. With rbnacl installed it works, and I see this is documented in the readme. Some sort of gate would be nice, like making that algo unavailable without rbnacl installed, but I'm sure the value of that feature will be outweighed by the weight of the implementation.

This can be closed as user error, thanks!

[khiav223577]

It should be OpenSSL::Digest.new("sha512-256") instead of OpenSSL::Digest.new("sha512256")
(I'm googling how to do SHA512-256 in ruby and find this issue)

[anakinj]

This is great. We could drop the rbnacl dependency for the hmac with sha512256 algo. I wonder if this algorithm was added to the openssl parts somewhere along the way.

[anakinj]

Seems to be working on Ruby 2.5 and openssl gem 2.1.2

docker run -it ruby:2.5
irb(main):001:0> require 'openssl'
=> true
irb(main):002:0> OpenSSL::Digest.new("sha512-256")
=> #<OpenSSL::Digest: c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a>
irb(main):003:0> OpenSSL::VERSION
=> "2.1.2"
irb(main):004:0>

On Ruby 2.3 it does not work:

docker run -it ruby:2.3
irb(main):001:0> require 'openssl'
=> true
irb(main):002:0> OpenSSL::Digest.new("sha512-256")
RuntimeError: Unsupported digest algorithm (sha512-256).: first num too large
	from (irb):2:in `initialize'
	from (irb):2:in `new'
	from (irb):2
	from /usr/local/bin/irb:11:in `<main>'

[anakinj]

I looked into this a bit more. Turns out that the HS512256 using ::RbNaCl::HMAC::SHA512256 is just a truncated version of HS512. The SHA-512/256 is a completely different signing algorithm.

The JWA RFC does not even include anything called HS512256. Think there might have been some misunderstanding back in the day when support for this algorithm was implemented.

I think we should just drop the support of this HS512256 and keep to what the standard states