Support older net-sftp API for Ruby 2.0 #529
Conversation
Old Rubies such as Ruby 2.0 resolve a version of net-sftp that does not have the `File#size` convenience method. To support these older versions, use the lower level `File#stat`, which works for versions of net-sftp all the way back to 2.1.2. This fixes a `NoMethodError` when SFTP is used via sshkit on Ruby 2.0.
|
@JasonPoll I'd appreciate it if you could also test this branch with your SFTP setup, when you have the chance. 🙏 |
Running into some difficulty. Surprisingly, getting Ruby 2.0.0(p648) installed was not one of them. Did you know that bundler was not a default gem back in Ruby 2.0.0? My initial set up: rbenv shell 2.0.0-p648
ruby --version
ruby 2.0.0p648 (2015-12-16 revision 53162) [x86_64-linux]
gem install -v1.17.3 bundler
rm Gemfile.lock
bundle install
bundle show --paths | grep -E "(net|ed255|bcryp|rbn)"
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/bcrypt_pbkdf-1.1.0
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/ed25519-1.2.4
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/net-scp-4.0.0
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/net-sftp-2.1.2
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/net-ssh-4.2.0
# verify my (RSA) keys are working, and that SFTP is functional to the remote host:
scp -v junk.dat vagrant@192.168.69.42:~/tmp/junk.dat
...
debug1: Offering public key: /home/vagrant/.ssh/id_rsa RSA SHA256:R42SqxvKIiLq9WA058xqybHEoq2WJlx6PyuGYB3kqHc explicit
debug1: Server accepts key: /home/vagrant/.ssh/id_rsa RSA SHA256:R42SqxvKIiLq9WA058xqybHEoq2WJlx6PyuGYB3kqHc explicit
...
debug1: Sending subsystem: sftp
junk.dat 100% 1024KB 69.0MB/s 00:00
...
Transferred: sent 1054632, received 5376 bytes, in 0.1 seconds
debug1: Exit status 0
# verify that SCP is disabled on the remote host:
scp -v -O junk.dat vagrant@192.168.69.42:~/tmp/junk.dat
...
debug1: Offering public key: /home/vagrant/.ssh/id_rsa RSA SHA256:R42SqxvKIiLq9WA058xqybHEoq2WJlx6PyuGYB3kqHc explicit
debug1: Server accepts key: /home/vagrant/.ssh/id_rsa RSA SHA256:R42SqxvKIiLq9WA058xqybHEoq2WJlx6PyuGYB3kqHc explicit
...
debug1: Sending command: scp -v -t ~/tmp/junk.dat
SCP protocol is forbidden via /etc/ssh/disable_scp
lost connection
...
debug1: Exit status 255Same test set up as in #524 - fire up a pry console and get myself set up: require 'sshkit'
require 'sshkit/dsl'
include SSHKit::DSL
def doit(ip, &block)
puts "- global transfer method: :#{SSHKit::Backend::Netssh.config.transfer_method}"
on [ip] do |host|
yield host if block_given?
as :vagrant do
within '/home/vagrant/tmp' do
upload! 'junk.dat', 'junk.dat'
download! 'junk.dat', 'junk_downloaded.dat'
end
end
end
end My issue is that when I try to use my test method, I get: doit("192.168.69.42")
- global transfer method: :scp
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/mutex_m-0.1.2/lib/mutex_m.rb:110: warning: toplevel constant Mutex referenced by Thread::Mutex
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/mutex_m-0.1.2/lib/mutex_m.rb:110: warning: toplevel constant Mutex referenced by Thread::Mutex
NotImplementedError: OpenSSH keys only supported if ED25519 is available
net-ssh requires the following gems for ed25519 support:
* rbnacl (>= 3.2, < 5.0)
* rbnacl-libsodium, if your system doesn't have libsodium installed.
* bcrypt_pbkdf (>= 1.0, < 2.0)
See https://github.com/net-ssh/net-ssh/issues/478 for more information
Gem::LoadError : "rbnacl is not part of the bundle. Add it to your Gemfile."
from /home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/net-ssh-4.2.0/lib/net/ssh/authentication/ed25519_loader.rb:19:in `raiseUnlessLoaded'The thing is, I'm not using ED25519 keys (as my earlier Am I doing something wrong, or am I missing something here? I'm not entirely sure why I'm getting the ED25519 error -- I only have RSA keys on this VM. 🤔 Should I temporarily add |
|
@JasonPoll thanks for all your investigation on this. I have a couple thoughts.
Even though the VM only uses RSA keys, net-ssh might be iterating through all of your SSH keys on the host machine, i.e. those in your In that case, yes, I think temporarily adding All that being said, I am mostly concerned that my changes in this PR didn't not cause a regression in the SFTP behavior that was previously working. For the purposes of merging this PR I would be satisfied if your tests work with a modern version of Ruby. So if getting Ruby 2.0 running is causing a problem, then feel free to test against whatever version is convenient. Thanks again for your help! |
There was a problem hiding this comment.
I'm good with this.
I got rbnacl and rbnacl-libsodium installed and tried again, and ran into more issues. Looking at net-ssh history, it looks like even my older RSA keys may be "too new" for net-ssh 4.2.0 to handle, so I did away with key auth entirely for these tests. Password auth should be sufficient for these tests.
With my previously describe test setup I tested the same scenarios as before (not every possible permutation that could be tested, but sufficient I believe. They all resulted as expected.
SSHKit backend configured with defaults(scp), and SCP enabled on remote host:
no errors, and file uploaded and downloaded via SCP as expected
doit("192.168.69.42")
- global transfer method: :scp
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/mutex_m-0.1.2/lib/mutex_m.rb:110: warning: toplevel constant Mutex referenced by Thread::Mutex
vagrant@192.168.69.42's password:
INFO Uploading junk.dat 10.94%
...
INFO Downloading junk_downloaded.dat 100.0%SSHKit backend configured to use SFTP, and SCP enabled on remote host:
no errors, and file uploaded and downloaded via SFTP as expected
SSHKit::Backend::Netssh.configure{|ssh| ssh.transfer_method = :sftp}
=> :sftp
doit("192.168.69.42")
- global transfer method: :sftp
vagrant@192.168.69.42's password:
INFO Uploading junk.dat 12.21%
...
INFO Downloading /home/vagrant/tmp/junk.dat 100.0%SSHKit backend configured with defaults(scp), and SCP disabled on remote host:
expected error raised
doit("192.168.69.42")
- global transfer method: :scp
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/mutex_m-0.1.2/lib/mutex_m.rb:110: warning: toplevel constant Mutex referenced by Thread::Mutex
/home/vagrant/.rbenv/versions/2.0.0-p648/lib/ruby/gems/2.0.0/gems/mutex_m-0.1.2/lib/mutex_m.rb:110: warning: toplevel constant Mutex referenced by Thread::Mutex
vagrant@192.168.69.42's password:
=> #<SSHKit::Runner::ExecuteError: Exception while executing on host 192.168.69.42: SCP did not finish successfully (255): >SSHKit backend configured with defaults(scp), SCP disabled on remote host, and SFTP specified at per-host level:
no errors, and file uploaded and downloaded via SFTP as expected
doit("192.168.69.42"){|host| host.transfer_method = :sftp}
- global transfer method: :scp
vagrant@192.168.69.42's password:
INFO Uploading junk.dat 12.21%
...
INFO Downloading /home/vagrant/tmp/junk.dat 100.0%SSHKit backend configured to use SFTP, and SCP disabled on remote host:
no errors, and file uploaded and downloaded via SFTP as expected
SSHKit::Backend::Netssh.configure{|ssh| ssh.transfer_method = :sftp}
=> :sftp
[6] pry(main)> doit("192.168.69.42")
- global transfer method: :sftp
vagrant@192.168.69.42's password:
INFO Uploading junk.dat 12.21%
...
INFO Downloading /home/vagrant/tmp/junk.dat 100.0%
|
@JasonPoll thank you again for the thorough testing! I'll merge this. |
@mattbrictson - oops, sorry, I missed this last bit of your message. Rest assured that I just re-ran my ad-hoc tests with Ruby 3.2.2 -- same setup as in #524. I did not experience any regression in SCP enabled/disabled on remote host vs SSHKit SCP-vs-SFTP (global or per-host configured,) behaviors. All functioned the same with modern tooling. My only remaining question would be when do you think we might see a new version of SSHKit cut, and see it incorporated into a fresh version of Capistrano? 😁 |
I'll probably cut a new release of sshkit next Tuesday. It should then automatically be incorporated into capistrano the next time you |
|
This has been released in sshkit 1.22.0. https://github.com/capistrano/sshkit/releases/tag/v1.22.0 |
Old Rubies such as Ruby 2.0 resolve a version of net-sftp that does not have the
File#sizeconvenience method. To support these older versions, use the lower levelFile#stat, which works for versions of net-sftp all the way back to 2.1.2.This fixes a
NoMethodErrorwhen SFTP is used via sshkit on Ruby 2.0.I was able to confirm this with this successful CI job: https://github.com/capistrano/sshkit/actions/runs/7373383846/job/20062512939
Fixes #528