Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Vagrant with Docker Compose for running functional tests #539

Merged
merged 6 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM ubuntu:22.04
WORKDIR /provision
COPY ./ubuntu_setup.sh ./
RUN ./ubuntu_setup.sh
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
22 changes: 22 additions & 0 deletions .docker/ubuntu_setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/bash

set -e

export DEBIAN_FRONTEND=noninteractive
apt -y update

# Create `deployer` user that can sudo without a password
apt-get -y install sudo
adduser --disabled-password deployer < /dev/null
echo "deployer:topsecret" | chpasswd
echo "deployer ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

# Install and configure sshd
apt-get -y install openssh-server
{
echo "Port 22"
echo "PasswordAuthentication yes"
echo "ChallengeResponseAuthentication no"
} >> /etc/ssh/sshd_config
mkdir /var/run/sshd
chmod 0755 /var/run/sshd
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,30 @@ jobs:
bundler-cache: true
- name: Run rubocop
run: bundle exec rake lint

functional:
runs-on: ubuntu-latest
strategy:
matrix:
ruby: ["2.0", "ruby"]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗒️ "ruby" in this context means the latest stable version of Ruby. As of this writing, it is 3.3.3. Using this keyword means the functional tests will always run against the latest version of Ruby, without us having to manually update the workflow.

steps:
- uses: actions/checkout@v4
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby }}
bundler-cache: true
- name: Run functional tests
run: bundle exec rake test:functional

functional-all:
runs-on: ubuntu-latest
needs: [functional]
if: always()
steps:
- name: All tests ok
if: ${{ !(contains(needs.*.result, 'failure')) }}
run: exit 0
- name: Some tests failed
if: ${{ contains(needs.*.result, 'failure') }}
run: exit 1
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@
bin/rake
.bundle
.yardoc
.vagrant*
test/tmp
Gemfile.lock
7 changes: 0 additions & 7 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,6 @@ Layout/IndentHash:
Exclude:
- 'test/functional/backends/test_local.rb'
- 'test/functional/backends/test_netssh.rb'
- 'test/support/vagrant_wrapper.rb'
- 'test/unit/formatters/test_custom.rb'
- 'test/unit/formatters/test_pretty.rb'
- 'test/unit/test_mapping_interaction_handler.rb'
Expand Down Expand Up @@ -445,12 +444,6 @@ Style/MethodName:
Exclude:
- 'test/unit/test_color.rb'

# Offense count: 1
# Cop supports --auto-correct.
Style/MutableConstant:
Exclude:
- 'Vagrantfile'

# Offense count: 1
# Cop supports --auto-correct.
# Configuration parameters: Strict.
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ using unsupported features.

## Tests

SSHKit has a unit test suite and a functional test suite. Some functional tests run against
[Vagrant](https://www.vagrantup.com/) VMs. If possible, you should make sure that the
SSHKit has a unit test suite and a functional test suite. Some functional tests run using
[Docker](https://docs.docker.com/get-docker/). If possible, you should make sure that the
tests pass for each commit by running `rake` in the sshkit directory. This is in case we
need to cherry pick commits or rebase. You should ensure the tests pass, (preferably on
the minimum and maximum ruby version), before creating a PR.
Expand Down
2 changes: 1 addition & 1 deletion RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
## How to release

1. Run `bundle install` to make sure that you have all the gems necessary for testing and releasing.
2. **Ensure the tests are passing by running `rake test`.** If functional tests fail, ensure you have [Vagrant](https://www.vagrantup.com) installed and have started it with `vagrant up`.
2. **Ensure the tests are passing by running `rake test`.** If functional tests fail, ensure you have [Docker installed](https://docs.docker.com/get-docker/) and running.
3. Determine which would be the correct next version number according to [semver](http://semver.org/).
4. Update the version in `./lib/sshkit/version.rb`.
5. Commit the `version.rb` change with a message like "Preparing vX.Y.Z"
Expand Down
4 changes: 0 additions & 4 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,6 @@ namespace :test do

end

Rake::Task["test:functional"].enhance do
warn "Remember there are still some VMs running, kill them with `vagrant halt` if you are finished using them."
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗒️ DockerWrapper will automatically start Docker Compose on demand and clean it up after tests are done, so this instruction is no longer relevant.

end

desc 'Run RuboCop lint checks'
RuboCop::RakeTask.new(:lint) do |task|
task.options = ['--lint']
Expand Down
24 changes: 0 additions & 24 deletions Vagrantfile

This file was deleted.

8 changes: 8 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
name: sshkit

services:
ssh_server:
build:
context: .docker
ports:
- "2122:22"
17 changes: 0 additions & 17 deletions test/boxes.json

This file was deleted.

2 changes: 1 addition & 1 deletion test/functional/backends/netssh_transfer_tests.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ def setup
end

def a_host
VagrantWrapper.hosts['one']
DockerWrapper.host
end

def test_upload_and_then_capture_file_contents
Expand Down
2 changes: 1 addition & 1 deletion test/functional/backends/test_netssh.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def setup
end

def a_host
VagrantWrapper.hosts['one']
DockerWrapper.host
end

def test_simple_netssh
Expand Down
24 changes: 0 additions & 24 deletions test/functional/test_ssh_server_comes_up_for_functional_tests.rb

This file was deleted.

46 changes: 4 additions & 42 deletions test/helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,51 +28,13 @@ def flush_connections
end

class FunctionalTest < Minitest::Test

def setup
unless VagrantWrapper.running?
warn "Vagrant VMs are not running. Please, start it manually with `vagrant up`"
end
end

private

def create_user_with_key(username, password = :secret)
username, password = username.to_s, password.to_s

keys = VagrantWrapper.hosts.collect do |_name, host|
Net::SSH.start(host.hostname, host.user, port: host.port, password: host.password) do |ssh|

# Remove the user, make it again, force-generate a key for him
# short keys save us a few microseconds
ssh.exec!("sudo userdel -rf #{username}; true") # The `rescue nil` of the shell world
ssh.exec!("sudo useradd -m #{username}")
ssh.exec!("sudo echo y | ssh-keygen -b 1024 -f #{username} -N ''")
ssh.exec!("sudo chown vagrant:vagrant #{username}*")
ssh.exec!("sudo echo #{username}:#{password} | chpasswd")
require_relative "support/docker_wrapper"
return if DockerWrapper.running?

# Make the .ssh directory, change the ownership and the
ssh.exec!("sudo mkdir -p /home/#{username}/.ssh")
ssh.exec!("sudo chown #{username}:#{username} /home/#{username}/.ssh")
ssh.exec!("sudo chmod 700 /home/#{username}/.ssh")

# Move the key to authorized keys and chown and chmod it
ssh.exec!("sudo cat #{username}.pub > /home/#{username}/.ssh/authorized_keys")
ssh.exec!("sudo chown #{username}:#{username} /home/#{username}/.ssh/authorized_keys")
ssh.exec!("sudo chmod 600 /home/#{username}/.ssh/authorized_keys")

key = ssh.exec!("cat /home/vagrant/#{username}")

# Clean Up Files
ssh.exec!("sudo rm #{username} #{username}.pub")

key
end
end

Hash[VagrantWrapper.hosts.collect { |n, _h| n.to_sym }.zip(keys)]
DockerWrapper.start
DockerWrapper.wait_for_ssh_server
end

end

#
Expand Down
71 changes: 71 additions & 0 deletions test/support/docker_wrapper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
require "socket"

Minitest.after_run do
DockerWrapper.stop if DockerWrapper.running?
end
Comment on lines +3 to +5
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗒️ This ensures that the container is cleaned up at exit.


module DockerWrapper
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗒️ This code was mostly lifted from capistrano/capistrano#2159.

SSH_SERVER_PORT = 2122

class << self
def host
SSHKit::Host.new(
user: "deployer",
hostname: "localhost",
port: SSH_SERVER_PORT,
password: "topsecret",
ssh_options: host_verify_options
)
end

def running?
out, status = run_compose_command("ps --status running", false)
status.success? && out.include?("ssh_server")
end

def start
run_compose_command("up -d")
end

def stop
run_compose_command("down")
end

def wait_for_ssh_server(retries=3)
Socket.tcp("localhost", SSH_SERVER_PORT, connect_timeout: 1).close
sleep(1)
rescue Errno::ECONNREFUSED, Errno::ETIMEDOUT
retries -= 1
sleep(2) && retry if retries.positive?
raise
end

private

def run_compose_command(command, echo=true)
$stderr.puts "[docker compose] #{command}" if echo
Open3.popen2e("docker compose #{command}") do |stdin, outerr, wait_thread|
stdin.close
output = Thread.new { capture_stream(outerr, echo) }
[output.value, wait_thread.value]
end
end

def capture_stream(stream, echo=true)
buffer = String.new
while (line = stream.gets)
buffer << line
$stderr.puts("[docker compose] #{line}") if echo
end
buffer
end

def host_verify_options
if Net::SSH::Version::MAJOR >= 5
{ verify_host_key: :never }
else
{ paranoid: false }
end
end
end
end
Loading