rke up v1.2.3 and v1.2.2 fails when upgrade_strategy is drain: true

[ranchersvet]

RKE version:
v1.2.3 and v1.2.2

Docker version:

root@onlinestr03:~# docker info
Client:
 Debug Mode: false

Server:
 Containers: 17
  Running: 14
  Paused: 0
  Stopped: 3
 Images: 12
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-51-generic
 Operating System: Ubuntu 20.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.775GiB
 Name: onlinestr03
 ID: YZSH:IQKS:7NXM:2BSU:RT7H:CNXM:EOHQ:D7DM:NPJJ:CMTG:JI7I:H62J
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false**

Operating system and kernel: (cat /etc/os-release, `root@onlinestr03:~# uname -r)

5.4.0-51-generic
root@onlinestr03:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
DigitalCloud
cluster.yml file:

nodes:
  - address: 157.230.84.103
    user: root
    ssh_key_path: /root/.ssh/id_rsa
    role: [etcd,controlplane,worker]
  - address: 142.93.114.58
    user: root
    ssh_key_path: /root/.ssh/id_rsa
    role: [etcd,controlplane,worker]
  - address: 142.93.201.74
    user: root
    ssh_key_path: /root/.ssh/id_rsa
    role: [etcd,controlplane,worker]

upgrade_strategy:
  drain: true
    #  - address: 192.168.1.40
    #user: root
    #ssh_key_path: /root/.ssh/id_rsa
    #role: [etcd]

services:
  etcd:
    snapshot: true
    creation: 6h
    retention: 24h
    uid: 52034
    gid: 52034

# Required for external TLS termination with
# ingress-nginx v0.22+
ingress:
  provider: nginx
  options:
    use-forwarded-headers: "true"

    #kube-controller:
    #:cluster_cidr: "10.42.0.0/22"
kubernetes_version: v1.16.8-rancher1-2

Steps to Reproduce:
Try to update rke cluster.
Configure upgrade_strategy is drain: true

Results:
Upgrade fails with error:

INFO[0056] [controlplane] Processing controlplane hosts for upgrade 1 at a time
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x13bb572]

goroutine 1 [running]:
github.com/rancher/rke/services.getDrainHelper(...)
        /go/src/github.com/rancher/rke/services/node_util.go:58
github.com/rancher/rke/services.UpgradeControlPlaneNodes(0x19da5a0, 0xc00003a088, 0xc0001f0420, 0xc000a544a0, 0x3, 0x4, 0x0, 0xc000646c00, 0xc000515b30, 0xc00085b300, ...)
        /go/src/github.com/rancher/rke/services/controlplane.go:72 +0x3f2
github.com/rancher/rke/cluster.(*Cluster).UpgradeControlPlane(0xc0003c3800, 0x19da5a0, 0xc00003a088, 0xc0001f0420, 0xc000515b30, 0xc000373f50, 0xc000373f80, 0xc000373fb0, 0xc0003ae000)
        /go/src/github.com/rancher/rke/cluster/cluster.go:225 +0xb46
github.com/rancher/rke/cluster.(*Cluster).DeployControlPlane(0xc0003c3800, 0x19da5a0, 0xc00003a088, 0xc000a7e4c0, 0x1, 0x0, 0x1b, 0xb, 0x0)
        /go/src/github.com/rancher/rke/cluster/cluster.go:161 +0x770
github.com/rancher/rke/cmd.ClusterUp(0x19da5a0, 0xc00003a088, 0x0, 0x0, 0x0, 0x0, 0x0, 0x176f959, 0xb, 0x0, ...)
        /go/src/github.com/rancher/rke/cmd/up.go:195 +0xcee
github.com/rancher/rke/cmd.clusterUpFromCli(0xc00035c9a0, 0x197e800, 0x0)
        /go/src/github.com/rancher/rke/cmd/up.go:302 +0x652
github.com/urfave/cli.HandleAction(0x15549e0, 0x1826af8, 0xc00035c9a0, 0xc000375f00, 0x0)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0x176913c, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1779baa, 0x14, 0x0, ...)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/command.go:210 +0x9e8
github.com/urfave/cli.(*App).Run(0xc0004689c0, 0xc00000c080, 0x2, 0x2, 0x0, 0x0)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:255 +0x741
main.mainErr(0xc0001e2f50, 0x19a0e20)
        /go/src/github.com/rancher/rke/main.go:80 +0x14b9
main.main()
        /go/src/github.com/rancher/rke/main.go:25 +0x4e

gz#13573

[superseb]

@ranchersvet Does it still panic when you add the following under upgrade_strategy?

  node_drain_input:
    force: false

[ranchersvet]

Test with:
node_drain_input:
force: false
Finished successfully with no errors

[superseb]

Thanks for testing, we can put a fix in for this.