Pull containers from private registry

[malomarrec]

Problem

Running server-side:

• I want to pull some images from a private docker registry
• I want to pull some other images from public dockerhub or another public registry

Current state: yes but not great

Currently, our managed executors don’t support feeding custom docker credentials so those would need to be public registries/images. One thing that’s possible is either modifying the executors deployment manually to include credentials or to give the docker registry mirror credentials so the executors don’t need them (maybe a bit more secure). Anyways, we don’t have a good way today to support this. Kubernetes has a magic secret for this, maybe we want the same, once secrets are implemented.

Impacted customers

Most customers will actually need this, and in particular Strategic customers.
Example:

• https://github.com/sourcegraph/accounts/issues/3
• https://github.com/sourcegraph/accounts/issues/4001

We have some ECR images that we’d like to run through the server-side executors.

[malomarrec]

Depends on: https://github.com/sourcegraph/sourcegraph/issues/27926.
Let's think about what the flow will look like for an end user.

[chrispine]

P1?

[eseliger]

this has a dependency on another ticket that is currently in backlog, so I think P1 cannot work.

[Piszmog]

As a note, when we work this, we should take a look into custom containers that set the user (see customer#1305)

[eseliger]

I think that's an orthogonal issue and also happens on public images, so I will file it as a separate ticket.

[eseliger]

diff --git a/enterprise/cmd/executor/internal/worker/handler.go b/enterprise/cmd/executor/internal/worker/handler.go
index 7f142cd136..ed94c52efc 100644
--- a/enterprise/cmd/executor/internal/worker/handler.go
+++ b/enterprise/cmd/executor/internal/worker/handler.go
@@ -2,6 +2,7 @@ package worker
 
 import (
 	"context"
+	"encoding/base64"
 	"fmt"
 	"time"
 
@@ -9,6 +10,11 @@ import (
 
 	"github.com/sourcegraph/log"
 
+	dockertypes "github.com/docker/docker/api/types"
+	"github.com/docker/docker/client"
+	dockerparser "github.com/novln/docker-parser"
+	dockerparsertypes "github.com/novln/docker-parser/docker"
+
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/executor/internal/command"
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/executor/internal/ignite"
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/executor/internal/janitor"
@@ -127,6 +133,39 @@ func (h *handler) Handle(ctx context.Context, logger log.Logger, record workerut
 		FirecrackerOptions: h.options.FirecrackerOptions,
 		ResourceOptions:    h.options.ResourceOptions,
 	}
+
+	cli, err := client.NewClientWithOpts(client.FromEnv)
+	if err != nil {
+		return err
+	}
+
+	parsedImage, err := dockerparser.Parse("ubuntu:20.04")
+	if err != nil {
+		return err
+	}
+
+	// TODO: Do this prior to each docker step.
+	// TODO: Add a temporary additional field in executors Job payload
+	// `AdditionalDockerImages` are are pulled pre-start. Those are required for
+	// SSBC for now.
+	// TODO: See https://medium.com/@dperny/forwarding-the-docker-socket-over-ssh-e6567cfab160 for how to pull images from the host using the in-firecracker docker.
+	// For the non-firecracker mode, we can simply use the on-host docker, and also just use the fallback method of ImagePull to prioritize locally configured credentials.
+	if parsedImage.Registry() != dockerparsertypes.DefaultHostname {
+		_, err = cli.ImagePull(context.Background(), "ubuntu:20.04", dockertypes.ImagePullOptions{
+			RegistryAuth: base64.StdEncoding.EncodeToString([]byte("asd")),
+		})
+	}
+	if err != nil {
+		return err
+	}
+
 	runner := h.runnerFactory(workspace.Path(), commandLogger, options, h.operations)
 
 	logger.Info("Setting up VM")