Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VNet: Make sure daemon client has adequate permissions for passed TELEPORT_HOME #44751

Merged
merged 10 commits into from
Jul 30, 2024
5 changes: 2 additions & 3 deletions lib/vnet/customdnszonevalidator.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ package vnet
import (
"context"
"errors"
"log/slog"
"net"
"slices"
"sync"
Expand Down Expand Up @@ -67,7 +66,7 @@ func (c *customDNSZoneValidator) validate(ctx context.Context, clusterName, cust
}

requiredTXTRecord := clusterTXTRecordPrefix + clusterName
slog.InfoContext(ctx, "Checking validity of custom DNS zone by querying for required TXT record.", "zone", customDNSZone, "record", requiredTXTRecord)
log.InfoContext(ctx, "Checking validity of custom DNS zone by querying for required TXT record.", "zone", customDNSZone, "record", requiredTXTRecord)

records, err := c.lookupTXT(ctx, customDNSZone)
if err != nil {
Expand All @@ -83,7 +82,7 @@ func (c *customDNSZoneValidator) validate(ctx context.Context, clusterName, cust
return trace.Wrap(errNoTXTRecord(customDNSZone, requiredTXTRecord))
}

slog.InfoContext(ctx, "Custom DNS zone has valid TXT record.", "zone", customDNSZone, "cluster", clusterName)
log.InfoContext(ctx, "Custom DNS zone has valid TXT record.", "zone", customDNSZone, "cluster", clusterName)

c.mu.Lock()
defer c.mu.Unlock()
Expand Down
10 changes: 9 additions & 1 deletion lib/vnet/daemon/client_darwin.go
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,15 @@ func startByCalling(ctx context.Context, bundlePath string, config Config) error
}

if errorDomain == nsCocoaErrorDomain && errorCode == errorCodeNSXPCConnectionCodeSigningRequirementFailure {
errC <- trace.Wrap(errXPCConnectionCodeSigningRequirementFailure, "the daemon does not appear to be code signed correctly")
// If the client submits TELEPORT_HOME to which the user doesn't have access, the daemon is
// going to shut down with an error soon after starting. Because of that, macOS won't have
// enough time to perform the verification of the code signing requirement of the daemon, as
// requested by the client.
//
// In that scenario, macOS is going to simply error that connection with
// NSXPCConnectionCodeSigningRequirementFailure. Without looking at logs, it's not possible
// to differentiate that from a "legitimate" failure caused by an incorrect requirement.
errC <- trace.Wrap(errXPCConnectionCodeSigningRequirementFailure, "either daemon is not signed correctly or it shut down before signature could be verified")
return
}

Expand Down
12 changes: 12 additions & 0 deletions lib/vnet/daemon/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,16 @@ type Config struct {
DNSAddr string
// HomePath points to TELEPORT_HOME that will be used by the admin process.
HomePath string
// ClientCred are the credentials of the unprivileged process that wants to start VNet.
ClientCred *ClientCred
}

// ClientCred are the credentials of the unprivileged process that wants to start VNet.
type ClientCred struct {
// Egid is the effective group ID of the unprivileged process.
Egid int
// Euid is the effective user ID of the unprivileged process.
Euid int
}

func (c *Config) CheckAndSetDefaults() error {
Expand All @@ -46,6 +56,8 @@ func (c *Config) CheckAndSetDefaults() error {
return trace.BadParameter("missing DNS address")
case c.HomePath == "":
return trace.BadParameter("missing home path")
case c.ClientCred == nil:
return trace.BadParameter("missing client credentials")
espadolini marked this conversation as resolved.
Show resolved Hide resolved
}
return nil
}
Expand Down
6 changes: 6 additions & 0 deletions lib/vnet/daemon/service_darwin.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import "C"
import (
"context"
"errors"
"fmt"
"time"
"unsafe"

Expand Down Expand Up @@ -77,6 +78,7 @@ func Start(ctx context.Context, workFn func(context.Context, Config) error) erro
"ipv6_prefix", config.IPv6Prefix,
"dns_addr", config.DNSAddr,
"home_path", config.HomePath,
"cred", fmt.Sprintf("%#v", config.ClientCred),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I had to look up the %#v format string, I think here %+v is probably slightly better because its prints field names without the type names

Suggested change
"cred", fmt.Sprintf("%#v", config.ClientCred),
"cred", fmt.Sprintf("%+v", config.ClientCred),

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be cleaner to implement String on the ClientCred type (and then remember to wrap it in logutils.StringerAttr) or maybe even just implement LogValue.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LogValue is pretty cool, TIL. It ends up being logged like this. Admittedly, slog.GroupValue seems to look better with JSON formatters but I'll take this.

2024-07-30T16:58:15+02:00 INFO [VNET:DAEM] Received VNet config socket_path:/var/folders/1x/xhqmbxsd3_v9zdv78ybr14zr0000gn/T/vnetffa74891-73de-43c8-9df1-23d80ca0873c.sock ipv6_prefix:fde3:5ebb:fece:: dns_addr:fde3:5ebb:fece::2 home_path:/Users/fav/.tsh creds_valid:true egid:20 euid:501 trace_id:8a25ef3f4bcc9bf975b59e63996dbe41 span_id:ff3cdafcd7d941c7 daemon/service_darwin.go:75

)

return trace.Wrap(workFn(ctx, config))
Expand Down Expand Up @@ -113,6 +115,10 @@ func waitForVnetConfig(ctx context.Context) (Config, error) {
IPv6Prefix: C.GoString(result.ipv6_prefix),
DNSAddr: C.GoString(result.dns_addr),
HomePath: C.GoString(result.home_path),
ClientCred: &ClientCred{
Egid: int(result.egid),
Euid: int(result.euid),
},
}
errC <- nil
}()
Expand Down
4 changes: 4 additions & 0 deletions lib/vnet/daemon/service_darwin.h
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ typedef struct VnetConfigResult {
const char *ipv6_prefix;
const char *dns_addr;
const char *home_path;
// egid is the effective group ID of the process on the other side of the XPC connection.
gid_t egid;
// euid is the effective user ID of the process on the other side of the XPC connection.
uid_t euid;
espadolini marked this conversation as resolved.
Show resolved Hide resolved
} VnetConfigResult;

// WaitForVnetConfig blocks until a client calls the daemon with a config necessary to start VNet.
Expand Down
8 changes: 8 additions & 0 deletions lib/vnet/daemon/service_darwin.m
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ @interface VNEDaemonService () <NSXPCListenerDelegate, VNEDaemonProtocol>
@property(nonatomic, readwrite) NSString *ipv6Prefix;
@property(nonatomic, readwrite) NSString *dnsAddr;
@property(nonatomic, readwrite) NSString *homePath;
@property(nonatomic, readwrite) gid_t egid;
@property(nonatomic, readwrite) uid_t euid;
@property(nonatomic, readwrite) dispatch_semaphore_t gotVnetConfigSema;

@end
Expand Down Expand Up @@ -106,6 +108,10 @@ - (void)startVnet:(VnetConfig *)vnetConfig completion:(void (^)(NSError *error))
_dnsAddr = @(vnetConfig->dns_addr);
_homePath = @(vnetConfig->home_path);

NSXPCConnection *currentConn = [NSXPCConnection currentConnection];
_egid = [currentConn effectiveGroupIdentifier];
_euid = [currentConn effectiveUserIdentifier];
ravicious marked this conversation as resolved.
Show resolved Hide resolved

dispatch_semaphore_signal(_gotVnetConfigSema);
completion(nil);
}
Expand Down Expand Up @@ -180,6 +186,8 @@ void WaitForVnetConfig(VnetConfigResult *outResult) {
outResult->ipv6_prefix = VNECopyNSString([daemonService ipv6Prefix]);
outResult->dns_addr = VNECopyNSString([daemonService dnsAddr]);
outResult->home_path = VNECopyNSString([daemonService homePath]);
outResult->egid = [daemonService egid];
outResult->euid = [daemonService euid];
outResult->ok = true;
}
}
71 changes: 44 additions & 27 deletions lib/vnet/osconfig.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ package vnet

import (
"context"
"log/slog"
"net"

"github.com/gravitational/trace"
Expand All @@ -28,6 +27,7 @@ import (
"github.com/gravitational/teleport/api/utils"
"github.com/gravitational/teleport/lib/client"
"github.com/gravitational/teleport/lib/client/clientcache"
"github.com/gravitational/teleport/lib/vnet/daemon"
)

type osConfig struct {
Expand All @@ -43,14 +43,16 @@ type osConfigurator struct {
clientStore *client.Store
clientCache *clientcache.Cache
clusterConfigCache *ClusterConfigCache
tunName string
tunIPv6 string
dnsAddr string
homePath string
tunIPv4 string
// daemonClientCred are the credentials of the process that contacted the daemon.
daemonClientCred daemon.ClientCred
tunName string
tunIPv6 string
dnsAddr string
homePath string
tunIPv4 string
}

func newOSConfigurator(tunName, ipv6Prefix, dnsAddr, homePath string) (*osConfigurator, error) {
func newOSConfigurator(tunName, ipv6Prefix, dnsAddr, homePath string, daemonClientCred daemon.ClientCred) (*osConfigurator, error) {
if homePath == "" {
// This runs as root so we need to be configured with the user's home path.
return nil, trace.BadParameter("homePath must be passed from unprivileged process")
Expand All @@ -61,11 +63,12 @@ func newOSConfigurator(tunName, ipv6Prefix, dnsAddr, homePath string) (*osConfig
tunIPv6 := ipv6Prefix + "1"

configurator := &osConfigurator{
tunName: tunName,
tunIPv6: tunIPv6,
dnsAddr: dnsAddr,
homePath: homePath,
clientStore: client.NewFSClientStore(homePath),
tunName: tunName,
tunIPv6: tunIPv6,
dnsAddr: dnsAddr,
homePath: homePath,
clientStore: client.NewFSClientStore(homePath),
daemonClientCred: daemonClientCred,
}
configurator.clusterConfigCache = NewClusterConfigCache(clockwork.NewRealClock())

Expand All @@ -89,18 +92,32 @@ func (c *osConfigurator) close() error {
return trace.Wrap(c.clientCache.Clear())
}

// updateOSConfiguration reads tsh profiles out of [c.homePath]. For each profile, it reads the VNet
// config of the root cluster and of each leaf cluster. Then it proceeds to update the OS based on
// information from that config.
//
// For the duration of reading data from clusters, it drops the root privileges, only to regain them
// before configuring the OS.
func (c *osConfigurator) updateOSConfiguration(ctx context.Context) error {
var dnsZones []string
var cidrRanges []string

profileNames, err := profile.ListProfileNames(c.homePath)
if err != nil {
return trace.Wrap(err, "listing user profiles")
}
for _, profileName := range profileNames {
profileDNSZones, profileCIDRRanges := c.getDNSZonesAndCIDRRangesForProfile(ctx, profileName)
dnsZones = append(dnsZones, profileDNSZones...)
cidrRanges = append(cidrRanges, profileCIDRRanges...)
// Drop privileges to ensure that the user who spawned the daemon client has privileges necessary
// to access c.homePath that it sent when starting the daemon.
// Otherwise a client could make the daemon read a profile out of any directory.
if err := c.doWithDroppedRootPrivileges(ctx, func() error {
profileNames, err := profile.ListProfileNames(c.homePath)
if err != nil {
return trace.Wrap(err, "listing user profiles")
}
for _, profileName := range profileNames {
profileDNSZones, profileCIDRRanges := c.getDNSZonesAndCIDRRangesForProfile(ctx, profileName)
dnsZones = append(dnsZones, profileDNSZones...)
cidrRanges = append(cidrRanges, profileCIDRRanges...)
}
return nil
}); err != nil {
return trace.Wrap(err)
}

dnsZones = utils.Deduplicate(dnsZones)
Expand All @@ -114,7 +131,7 @@ func (c *osConfigurator) updateOSConfiguration(ctx context.Context) error {
}
}

err = configureOS(ctx, &osConfig{
err := configureOS(ctx, &osConfig{
tunName: c.tunName,
tunIPv6: c.tunIPv6,
tunIPv4: c.tunIPv4,
Expand All @@ -137,22 +154,22 @@ func (c *osConfigurator) getDNSZonesAndCIDRRangesForProfile(ctx context.Context,
defer func() {
if shouldClearCacheForRoot {
if err := c.clientCache.ClearForRoot(profileName); err != nil {
slog.ErrorContext(ctx, "Error while clearing client cache", "profile", profileName, "error", err)
log.ErrorContext(ctx, "Error while clearing client cache", "profile", profileName, "error", err)
}
}
}()

rootClient, err := c.clientCache.Get(ctx, profileName, "" /*leafClusterName*/)
if err != nil {
slog.WarnContext(ctx,
log.WarnContext(ctx,
"Failed to get root cluster client from cache, profile may be expired, not configuring VNet for this cluster",
"profile", profileName, "error", err)

return
}
clusterConfig, err := c.clusterConfigCache.GetClusterConfig(ctx, rootClient)
if err != nil {
slog.WarnContext(ctx,
log.WarnContext(ctx,
"Failed to load VNet configuration, profile may be expired, not configuring VNet for this cluster",
"profile", profileName, "error", err)

Expand All @@ -164,7 +181,7 @@ func (c *osConfigurator) getDNSZonesAndCIDRRangesForProfile(ctx context.Context,

leafClusters, err := getLeafClusters(ctx, rootClient)
if err != nil {
slog.WarnContext(ctx,
log.WarnContext(ctx,
"Failed to list leaf clusters, profile may be expired, not configuring VNet for leaf clusters of this cluster",
"profile", profileName, "error", err)

Expand All @@ -179,15 +196,15 @@ func (c *osConfigurator) getDNSZonesAndCIDRRangesForProfile(ctx context.Context,
for _, leafClusterName := range leafClusters {
clusterClient, err := c.clientCache.Get(ctx, profileName, leafClusterName)
if err != nil {
slog.WarnContext(ctx,
log.WarnContext(ctx,
"Failed to create leaf cluster client, not configuring VNet for this cluster",
"profile", profileName, "leaf_cluster", leafClusterName, "error", err)
continue
}

clusterConfig, err := c.clusterConfigCache.GetClusterConfig(ctx, clusterClient)
if err != nil {
slog.WarnContext(ctx,
log.WarnContext(ctx,
"Failed to load VNet configuration, not configuring VNet for this cluster",
"profile", profileName, "leaf_cluster", leafClusterName, "error", err)
continue
Expand Down
44 changes: 38 additions & 6 deletions lib/vnet/osconfig_darwin.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ package vnet
import (
"bufio"
"context"
"log/slog"
"os"
"os/exec"
"path/filepath"
"syscall"

"github.com/gravitational/trace"
)
Expand All @@ -34,28 +34,28 @@ func configureOS(ctx context.Context, cfg *osConfig) error {
// process exits and the TUN is deleted.

if cfg.tunIPv4 != "" {
slog.InfoContext(ctx, "Setting IPv4 address for the TUN device.", "device", cfg.tunName, "address", cfg.tunIPv4)
log.InfoContext(ctx, "Setting IPv4 address for the TUN device.", "device", cfg.tunName, "address", cfg.tunIPv4)
cmd := exec.CommandContext(ctx, "ifconfig", cfg.tunName, cfg.tunIPv4, cfg.tunIPv4, "up")
if err := cmd.Run(); err != nil {
return trace.Wrap(err, "running %v", cmd.Args)
}
}
for _, cidrRange := range cfg.cidrRanges {
slog.InfoContext(ctx, "Setting an IP route for the VNet.", "netmask", cidrRange)
log.InfoContext(ctx, "Setting an IP route for the VNet.", "netmask", cidrRange)
cmd := exec.CommandContext(ctx, "route", "add", "-net", cidrRange, "-interface", cfg.tunName)
if err := cmd.Run(); err != nil {
return trace.Wrap(err, "running %v", cmd.Args)
}
}

if cfg.tunIPv6 != "" {
slog.InfoContext(ctx, "Setting IPv6 address for the TUN device.", "device", cfg.tunName, "address", cfg.tunIPv6)
log.InfoContext(ctx, "Setting IPv6 address for the TUN device.", "device", cfg.tunName, "address", cfg.tunIPv6)
cmd := exec.CommandContext(ctx, "ifconfig", cfg.tunName, "inet6", cfg.tunIPv6, "prefixlen", "64")
if err := cmd.Run(); err != nil {
return trace.Wrap(err, "running %v", cmd.Args)
}

slog.InfoContext(ctx, "Setting an IPv6 route for the VNet.")
log.InfoContext(ctx, "Setting an IPv6 route for the VNet.")
cmd = exec.CommandContext(ctx, "route", "add", "-inet6", cfg.tunIPv6, "-prefixlen", "64", "-interface", cfg.tunName)
if err := cmd.Run(); err != nil {
return trace.Wrap(err, "running %v", cmd.Args)
Expand All @@ -78,7 +78,7 @@ func configureDNS(ctx context.Context, nameserver string, zones []string) error
return trace.BadParameter("empty nameserver with non-empty zones")
}

slog.DebugContext(ctx, "Configuring DNS.", "nameserver", nameserver, "zones", zones)
log.DebugContext(ctx, "Configuring DNS.", "nameserver", nameserver, "zones", zones)
if err := os.MkdirAll(resolverPath, os.FileMode(0755)); err != nil {
return trace.Wrap(err, "creating %s", resolverPath)
}
Expand Down Expand Up @@ -140,3 +140,35 @@ func vnetManagedResolverFiles() (map[string]struct{}, error) {
}
return matchingFiles, nil
}

// doWithDroppedRootPrivileges drops the privileges of the current process to those of the client
// process that called the VNet daemon.
func (c *osConfigurator) doWithDroppedRootPrivileges(ctx context.Context, fn func() error) (err error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing prevents this from being called in parallel. Should we have a package-level mutex (or maybe just a package level atomic bool) that prevents more than one goroutine from running while we do these privilege shenanigans?

Copy link
Member Author

@ravicious ravicious Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add an atomic bool and make sure that this method returns an error if compare and swap fails. If someone runs into that error during development, they'll have to consciously think about what the behavior of this function should be, instead of simply waiting for a mutex (which could be a valid option, but I don't think we should make this decision now).

rootEgid := os.Getegid()
rootEuid := os.Geteuid()

if err := syscall.Setegid(c.daemonClientCred.Egid); err != nil {
return trace.Wrap(err, "setting egid")
}
defer func() {
syscallErr := trace.Wrap(syscall.Setegid(rootEgid), "reverting egid")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to do anything but panic if the OS prevents us from restoring the egid/euid that we had before? The whole process is basically unusable at that point right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, true. I've never had a legitimate reason to use panic in any program that I've written, so I was hesitant to use it. But an error is technically recoverable, while here we just shouldn't continue execution at all.

err = trace.NewAggregate(err, syscallErr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trace.aggregate composes really poorly with itself, I'd find some other way to keep track of the multierror before returning if we really care about that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved by removing trace.NewAggregate altogether in favor of using panic.

}()

if err := syscall.Seteuid(c.daemonClientCred.Euid); err != nil {
return trace.Wrap(err, "setting euid")
}
defer func() {
syscallErr := trace.Wrap(syscall.Seteuid(rootEuid), "reverting euid")
err = trace.NewAggregate(err, syscallErr)
}()

log.InfoContext(ctx, "Temporarily dropping root privileges.", "egid", c.daemonClientCred.Egid, "euid", c.daemonClientCred.Euid)
defer func() {
if err == nil {
log.InfoContext(ctx, "Restored root privileges.", "egid", rootEgid, "euid", rootEuid)
}
}()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend moving this to the top of the function, before the Setegid/Seteuid calls/defers. Then we'll actually print "Temporarily dropping root privileges." before the attempt, and "Restored root privileges." after successfully doing that


return trace.Wrap(fn())
}
Loading
Loading