Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ftr] move SAML auth to kbn-test #172678

Merged
merged 21 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
bf8c7f9
[ftr] move SAML auth to kbn-test
dmlemeshko Dec 6, 2023
3ecba70
fetch kibana version only for cloud saml session
dmlemeshko Dec 6, 2023
cdbf959
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine Dec 6, 2023
30a9fa1
revert local prettier change
dmlemeshko Dec 6, 2023
b931c6c
Merge remote-tracking branch 'upstream/main' into ftr/move-saml-auth-…
dmlemeshko Dec 6, 2023
09f7893
Merge branch 'ftr/move-saml-auth-to-kbn-test' of github.com:dmlemeshk…
dmlemeshko Dec 6, 2023
d217a4c
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Dec 6, 2023
82f750b
change interface to not rely on FTR config
dmlemeshko Dec 7, 2023
eaf24d6
Merge branch 'ftr/move-saml-auth-to-kbn-test' of github.com:dmlemeshk…
dmlemeshko Dec 7, 2023
740ab77
Merge branch 'main' of github.com:elastic/kibana into ftr/move-saml-a…
dmlemeshko Dec 7, 2023
3f46466
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Dec 7, 2023
fac981e
clean code
dmlemeshko Dec 7, 2023
7ef742e
add unit tests
dmlemeshko Dec 7, 2023
692b618
Merge branch 'ftr/move-saml-auth-to-kbn-test' of github.com:dmlemeshk…
dmlemeshko Dec 7, 2023
27afd47
Merge remote-tracking branch 'upstream/main' into ftr/move-saml-auth-…
dmlemeshko Dec 7, 2023
63537f9
Update packages/kbn-test/src/auth/session_manager.ts
dmlemeshko Dec 7, 2023
568d02a
update tests
dmlemeshko Dec 7, 2023
2ece8e7
Merge branch 'ftr/move-saml-auth-to-kbn-test' of github.com:dmlemeshk…
dmlemeshko Dec 7, 2023
93664d5
update api integration test example
dmlemeshko Dec 7, 2023
405dde8
Merge remote-tracking branch 'upstream/main' into ftr/move-saml-auth-…
dmlemeshko Dec 7, 2023
c4ead20
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Dec 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/kbn-test/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export { startServersCli, startServers } from './src/functional_tests/start_serv

// @internal
export { runTestsCli, runTests } from './src/functional_tests/run_tests';

export { SAMLSessionManager } from './src/auth';
export { runElasticsearch, runKibanaServer } from './src/functional_tests/lib';
export { getKibanaCliArg, getKibanaCliLoggers } from './src/functional_tests/lib/kibana_cli_args';

Expand Down
37 changes: 37 additions & 0 deletions packages/kbn-test/src/auth/helper.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { REPO_ROOT } from '@kbn/repo-info';
import * as fs from 'fs';
import { resolve } from 'path';
import { User } from './session_manager';

export const getProjectType = (serverArgs: string[]) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: I see this function isn't used anywhere yet, is it just a leftover or is it reserved for some future use?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remove it in fac981e. Originally I planned to split svl roles yml into 3 files and validate roles per project, but decided to move it to the follow-up PR.

const svlArg = serverArgs.filter((arg) => arg.startsWith('--serverless'));
if (svlArg.length === 0) {
throw new Error('--serverless argument is missing in kbnTestServer.serverArgs');
}
return svlArg[0].split('=')[1];
};

/**
* Loads cloud users from '.ftr/role_users.json'
* QAF prepares the file for CI pipelines, make sure to add it manually for local run
*/
export const readCloudUsersFromFile = (): Array<[string, User]> => {
const cloudRoleUsersFilePath = resolve(REPO_ROOT, '.ftr', 'role_users.json');
if (!fs.existsSync(cloudRoleUsersFilePath)) {
throw new Error(`Please define user roles with email/password in ${cloudRoleUsersFilePath}`);
}
const data = fs.readFileSync(cloudRoleUsersFilePath, 'utf8');
if (data.length === 0) {
throw new Error(`'${cloudRoleUsersFilePath}' is empty: no roles are defined`);
}

return Object.entries(JSON.parse(data)) as Array<[string, User]>;
};
9 changes: 9 additions & 0 deletions packages/kbn-test/src/auth/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

export { SAMLSessionManager } from './session_manager';
Original file line number Diff line number Diff line change
@@ -1,40 +1,32 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { createSAMLResponse as createMockedSAMLResponse } from '@kbn/mock-idp-plugin/common';
import { ToolingLog } from '@kbn/tooling-log';
import axios, { AxiosResponse } from 'axios';
import * as cheerio from 'cheerio';
import { parse as parseCookie } from 'tough-cookie';
import { Cookie, parse as parseCookie } from 'tough-cookie';
import Url from 'url';
import { Session } from './svl_user_manager';

export interface CloudSamlSessionParams {
email: string;
password: string;
kbnHost: string;
kbnVersion: string;
log: ToolingLog;
}

export interface LocalSamlSessionParams {
username: string;
email: string;
fullname: string;
role: string;
kbnHost: string;
log: ToolingLog;
}
import { CloudSamlSessionParams, CreateSamlSessionParams, LocalSamlSessionParams } from './types';

export class Session {
readonly cookie;
readonly email;
readonly fullname;
constructor(cookie: Cookie, email: string, fullname: string) {
this.cookie = cookie;
this.email = email;
this.fullname = fullname;
}

export interface CreateSamlSessionParams {
hostname: string;
email: string;
password: string;
log: ToolingLog;
getCookieValue() {
return this.cookie.value;
}
}

const cleanException = (url: string, ex: any) => {
Expand Down
135 changes: 135 additions & 0 deletions packages/kbn-test/src/auth/session_manager.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { ToolingLog } from '@kbn/tooling-log';
import Url from 'url';
import type { Config } from '../functional_test_runner';
import { KbnClient } from '../kbn_client';
import { getProjectType, readCloudUsersFromFile } from './helper';
import { createCloudSAMLSession, createLocalSAMLSession, Session } from './saml_auth';

export interface User {
readonly email: string;
readonly password: string;
}

export type Role = string;

export class SAMLSessionManager {
private readonly config: Config;
private readonly log: ToolingLog;
private readonly sessionCache;
private readonly isCloud: boolean;
private readonly kbnHost: string;
private readonly kbnClient: KbnClient;
private readonly roleToUserMap: Map<string, User>;
private readonly svlProjectType: string;
constructor(config: Config, log: ToolingLog, isCloud: boolean) {
this.config = config;
this.log = log;
this.sessionCache = new Map<Role, Session>();
this.isCloud = isCloud;
this.roleToUserMap = new Map<string, User>();

const hostOptions = {
protocol: this.config.get('servers.kibana.protocol'),
hostname: this.config.get('servers.kibana.hostname'),
port: isCloud ? undefined : this.config.get('servers.kibana.port'),
};
this.kbnHost = Url.format(hostOptions);
this.kbnClient = new KbnClient({
log: this.log,
url: Url.format({
...hostOptions,
auth: `${this.config.get('servers.kibana.username')}:${this.config.get(
'servers.kibana.password'
)}`,
}),
});
this.svlProjectType = getProjectType(this.config.get('kbnTestServer.serverArgs'));
}

// we should split packages/kbn-es/src/serverless_resources/roles.yml into 3 different files

// getLocalUsers = () => {
// const rolesDefinitionFilePath = resolve(
// REPO_ROOT,
// 'packages/kbn-es/src/serverless_resources/roles.yml'
// );
// const roles: string[] = Object.keys(loadYaml(fs.readFileSync(rolesDefinitionFilePath, 'utf8')));
// };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this commented out intentionally?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking to split packages/kbn-es/src/serverless_resources/roles.yml into 3 files and read as part of role validation when we run tests locally. But I will do it later to not keep PR open for long

Copy link
Member Author

@dmlemeshko dmlemeshko Dec 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Currently folks can specify non-existing role in the test and I believe tests will pass locally/Kibana CI. But it will fail on MKI pipeline since that role does not exist in real Cloud env.


private getCloudUsers = () => {
// Loading cloud users on the first call
if (this.roleToUserMap.size === 0) {
const data = readCloudUsersFromFile();
for (const [roleName, user] of data) {
this.roleToUserMap.set(roleName, user);
}
}

return this.roleToUserMap;
};

private getCloudUserByRole = (role: string) => {
if (this.getCloudUsers().has(role)) {
return this.getCloudUsers().get(role)!;
} else {
throw new Error(`User with '${role}' role is not defined for ${this.svlProjectType} project`);
}
};

private getSessionByRole = async (role: string) => {
if (this.sessionCache.has(role)) {
return this.sessionCache.get(role)!;
}

let session: Session;

if (this.isCloud) {
this.log.debug(`new SAML authentication with '${role}' role`);
const kbnVersion = await this.kbnClient.version.get();
const { email, password } = this.getCloudUserByRole(role);
session = await createCloudSAMLSession({
email,
password,
kbnHost: this.kbnHost,
kbnVersion,
log: this.log,
});
} else {
this.log.debug(`new fake SAML authentication with '${role}' role`);
session = await createLocalSAMLSession({
username: `elastic_${role}`,
email: `elastic_${role}@elastic.co`,
fullname: `test ${role}`,
role,
kbnHost: this.kbnHost,
log: this.log,
});
}

this.sessionCache.set(role, session);
return session;
};

async getApiCredentialsForRole(role: string) {
const session = await this.getSessionByRole(role);
return { Cookie: `sid=${session.getCookieValue()}` };
}

async getSessionCookieForRole(role: string) {
const session = await this.getSessionByRole(role);
return session.getCookieValue();
}

async getUserData(role: string) {
const { email, fullname } = await this.getSessionByRole(role);
return { email, fullname };
}
}
33 changes: 33 additions & 0 deletions packages/kbn-test/src/auth/types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { ToolingLog } from '@kbn/tooling-log';

export interface CloudSamlSessionParams {
kbnHost: string;
kbnVersion: string;
email: string;
password: string;
log: ToolingLog;
}

export interface LocalSamlSessionParams {
kbnHost: string;
email: string;
username: string;
fullname: string;
role: string;
log: ToolingLog;
}

export interface CreateSamlSessionParams {
hostname: string;
email: string;
password: string;
log: ToolingLog;
}
1 change: 1 addition & 0 deletions packages/kbn-test/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,6 @@
"@kbn/babel-register",
"@kbn/repo-packages",
"@kbn/core-saved-objects-api-server",
"@kbn/mock-idp-plugin",
]
}
4 changes: 2 additions & 2 deletions x-pack/test_serverless/shared/services/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
* 2.0.
*/

import { SvlReportingServiceProvider } from './svl_reporting';
import { SupertestProvider, SupertestWithoutAuthProvider } from './supertest';
import { SvlCommonApiServiceProvider } from './svl_common_api';
import { SvlUserManagerProvider } from './user_manager/svl_user_manager';
import { SvlReportingServiceProvider } from './svl_reporting';
import { SvlUserManagerProvider } from './svl_user_manager';

export const services = {
supertest: SupertestProvider,
Expand Down
19 changes: 19 additions & 0 deletions x-pack/test_serverless/shared/services/svl_user_manager.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { SAMLSessionManager } from '@kbn/test';
import { FtrProviderContext } from '../../functional/ftr_provider_context';

export function SvlUserManagerProvider({ getService }: FtrProviderContext) {
const config = getService('config');
const log = getService('log');
const isCloud = !!process.env.TEST_CLOUD;
// Sharing the instance within FTR config run means cookies are persistent for each role between tests.
const sessionManager = new SAMLSessionManager(config, log, isCloud);
Copy link
Member Author

@dmlemeshko dmlemeshko Dec 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if Cypress test runner has access to config instance, so I imagine a possible alternative like:

new SAMLSessionManager(kbnHost, auth, log, isCloud)

where kbnHost and auth FTR and Cypress runner should provide on its own:

const kbnHost = {
    protocol: config.get('servers.kibana.protocol'),
    hostname: config.get('servers.kibana.hostname'),
    port: isCloud ? undefined : config.get('servers.kibana.port'),
};
const auth = { username: config.get('servers.kibana.username')}, password: config.get('servers.kibana.password') }

@MadameSheema @pheyos wdyt?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the alternative is the best one for Cypress usage. Thanks :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in 82f750b


return sessionManager;
}
Loading