Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Validate exception list size when adding new items #73399

Merged
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions x-pack/plugins/lists/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,5 @@ export const ENDPOINT_LIST_NAME = 'Elastic Endpoint Security Exception List';

/** The description of the single global space agnostic endpoint list */
export const ENDPOINT_LIST_DESCRIPTION = 'Elastic Endpoint Security Exception List';

export const MAX_EXCEPTION_LIST_SIZE = 10000;
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import { IRouter } from 'kibana/server';

import { ENDPOINT_LIST_ITEM_URL } from '../../common/constants';
import { ENDPOINT_LIST_ID, ENDPOINT_LIST_ITEM_URL } from '../../common/constants';
import { buildRouteValidation, buildSiemResponse, transformError } from '../siem_server_deps';
import { validate } from '../../common/siem_common_deps';
import {
Expand All @@ -16,6 +16,7 @@ import {
} from '../../common/schemas';

import { getExceptionListClient } from './utils/get_exception_list_client';
import { validateExceptionListSize } from './validate';

export const createEndpointListItemRoute = (router: IRouter): void => {
router.post(
Expand Down Expand Up @@ -71,6 +72,18 @@ export const createEndpointListItemRoute = (router: IRouter): void => {
if (errors != null) {
return siemResponse.error({ body: errors, statusCode: 500 });
} else {
const listSizeError = await validateExceptionListSize(
exceptionLists,
ENDPOINT_LIST_ID,
'agnostic'
);
if (listSizeError != null) {
await exceptionLists.deleteExceptionListItemById({
id: createdList.id,
namespaceType: 'agnostic',
});
return siemResponse.error(listSizeError);
}
return response.ok({ body: validated ?? {} });
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {

import { getExceptionListClient } from './utils/get_exception_list_client';
import { endpointDisallowedFields } from './endpoint_disallowed_fields';
import { validateExceptionListSize } from './validate';

export const createExceptionListItemRoute = (router: IRouter): void => {
router.post(
Expand Down Expand Up @@ -104,6 +105,18 @@ export const createExceptionListItemRoute = (router: IRouter): void => {
if (errors != null) {
return siemResponse.error({ body: errors, statusCode: 500 });
} else {
const listSizeError = await validateExceptionListSize(
exceptionLists,
listId,
namespaceType
);
if (listSizeError != null) {
await exceptionLists.deleteExceptionListItemById({
id: createdList.id,
namespaceType,
});
return siemResponse.error(listSizeError);
}
return response.ok({ body: validated ?? {} });
}
}
Expand Down
56 changes: 56 additions & 0 deletions x-pack/plugins/lists/server/routes/validate.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { ExceptionListClient } from '../services/exception_lists/exception_list_client';
import { MAX_EXCEPTION_LIST_SIZE } from '../../common/constants';
import { foundExceptionListItemSchema } from '../../common/schemas';
import { NamespaceType } from '../../common/schemas/types';
import { validate } from '../../common/siem_common_deps';

export const validateExceptionListSize = async (
exceptionLists: ExceptionListClient,
listId: string,
namespaceType: NamespaceType
): Promise<{ body: string; statusCode: number } | null> => {
const exceptionListItems = await exceptionLists.findExceptionListItem({
filter: undefined,
listId,
namespaceType,
page: undefined,
perPage: undefined,
sortField: undefined,
sortOrder: undefined,
});
if (exceptionListItems == null) {
// If exceptionListItems is null then we couldn't find the list so it may have been deleted
return {
body: `Unable to find list id: ${listId} to verify max exception list size`,
statusCode: 500,
};
}
const [validatedItems, err] = validate(exceptionListItems, foundExceptionListItemSchema);
if (err != null) {
return {
body: err,
statusCode: 500,
};
}
// Unnecessary since validatedItems comes from exceptionListItems which is already
// checked for null, but typescript fails to detect that
if (validatedItems == null) {
return {
body: `Unable to find list id: ${listId} to verify max exception list size`,
statusCode: 500,
};
}
if (validatedItems.total > MAX_EXCEPTION_LIST_SIZE) {
return {
body: `Failed to add exception item, exception list would exceed max size of ${MAX_EXCEPTION_LIST_SIZE}`,
statusCode: 400,
Copy link
Contributor

@FrankHassanabad FrankHassanabad Jul 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically use statusCode camel case for internals but when we return the actual status code we transform it at the last moment to be status_code to keep it as snake case. Will this return a camel case to the rest response or will it return snake_case? Looks like camel case.

For historical perspective, not all teams return status_code in even newer code. Some return statusCode and the reason that some utilities and teams still use statusCode is that it is from Hapi Boom era and there's just a lot of code still using that.

However, for detection rules and this repo we are trying to use the newer status_code and stay one step ahead if possible and on the front end a lot of times our utilities can inspect either the statusCode or the status_code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the historical reasoning is interesting. The result of this function is passed in to siemResponse.error which does the conversion so the REST response will eventually return it as status_code.

};
}
return null;
};
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { SavedObjectsClientContract } from 'kibana/server';

import {
ExceptionListItemSchema,
Id,
IdOrUndefined,
ItemIdOrUndefined,
NamespaceType,
Expand All @@ -23,6 +24,12 @@ interface DeleteExceptionListItemOptions {
savedObjectsClient: SavedObjectsClientContract;
}

interface DeleteExceptionListItemByIdOptions {
id: Id;
namespaceType: NamespaceType;
savedObjectsClient: SavedObjectsClientContract;
}

export const deleteExceptionListItem = async ({
itemId,
id,
Expand All @@ -43,3 +50,12 @@ export const deleteExceptionListItem = async ({
return exceptionListItem;
}
};

export const deleteExceptionListItemById = async ({
id,
namespaceType,
savedObjectsClient,
}: DeleteExceptionListItemByIdOptions): Promise<void> => {
const savedObjectType = getSavedObjectType({ namespaceType });
await savedObjectsClient.delete(savedObjectType, id);
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function above this works where you can call:

deleteExceptionListItem({ itemId: undefined, id, namespaceType, savedObjectsClient  });

Are you just trying to avoid the extra call to get the item if it does not exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That, and also it avoids having to check for a null return value as an error case - savedObjectsClient.delete should either succeed or throw

Copy link
Contributor

@FrankHassanabad FrankHassanabad Jul 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I hear ya'.

I think there's been a bit of back and forth throughout the API's throughout the code with regards to throws and return statements that can contain errors. Sometimes people will want to do a throw and unwind the stack and then other times I see people wrapping those in a catch everywhere and then returning null and preferring the non-throw states (or wrapping it in an object/array that can contain an error and a response and optionally typing both).

In some cases people have been using more of a disjoint union such as the Either and io-ts does use this for their return types:
https://gcanti.github.io/fp-ts/modules/Either.ts.html

(Similar to RustLang but Rust has more first class citizen treatment of it)

So I have seen people kind of go back and forth depending on who I program with. You don't have to change anything, I'm fine with it, as it works for your use case. I think in newer contemporary programming languages (golang, rustlang) most errors are now returned and not many throws are used unless they are something like a panic. Regardless of if they're using a disjointed union type, array type, etc...

TypeScript makes it a pain and close to impossible to strongly type throws or advertise what something can throw and when it can throw, so some circles of people tend to reduce their usage unless it's a panic type thing or they have nothing left to do way down the stack and kind of just say, "Ugh...Guess I should just unwind with a throw or I have to re-do all the return types throughout this piece of code".

I kind of do all of the above ;-) depending mostly on where I am in the code base and sometimes where my head is at, but for the validation type things which are right now returning arrays I know that we have been considering embracing more io-ts Either instead of flattening it out into an array and returning that as an Either at some point.

Again, no changes this is fine as is, just some thoughts of what I have noticed.

Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
CreateExceptionListItemOptions,
CreateExceptionListOptions,
DeleteEndpointListItemOptions,
DeleteExceptionListItemByIdOptions,
DeleteExceptionListItemOptions,
DeleteExceptionListOptions,
FindEndpointListItemOptions,
Expand All @@ -40,7 +41,7 @@ import { createExceptionListItem } from './create_exception_list_item';
import { updateExceptionList } from './update_exception_list';
import { updateExceptionListItem } from './update_exception_list_item';
import { deleteExceptionList } from './delete_exception_list';
import { deleteExceptionListItem } from './delete_exception_list_item';
import { deleteExceptionListItem, deleteExceptionListItemById } from './delete_exception_list_item';
import { findExceptionListItem } from './find_exception_list_item';
import { findExceptionList } from './find_exception_list';
import { findExceptionListsItem } from './find_exception_list_items';
Expand Down Expand Up @@ -326,6 +327,18 @@ export class ExceptionListClient {
});
};

public deleteExceptionListItemById = async ({
id,
namespaceType,
}: DeleteExceptionListItemByIdOptions): Promise<void> => {
const { savedObjectsClient } = this;
return deleteExceptionListItemById({
id,
namespaceType,
savedObjectsClient,
});
};

/**
* This is the same as "deleteExceptionListItem" except it applies specifically to the endpoint list.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import {
ExceptionListType,
ExceptionListTypeOrUndefined,
FilterOrUndefined,
Id,
IdOrUndefined,
Immutable,
ItemId,
Expand Down Expand Up @@ -93,6 +94,11 @@ export interface DeleteExceptionListItemOptions {
namespaceType: NamespaceType;
}

export interface DeleteExceptionListItemByIdOptions {
id: Id;
namespaceType: NamespaceType;
}

export interface DeleteEndpointListItemOptions {
id: IdOrUndefined;
itemId: ItemIdOrUndefined;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import { ListArrayOrUndefined } from '../../../../common/detection_engine/schema
import { BulkResponse, BulkResponseErrorAggregation, isValidUnit } from './types';
import { BuildRuleMessage } from './rule_messages';
import { hasLargeValueList } from '../../../../common/detection_engine/utils';
import { MAX_EXCEPTION_LIST_SIZE } from '../../../../../lists/common/constants';

interface SortExceptionsReturn {
exceptionsWithValueLists: ExceptionListItemSchema[];
Expand Down Expand Up @@ -183,7 +184,7 @@ export const getExceptions = async ({
listId: foundList.list_id,
namespaceType,
page: 1,
perPage: 10000,
perPage: MAX_EXCEPTION_LIST_SIZE,
filter: undefined,
sortOrder: undefined,
sortField: undefined,
Expand Down