Skip to content

Commit

Permalink
Ensure that data is not lost when parsing EQL responses
Browse files Browse the repository at this point in the history
The shared search utilities expect that response data exists in the
response's body field. However, in an EQL response this information also
exists as a sibling to the body field, and so we must normalize this data
into the body before we can leverage these utilities with EQL queries.
  • Loading branch information
rylnd committed Nov 9, 2020
1 parent b08677b commit 9e5abf4
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
*/

import { of, merge, timer, throwError } from 'rxjs';
import { takeWhile, switchMap, expand, mergeMap, tap } from 'rxjs/operators';
import { map, takeWhile, switchMap, expand, mergeMap, tap } from 'rxjs/operators';
import { ApiResponse } from '@elastic/elasticsearch';

import {
AbortError,
Expand Down Expand Up @@ -35,6 +36,15 @@ export const doPartialSearch = <SearchResponse = any>(
takeWhile((response) => !isCompleteResponse(response), true)
);

export const normalizeEqlResponse = <SearchResponse extends ApiResponse = ApiResponse>() =>
map<SearchResponse, SearchResponse>((eqlResponse) => ({
...eqlResponse,

This comment has been minimized.

Copy link
@yctercero

yctercero Nov 12, 2020

Contributor

So looks like the shape of the response had changed, but not necessarily it's contents? That's good. I was worried we wouldn't have access to that meta information any more.

This comment has been minimized.

Copy link
@rylnd

rylnd Nov 12, 2020

Author Contributor

On the elasticsearch side, nothing has changed. The use of toKibanaSearchResponse was added to the EQL strategy, which makes some assumptions about where the response data lives (in the body field). This new normalizer function takes all the fields we need in the response meta, statusCode and copies them into body, which toKibanaSearchResponse then converts into rawResponse in the client response.

body: {
...eqlResponse.body,
...eqlResponse,
},
}));

export const throwOnEsError = () =>
mergeMap((r: IKibanaSearchResponse) =>
isErrorResponse(r) ? merge(of(r), throwError(new AbortError())) : of(r)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ const getMockEqlResponse = () => ({
sequences: [],
},
},
meta: {},
statusCode: 200,
});

describe('EQL search strategy', () => {
Expand Down Expand Up @@ -193,5 +195,24 @@ describe('EQL search strategy', () => {
expect(requestOptions).toEqual(expect.objectContaining({ ignore: [400] }));
});
});

describe('response', () => {
it('contains a rawResponse with body and meta', async () => {
const eqlSearch = await eqlSearchStrategyProvider(mockLogger);
const response = await eqlSearch
.search({ id: 'my-search-id', options: { ignore: [400] } }, {}, mockDeps)
.toPromise();

expect(response).toEqual(
expect.objectContaining({
rawResponse: expect.objectContaining({
body: expect.any(Object),
meta: expect.any(Object),
statusCode: 200,
}),
})
);
});
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ import type { Logger } from 'kibana/server';
import type { ApiResponse } from '@elastic/elasticsearch';

import { search } from '../../../../../src/plugins/data/server';
import { doPartialSearch } from '../../common/search/es_search/es_search_rxjs_utils';
import {
doPartialSearch,
normalizeEqlResponse,
} from '../../common/search/es_search/es_search_rxjs_utils';
import { getAsyncOptions, getDefaultSearchParams } from './get_default_search_params';

import type { ISearchStrategy, IEsRawSearchResponse } from '../../../../../src/plugins/data/server';
Expand Down Expand Up @@ -64,7 +67,7 @@ export const eqlSearchStrategyProvider = (
(response) => response.body.id,
request.id,
options
).pipe(utils.toKibanaSearchResponse());
).pipe(normalizeEqlResponse(), utils.toKibanaSearchResponse());
},
};
};

0 comments on commit 9e5abf4

Please sign in to comment.