-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add filtering to fieldcaps endpoint #83636
Changes from all commits
abe01d8
3771d63
27687fe
5a6868e
86f932e
2679222
32a5c35
1a65377
851755f
6c0d3e9
196016f
37a70fc
31da51b
375ad2c
2a9f5d7
77d0557
7d65530
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
pr: 83636 | ||
summary: Add filtering to fieldcaps endpoint | ||
area: Search | ||
type: enhancement | ||
issues: | ||
- 82966 |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -77,6 +77,15 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailab | |
(Optional, Boolean) If `true`, unmapped fields are included in the response. | ||
Defaults to `false`. | ||
|
||
`filters`:: | ||
(Optional, string) Comma-separated list of filters to apply to the response. The | ||
following filters are supported: +metadata,-metadata,-parent,-nested,-multifield | ||
|
||
`types`:: | ||
(Optional, string) Comma-separated list of field types to include. Any fields that | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. shall we link the docs page with all of the field types here? Maybe also include an example at the end of the sentence? |
||
do not match one of these types will be excluded from the results. Defaults to empty, | ||
meaning that all field types are returned. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One more question, just to double check: types:keyword means either runtime or indexed field, no distinction between the two for field_caps? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Keyword runtime fields return There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. let's wait for them to ask? :) |
||
|
||
[[search-field-caps-api-request-body]] | ||
==== {api-request-body-title} | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,213 @@ | ||
--- | ||
setup: | ||
- skip: | ||
version: "- 8.1.99" | ||
reason: Field type filters were added in 8.2 | ||
- do: | ||
indices.create: | ||
index: test1 | ||
body: | ||
mappings: | ||
properties: | ||
text: | ||
type: text | ||
fields: | ||
keyword: | ||
type: keyword | ||
keyword: | ||
type: keyword | ||
number: | ||
type: double | ||
geo: | ||
type: geo_point | ||
misc: | ||
type: text | ||
object: | ||
type: object | ||
properties: | ||
nested1 : | ||
type : text | ||
index: false | ||
nested2: | ||
type: float | ||
doc_values: false | ||
level1: | ||
type: nested | ||
properties: | ||
level2: | ||
type: object | ||
properties: | ||
leaf1: | ||
type: text | ||
index: false | ||
runtime: | ||
misc.keyword: | ||
type: keyword | ||
|
||
- do: | ||
indices.create: | ||
index: test2 | ||
body: | ||
mappings: | ||
properties: | ||
text: | ||
type: text | ||
keyword: | ||
type: keyword | ||
number: | ||
type: double | ||
date: | ||
type: date | ||
geo: | ||
type: geo_point | ||
object: | ||
type: object | ||
properties: | ||
nested1 : | ||
type : text | ||
index: true | ||
nested2: | ||
type: float | ||
doc_values: true | ||
level1: | ||
type: nested | ||
properties: | ||
level2: | ||
type: object | ||
properties: | ||
leaf1: | ||
type: text | ||
index: false | ||
- do: | ||
indices.create: | ||
index: test3 | ||
body: | ||
mappings: | ||
properties: | ||
text: | ||
type: text | ||
keyword: | ||
type: keyword | ||
number: | ||
type: long | ||
date: | ||
type: date | ||
non_indexed_date: | ||
type: date | ||
index: false | ||
non_indexed_keyword: | ||
type: keyword | ||
index: false | ||
non_indexed_boolean: | ||
type: boolean | ||
index: false | ||
non_indexed_ip: | ||
type: ip | ||
index: false | ||
non_indexed_geo_point: | ||
type: geo_point | ||
index: false | ||
geo: | ||
type: keyword | ||
object: | ||
type: nested | ||
properties: | ||
nested1 : | ||
type : long | ||
index: false | ||
nested2: | ||
type: keyword | ||
doc_values: false | ||
--- | ||
"No filters includes all the following fields": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
|
||
- is_true: fields.object | ||
- is_true: fields.text | ||
- is_true: fields.text\\.keyword | ||
- is_true: fields._seq_no | ||
- is_true: fields.level1\\.level2\\.leaf1 | ||
- is_true: fields.level1 | ||
|
||
--- | ||
"Exclude parent objects": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
filters: '-parent' | ||
|
||
- is_true: fields.object\\.nested1 | ||
- is_false: fields.object | ||
|
||
--- | ||
"Exclude metadata fields": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
filters: '-metadata' | ||
|
||
- is_false: fields._seq_no | ||
|
||
--- | ||
"Exclude non-metadata fields": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
filters: '+metadata' | ||
|
||
- is_true: fields._seq_no | ||
- is_false: fields.text | ||
|
||
--- | ||
"Exclude nested fields": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
filters: '-nested' | ||
|
||
- is_false: fields.level1 | ||
- is_false: fields.level1\\.level2\\.leaf1 | ||
|
||
--- | ||
"Exclude multifields": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
filters: '-multifield' | ||
|
||
- is_false: fields.text\\.keyword | ||
- is_true: fields.misc\\.keyword | ||
|
||
--- | ||
"Field type filters": | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add a test case for multiple values in the |
||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: '*' | ||
types: 'text,keyword,long' | ||
|
||
- is_false: fields.date | ||
- is_false: fields.non_indexed_boolean | ||
- is_true: fields.non_indexed_keyword | ||
- is_true: fields.misc | ||
|
||
--- | ||
"Field type filters with field name restrictions": | ||
- do: | ||
field_caps: | ||
index: 'test1,test2,test3' | ||
fields: 'non_*,text' | ||
types: 'text,keyword,long' | ||
|
||
- is_false: fields.non_indexed_boolean | ||
- is_true: fields.non_indexed_keyword | ||
- is_false: fields.misc | ||
- is_true: fields.text |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is the
+
notation supported only for metadata? Would it make sense to expand on what each one of these mean, with a link to the relevant docs explanation for each one?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll expand the docs. The idea is that
-
and+
work in much the same way they do in our query parsers =-
means 'must not match', and+
means 'must match'. So you can ask for only metadata, or to exclude metadata. For the others it doesn't make much sense to do anything except exclude them, with the exception ofparent
where it sort of makes sense to say 'include parent data as well as everything else', so a bareparent
term is supported.