NAME
Elastic::Manual::QueryDSL::Filters - Overview of the filters available in Elasticsearch
VERSION
version 0.52
INTRODUCTION
Filters are pretty self explanatory. We present some of the most common filters below. To understand more about what filters are available, and how they work, you should read the about the Query DSL on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html and, optionally, the ElasticSearch::SearchBuilder syntax. All examples will be given first in the SearchBuilder syntax (with filterb()
), then in the native Query DSL.
USING FILTERS ON ANALYZED TEXT FIELDS
It is important to remember that filters work on exact values. So "Foo"
will not match "FOO"
.
While you can use a filter on an analyzed text field, you need to filter on the actual terms that are stored in that field. For instance, if the attribute value is: "A quick Brown FOX"
, it might be analyzed to the terms ['quick','brown','fox']
. Filtering on "Brown"
will not work. You need to use the actual term: "brown"
instead.
Alternatively, if you need the analysis phase, you can use a query as a filter.
COMMONLY USED FILTERS
Equality
- SearchBuilder:
-
# WHERE status = 'active'
$view
->filterb(
status
=>
'active'
);
# WHERE count = 5
$view
->filterb(
count
=> 5 );
# WHERE tags IN ('perl','python')
$view
->filterb(
tags
=> [
'perl'
,
'python'
]);
- QueryDSL:
-
# WHERE status = 'active'
$view
->filter(
term
=> {
status
=>
'active'
} );
# WHERE count = 5
$view
->filter(
term
=> {
count
=> 5 );
# WHERE tags IN ('perl','python')
$view
->filter(
terms
=> {
tag
=> [
'perl'
,
'python'
]})
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html and http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html.
Range
- SearchBuilder:
-
# WHERE date BETWEEN '2012-01-01' AND '2013-01-01'
$view
->filterb(
date
=> {
gte
=>
'2012-01-01'
,
lt
=>
'2013-01-01'
}
);
- QueryDSL:
-
# WHERE date BETWEEN '2012-01-01' AND '2013-01-01'
$view
->filter(
range
=> {
date
=> {
gte
=>
'2012-01-01'
,
lt
=>
'2013-01-01'
}
}
);
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-range-filter.html
And, Or and Not
- SearchBuilder:
-
See "AND|OR LOGIC" in ElasticSearch::SearchBuilder
- And
-
# WHERE status = 'active' AND count > 5
$view
->filterb(
status
=>
'active'
,
count
=> {
gt
=> 5 } );
- Or
-
# WHERE status = 'active' OR count > 5
$view
->filterb([
status
=>
'active'
,
count
=> {
gt
=> 5 } ]);
- Not
-
# WHERE status <> 'active'
$view
->filterb(
status
=> {
'!='
=>
'active'
});
# WHERE tags NOT IN ('perl','python')
$view
->filterb(
tags
=> {
'!='
=> [
'perl'
,
'python'
] });
# WHERE NOT ( x = 1 AND y = 2 )
$view
->filterb(
-not
=> {
x
=> 1,
y
=> 2 });
# WHERE NOT ( x = 1 OR y = 2 )
$view
->filterb(
-not
=> [
x
=> 1,
y
=> 2 ]);
- QueryDSL:
-
- And
-
# WHERE status = 'active' AND count > 5
$view
->filter(
and
=> [
{
term
=> {
status
=>
'active'
}},
{
range
=> {
count
=> {
gt
=> 5 }}}
]
);
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-and-filter.html
- Or
-
# WHERE status = 'active' OR count > 5
$view
->filter(
or
=> [
{
term
=> {
status
=>
'active'
}},
{
range
=> {
count
=> {
gt
=> 5 }}}
]
);
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-or-filter.html
- Not
-
# WHERE status <> 'active'
$view
->filter(
not
=> {
term
=> {
status
=>
'active'
}});
# WHERE tags NOT IN ('perl','python')
$view
->filter(
not
=> {
terms
=> {
tags
=> [
'perl'
,
'python'
] }});
# WHERE NOT ( x = 1 AND y = 2 )
$view
->filter(
not
=> {
and
=> [
{
term
=> {
x
=> 1 }},
{
term
=> {
y
=> 2 }}
]
}
);
# WHERE NOT ( x = 1 OR y = 2 )
$view
->filter(
not
=> {
or
=> [
{
term
=> {
x
=> 1 }},
{
term
=> {
y
=> 2 }}
]
}
);
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-not-filter.html
Exists and Missing
- SearchBuilder:
-
# WHERE status IS NULL
$view
->filterb(
-missing
=>
'status'
);
$view
->filterb(
status
=>
undef
);
# WHERE status IS NOT NULL
$view
->filterb(
-exists
=>
'status'
);
See "MISSING OR NULL VALUES" in ElasticSearch::SearchBuilder.
- QueryDSL:
-
# WHERE status IS NULL
$view
->filter(
missing
=> {
field
=>
'status'
});
# WHERE status IS NOT NULL
$view
->filter(
exists
=> {
field
=>
'status'
});
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-missing-filter.html and http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-exists-filter.html.
Prefix
Warning: The prefix filter does not peform well. First it has to load all terms into memory to find those that begin with the prefix. Then it searches on all of those terms.
If you find yourself wanting to use a prefix filter, then you should rather use the edge_ngram token filter to prepare your field correctly for partial matching (eg "ABC"
becomes "A"
, "AB"
, "ABC"
), and use simple "Equality" instead.
- SearchBuilder
-
# WHERE code LIKE 'XYZ_%'
$view
->filterb(
code
=> {
'^'
=>
'XYZ_'
});
$view
->filterb(
code
=> {
prefix
=>
'XYZ_'
});
- Query DSL
-
# WHERE code LIKE 'XYZ_%'
$view
->filter(
prefix
=> {
code
=>
'XYZ_'
});
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-prefix-filter.html.
Geolocation
- SearchBuilder:
-
# where "point" is less than 50km from (lat:10, lon:5)
$view
->filterb(
point
=> {
-geo_distance
=> {
location
=> {
lat
=> 10,
lon
=> 5 },
distance
=>
'50km'
}
}
);
See:
Geo-distance: "-geo_distance | -not_geo_distance" in ElasticSearch::SearchBuilder
Geo-distance ranges: "-geo_distance_range | -not_geo_distance_range" in ElasticSearch::SearchBuilder.
Geo-bounding boxes: "-geo_bounding_box | -geo_bbox | -not_geo_bounding_box | -not_geo_bbox" in ElasticSearch::SearchBuilder.
Geo-polygons: "-geo_polygon | -not_geo_polygon" in ElasticSearch::SearchBuilder.
- QueryDSL:
-
$view
->filter(
geo_distance
=> {
distance
=>
'50km'
,
point
=> {
lat
=> 10,
lon
=> 5 }
}
);
See:
Geo-distance: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-filter.html
Geo-distance ranges: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-range-filter.html.
Geo-bounding boxes: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-bounding-box-filter.html.
Geo-polygons: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-polygon-filter.html.
Types, IDs and Elastic::Model::UID
- SearchBuilder
-
# WHERE doc.id = 1
$view
->filterb(
_id
=> 1 );
# WHERE doc.id IN (1,2,3)
$view
->filterb(
_id
=> [1,2,3] );
# WHERE doc.id in (1,2,3) AND doc.type in ('user','post')
$view
->filterb(
_id
=> [1,2,3],
_type
=> [
'user'
,
'post'
] );
Assuming that a document has an Elastic::Doc attribute
user
:# WHERE user_id = 1
$view
->filterb(
'user.uid.id'
=> 1 );
# WHERE user_type = 'user'
$view
->filterb(
'user.uid.type'
=>
'user'
);
- Query DSL
-
# WHERE doc.id = 1
$view
->filter(
term
=> {
_id
=> 1 } );
# WHERE doc.id IN (1,2,3)
$view
->filter(
terms
=> {
_id
=> [1,2,3] });
# WHERE doc.id in (1,2,3) AND doc.type in ('user','post')
$view
->filter(
and
=> [
{
terms
=> {
_id
=> [1,2,3] }},
{
terms
=> {
_type
=> [
'user'
,
'post'
] }}
]
);
Assuming that a document has an Elastic::Doc attribute
user
:# WHERE user_id = 1
$view
->filter(
term
=> {
'user.uid.id'
=> 1 });
# WHERE user_type = 'user'
$view
->filter(
term
=> {
'user.uid.type'
=>
'user'
});
Scripts
Script filters can be written in mvel, groovy, javascript, python or java. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html. Scripts can be useful, but they do have a performance impact, so consider whether you really need a script, or whether you could achieve your goals by indexing a field differently.
Note: Mvel is deprecated and will be removed in Elasticsearch 1.4. In Elasticsearch 1.2 and above it is disabled by default. The new default scripting language is Groovy.
See http://www.elasticsearch.org/blog/scripting/ for more.
- SearchBuilder
-
$view
->filterb(
-script
=> {
script
=>
"doc['foo'].value > minimum"
,
params
=> {
minimum
=> 5 },
lang
=>
'groovy'
}
);
- QueryDSL
-
$view
->filter(
script
=> {
script
=>
"doc['foo'].value > minimum"
,
params
=> {
minimum
=> 5 },
lang
=>
'groovy'
}
);
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html
Using a query as a filter
If you need to use a query as a filter (ie it can do full text matching, but won't be scored), you can embed a query in a filter:
- SearchBuilder
-
# WHERE status = 'active' AND matches(title, 'perl python')
$view
->filterb(
status
=>
'active'
,
-query
=> {
title
=>
'perl python'
}
);
See "QUERY / FILTER CONTEXT" in ElasticSearch::SearchBuilder.
- QueryDSL
-
# WHERE status = 'active' AND matches(title, 'perl python')
$view
->filter(
and
=> [
{
term
=> {
status
=>
'active'
}},
{
query
=> {
text
=> {
title
=>
'perl python'
}
}}
]
);
See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html.
Parent-child filters
Parent-child relationships are not yet supported natively in Elastic::Model. They will be soon.
In the meantime, see:
Nested filters
See Elastic::Manual::QueryDSL::Nested.
SEE ALSO
AUTHOR
Clinton Gormley <drtech@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2015 by Clinton Gormley.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.