4.3-rc1 (May 17 2018)

Improvements

ENGINE-1075 Add Prometheus metrics endpoint

The engine now directly supports Prometheus metrics via a new endpoint at /ws/metrics. See Metrics Service for more information.

ENGINE-1073 Allow indexValues, values, and export APIs to export the indexed groupBy value

The indexValues, values, and export APIs now all support expose the indexed groupBy value for an item. Previously you would get no value back if requested.

ENGINE-1072 Streaming export API

The new streaming export API that can be used to export all data or to retrieve all results from a query. The results are not sorted and are returned in an unstable order. Subsequent calls are not guaranteed to return the results in the same order.

It supports regular search criteria and can output a combination of meta data (id, score, exact), item properties, and indexed values.

It does not support sortBy, groupBy or any pagination specific features from the query API.

See Export API for more details.

ENGINE-1040 ENGINE-1071 Upgrades Apache Lucene from 6.5.1 to 7.3.1

The Apache Lucene library has been upgraded from 6.5.1 to 7.3.1.

ENGINE-1070 Upgrades dependent libraries

Upgrades dependent libraries.

library previous current
commons-codec 1.10 1.11
commons-fileupload 1.3.2 1.3.3
commons-io 2.5 2.6
icu4j 58.2 59.1
jetty 9.4.1 9.4.10
prometheus   0.4.0
springframework 4.3.6 5.0.6
slf4j 1.7.22 1.7.25

ENGINE-1069 Index values should use LeafReader instead of MultiReader

The indexValues and values APIs now use LeafReader instead of MultiReader giving them a small performance boost.

ENGINE-1068 Less chatty JSON encoding of the dollar sign

Dollar signs in JSON strings were encoded as \u0024 instead of $. This change does not affect compatibility for consumers of any API other than to reduce the size of payloads that contain dollar signs.

ENGINE-1067 Do not store duplicate _id field in item properties

When storing the properties for an item the engine no longer includes a copy of the identifier as _id. It has always filtered out reserved properties (those that begin with an _) but previously used to store a duplicate copy of the id with the properties. This reduces storage requirements a little.

Any APIs that return properties which include _id remain unchanged, the engine dynamically includes this extra property to avoid any compatibility issues.

ENGINE-1065 Adds compressed response support to all API endpoints

All API endpoints can now generate gzip compressed responses. This impacts a handful of internal endpoints used to drive the admin interface and the following documented endpoints:

  • /ws/didyoumean/{id}
  • /ws/item/{id}
  • /ws/items

The response will be compressed if the Accept-Encoding request header contains the value gzip.

ENGINE-1066 Use sparse bitsets at search time

The engine now uses sparse bitsets when appropriate for query time scratch space and when applying facets. This optimization improves the speed and memory use of some queries.

Bug Fixes

ENGINE-1079 Race condition in progress monitoring

A race condition has been found that can cause a NullPointerException when generating progress reports used to drive the main status page of the engine. This would cause the page to be unavailable. The engine now correctly generates this report and thus renders the page.

ENGINE-1077 Ensure forceSnapshot doesn’t trigger on reboot

The forceSnapshot feed setting is a one-shot deal, it causes a one time full feed fetch (with retries on failure). It was not saved to disk after a success fetch which would then trigger unnecessary feed fetches during startup. The engine now correctly saves this setting on success.

ENGINE-1074 Debug sortBy response field renamed

The introduction of a separate fuzzy sort in release 4.2 altered the debug output for sortBy renaming the response field from sortBy to sections.

When groupBy is enabled this rename did not take affect, this is now fixed and the following debug response fields have been renamed when groupBy is in action.

from to
sortBy sections
sortByWithinGroup sectionsWithinGroups

Compatability

ENGINE-1075 Add Prometheus metrics endpoint

The Statistics Service is now deprecated and will be removed in a future release. It has been replaced by the new Metrics Service.