3.16 (May 8 2015)

Improvements

ENGINE-986 The isGrouped property now occurs after relevanceValues

The order of keys in the query output has changed slightly, now isGrouped appears after relevanceValues. Additionally the groups and legacyGroupBy fields are only included when grouping occurs.

Before:

{
    "itemIds": ["example"],
    "isGrouped": true,
    "groups": {
        "itemIds": [["example"]],
        "exactMatches": [[true]],
        "relevanceValues": [[1.0]]},
    "exactMatches": [true],
    "relevanceValues": [1.0],
     "pageSize": 10,
    "currentPageSize": 1,
    "startIndex": 0,
    "exactSize": 1,
    "totalSize": 1,
    "datasetSize": 100
}

After:

{
    "itemIds": ["example"],
    "exactMatches": [true],
    "relevanceValues": [1.0],
    "isGrouped": true,
    "groups": {
        "itemIds": [["example"]],
        "exactMatches": [[true]],
        "relevanceValues": [[1.0]]},
    "pageSize": 10,
    "currentPageSize": 1,
    "startIndex": 0,
    "exactSize": 1,
    "totalSize": 1,
    "datasetSize": 100
}

ENGINE-989 Better handling of top-level weights with exactRelevance, minRelevance and maxRelevance

The interaction between a top-level weight and exactRelevance, minRelevance, and maxRelevance is now well defined.

exactRelevance now respects any top-level weight.

minRelevance and maxRelevance options apply against the final score and happen after any exactRelevance override.

Example response:

{
    "itemIds": ["a", "b"],
    "exactMatches": [true, false],
    "relevanceValues": [1.0, 0.5]
}

Combining exactRelevance and weight:

{
    "criteria": ["..."],
    "exactRelevance": 0.8,
    "weight": 0.1
}

Produces:

{
    "itemIds": ["a", "b"],
    "exactMatches": [true, false],
    "relevanceValues": [0.08, 0.05]
}

Combining minRelevance and weight:

{
    "criteria": ["..."],
    "minRelevance": 0.4,
    "weight": 0.5
}

Produces:

{
    "itemIds": ["a"],
    "exactMatches": [true],
    "relevanceValues": [0.5]
}

ENGINE-993 Supports more headers for CORS requests

CORS requests now allow Content-Encoding and X-Requested-With headers. This enables easier custom tooling from the browser.

ENGINE-991 Upgrades dependent libraries

Upgrades dependent libraries.

library previous current
slf4j 1.7.7 1.7.12
spring-framework 4.0.6 4.1.6
woodstox-core-asl 4.4.0 4.4.1
jackson-core-asl 2.4.2 2.5.3
commons-lang3 3.3.2 3.4
commons-codec 1.9 1.10
icu4j 53.1 55.1

ENGINE-994 Empty facet requests are now respected

An empty facet request is no longer ignored and will instead produce an empty facet response.

The following query:

{
    "facets": {}
}

Used to produce:

{
}

Now produces:

{
    "facets": {}
}

ENGINE-997 Facets API now ignores includeLabel, maxDepth and rootId for keyword dimensions

The includeLabel, maxDepth, and rootId facet options were previously undefined when faceting over a keyword dimension. They are now explicitly ignored.

ENGINE-999 Multi-level sort in the facets API

Facet sort now includes labelAsc and labelDesc options and supports multiple levels of sorting. The previous behavior had undocumented secondary and tertiary sort options applied, this is no longer the case and all sorting is explicit.

Single sort option:

{
    "facets": {
        "example": {"sortBy": "countDesc"}
    }
}

Multiple sort options:

{
    "facets": {
        "example": {"sortBy": ["countDesc", "labelAsc"]}
    }
}

ENGINE-990, ENGINE-1008 Better handling of thread interrupts

All thread interrupt handling logic has been reviewed and fixed. The engine no longer interrupts itself unless shutting down. This improves compatibility with Java NIO.

ENGINE-1012 Adds isolatedId option to control facet isolation

You can now specify an isolatedId for any top-level criterion to control facet isolation. It defaults to the value of dimension from the same criterion. You can disable facet isolation for a criterion by setting isolatedId to "" or null. This is useful to control isolation on top-level nested criteria.

Don’t isolated on the first criterion:

{
  "criteria": [
      {
          "dimension": "example",
          "value": "a",
          "isolatedId": null
      },
      {
          "dimension": "example",
          "value": "b"
      }

  ],
  "facets": {
    "example": {}
  }
}

Isolate on the nested criterion:

{
  "criteria": [
      {
          "dimension": "type",
          "id": "my-content",
      },
      {
          "isolatedId": "geotag",
          "operator": "or",
          "criteria": [
              {
                  "dimension": "geoloc",
                  "longitude": -72,
                  "latitude": 42
                  "cullDistance": 10,
                  "exactMatch": true
              },
              {
                  "dimension": "geotag",
                  "id": "my-place"
              }
          ]
      }
  ],
  "facets": {
    "type": {},
    "geotag": {}
  }
}

ENGINE-1013 Introduces query logging

The Settings tab of the admin interface has two new options under Logging which enable logging of query requests and responses.

The log entries are stored in a new file log/queries.log (next to discovery.log). This log file has a limit of 10MB with up to 10 rotations.

Logging just query requests:

[2015-05-07 15:20:30,000] Request 0ms {"criteria":[]}

Logging just query responses:

[2015-05-07 15:25:30,000] Response 0ms {"itemIds":[],"exactMatches":[],"relevanceValues":[],"currentPageSize":0,"startIndex":0,"exactSize":0,"totalSize":0,"datasetSize":0}

Logging both query requests and responses:

[2015-05-07 15:30:30,000] RequestResponse 0ms {"criteria":[]} {"itemIds":[],"exactMatches":[],"relevanceValues":[],"currentPageSize":0,"startIndex":0,"exactSize":0,"totalSize":0,"datasetSize":0}

Note that because JSON requires that control characters in strings be escaped, each of these entries is guaranteed to be one one line only.

Compatibility Changes

ENGINE-996 Facets API no longer filters out unknown keyword facets

When asking for facet counts on a keyword dimension, facets that do not exist will now report a count of 0.

The following query:

{
    "facets": {
        "example": {
            "dataIds": ["exists", "doesNotExist"],
            "navigable": false
        }
    }
}

Used to produce:

{
    "facets": {
        "example": {
            "dataIds": ["exists"],
            "data": {
                "exists": {"count": 10}
            }
        }
    }
}

Now produces:

{
    "facets": {
        "example": {
            "dataIds": ["exists", "doesNotExist"],
            "data": {
                "exists": {"count": 10},
                "doesNotExist": {"count": 0}
            }
        }
    }
}

ENGINE-998 Facets API no longer filters invalid selected ids from navIds for keyword dimensions

A follow on from ENGINE-996, invalid facets are now respected in the navIds list for keyword dimensions.

The following query:

{
    "criteria": ["dimension": "example", "value": "foo"],
    "facets": {
        "example": {}
    }
}

Used to produce:

{
    "facets": {
        "example": {}
    }
}

Now produces:

{
    "facets": {
        "example": {
            "navIds": ["foo"],
            "data": {
                "foo": {"count": 0, "selected": true}
            }
        }
    }
}

ENGINE-999 Different default sort for facets API

The default sort order of facets requests against keyword dimenions has changed from declaredAsc to idAsc.

Additionally, the previous behavior (for all dimensions) had hidden secondary and tertiary sort options. This is no longer the case and you can completely specify your sort order using an array of options.

See the other summary of ENGINE-999 for more detail.

ENGINE-1001 Drops availableSize from search response

The search response no longer contains the availableSize property. This is a relic from a sharding feature that was removed a few years ago. If you were reyling on this value, use totalSize instead.

Before:

{
  "itemIds": [],
  "exactMatches": [],
  "relevanceValues": [],
  "currentPageSize": 0,
  "startIndex": 0,
  "exactSize": 5,
  "totalSize": 10,
  "availableSize": 10,
  "datasetSize": 100
}

After:

{
  "itemIds": [],
  "exactMatches": [],
  "relevanceValues": [],
  "currentPageSize": 0,
  "startIndex": 0,
  "exactSize": 5,
  "totalSize": 10,
  "datasetSize": 100
}

ENGINE-1005 Removes simple profiler

Removes support for a poor man’s profiler. This was never documented and used in internal toolchains only. Was exposed via the X-Profile HTTP header.

ENGINE-1010 Removes query function override

Removes support for a non-documented function override of nested queries.

ENGINE-1011 minRelevance and maxRelevance now disable facet isolation

Facet isolation did not function correctly when combined with minRelevance or maxRelevance top-level overrides. It is now disabled if they are present.

Facet isolation will be disabled with this query, you’ll get uninsolated counts instead:

{
  "criteria": [
      {
          "dimension": "example",
          "value": "example"
      }
  ],
  "facets": {
    "example": {}
  },
  "minRelevance": 0.5
}

ENGINE-1012 Facet isolation now constrained to top level AND criteria only

Facet isolation did not function correctly for nested OR queries. It is now constrained to work only for top-level AND queries.

A new isolatedId option has been added that can be used in top-level criteria.

See the other summary of ENGINE-1012 for more detail.

Bug Fixes

ENGINE-987 Search over a groupBy dimension now respects the cull option

If you perform a direct search over a groupBy dimension it will now respect the cull option instead of always culling.

This is an obscure edge case.

<dimensions>
    <dimension id="example" type="groupBy"/>
</dimensions>
{
    "criteria": [
        {
            "dimension": "example",
            "cull": false
        }
    ]
}

ENGINE-988 Fixes JSON serialization of Infinity, -Infinity and NaN

JSON cannot represent Infinity, -Infinity or NaN directly, instead of generating invalid JSON the engine now converts these values to string form.

Before:

[Infinity, -Infinity, NaN, 1, 2, 3]

After:

["Infinity", "-Infinity", "NaN", 1, 2, 3]

ENGINE-1000 Fixes counts in the Children column on the dimensions tab

The counts in the Children column of the dimensions tab in the admin interface did not include all nested children. This has been corrected.

ENGINE-985, ENGINE-909 Query tab breaks with some queries

A query that returned itemIds but not exactMatches or relevanceValues no longer breaks the Query tab of the admin interface.

Example query:

{
    "items": ["example"],
    "properties": []
}