3.11 (September 3 2013)

Improvements

ENGINE-848 Some text queries take a long time to complete

Fixes a bug that caused text queries to sometimes taking a very long time to finish.

ENGINE-852 Unindex item with group by causes array index out of bounds

Fixes a bug in groupBy dimensions that manifested itself when items with no group were deleted from the end of an internal structure.

ENGINE-880 Engine fails to apply some large changesets

Upgrades Derby to obtain a bug fix. Occasionally large changesets would fail to apply with a StackOverflowError. Here is an example of the error in the discovery-0.log.

[20130829 15:36:03,077] [0000000e] [WARNING] [com.t11e.discovery.item.changeset.ItemChangesetProcessor] \
  Failed to apply changeset ebe8ca5245e274b555d0c4ffeb297dd5: java.sql.SQLException: Java exception: ': java.lang.StackOverflowError'.
...
at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
at java.util.Observable.notifyObservers(Observable.java:142)

More information about the Derby issue is available here: https://issues.apache.org/jira/browse/DERBY-5624

ENGINE-821 Duplicate attributes in dimensions file break dimensions tab

Upgrades to a more recent Woodstox XML parsing library to get upstream bug fix for XML attribute validation.

This allows the engine to better detect duplicate attributes in the dimensions file.

More information about the Woodstox issue is available here: http://jira.codehaus.org/browse/WSTX-286

ENGINE-878 Makes JSON (de)serialization code preserve numeric precision

We now internally use BigDecimal and BigInteger when (de)serializing numbers from JSON to avoid precision errors.

ENGINE-847 ENGINE-874 ENGINE-875 ENGINE-876 Upgrades dependent libraries

Upgrades dependent libraries.

library previous current
commons-codec 1.6 1.8
commons-fileupload 1.2.2 1.3
commons-httpclient 3.0.1 3.1
commons-io 1.4 2.4
commons-lang 2.5 2.6
commons-logging 1.1.1 1.1.3
derby 10.8.1.2 10.8.3.0
jackson 1.9.7 2.2.3
jetty 6.1.25 6.1.26
jts trunk-335 1.13
lucene/solr 4.1.0 4.4.0
slf4j 1.5.10 1.7.5
spring-framework 3.1.2 3.2.4

Documentation

ENGINE-868 Remove kind/provider/locator from Data Tool documentation

The kind/provider/locator support in the Data Tool was inadvertently documented. It is an experimental feature that is slated for removal.

ENGINE-869 Update Data Tool documentation with integration how-to

Adds detailed documentation describing how to perform an integration using the Discovery Data Tool.

ENGINE-870 Update sortBy doc to indicate that you cannot sort on tree dimensions

Improves the documentation for sortBy to more clearly state which kinds of dimensions may be used.

Compatibility Changes

ENGINE-854 Keyword dimensions ignore notValue when maxBuckets=0

Keyword dimensions used to incorrectly ignore notValue in criteria when the maxBuckets attribute was set to 0.

ENGINE-855 Keyword dimensions ignore cull=true when nullExactMatch or nullRelevance are in use

Keyword dimensions used to incorrectly ignore "cull": true in criteria when either nullExactMatch or nullRelevance were specified.

Previous behavior was to leave cull candidates in the results as exact matches. Now they are correctly tagged as fuzzy matches with a score of 0.

ENGINE-856 Keyword dimensions should respect notId and notValue when cull=false

Keyword dimensions used to incorrectly ignore notId and notValue in criteria when cull was set to false.

ENGINE-857 Relevance and exact match overrides should be consistently applied across dimensions

All dimensions now consistently apply the following criteria options:

  • nullRelevance
  • nullExactMatch
  • exactMatch
  • exactRelevance
  • fuzzyRelevance

ENGINE-867 Items outside of cull distance (min, max, cullDistance) should have a score of 0 when cull=false

The integer, double, long, time, and geoloc dimensions now consistently generate a score of 0 for items that are outside of the cullDistance (or min, max) when cull is set to false.

ENGINE-879 Prevents keyword dimensions from supporting startsWith for notValues

Keyword dimensions no longer try and apply prefix search to notValues.

ENGINE-871 Add support for escaping / in dimension key attributes

Fixes bug introduce in release 2.6.19 when we added support for indexing inside of nested objects using / as a path separator.Previously there was no way to index a property whose name contained a /.

To index a nested object:

{
  "title": "I want to go bowling",
  "creator": {
    "name": "Fred Flintstone"
}
<dimension id="example" type="text" key="creator/name"/>

To index a property whose name contains /:

{
  "title": "I want to go bowling",
  "creator/name": "Fred Flintstone"
}
<dimension id="example" type="text" key="creator//name"/>

Removed features

ENGINE-844 Remove textTokens (obsoleted by highlighter)

Removes obsolete and undocumented feature that exposed the processed tokens used for matching. Use the highlighting API instead.

ENGINE-850 Remove multi-threaded query option

Removes experimental support for multi-threaded queries, originally introduced in release 2.8.7. It was never proven to improve the performance of queries.

ENGINE-877 Removes Voronoi Tessellation support

Removes experimental Voronoi Tesselation support, originally introduced in release 3.3.0. It added an unacceptable cost to the changeset application process.