A software library of stochastic streaming algorithms, a.k.a. sketches.
Repo apache/datasketches-java, read from GitHub 16 min ago. apache has not joined Take My Repo.
This is a minor release, source and binary compatible with 9.0.0, and like 9.0.0 it requires Java 25. It adds a new filter and several new operations, makes deserialization of every sketch family much stricter, and brings the serialized images of several families into byte-for-byte agreement with C++.
FrequentLongsSketch.getInstance(String);ArrayOfNumbersSerDe and the string SerDe.XorFilter and XorFilterBuilder classes in the filters package. An Xor filter is an immutable approximate-membership filter built from a fixed set of items. It stores 8-bit or 16-bit fingerprints and uses about 1.23 times the fingerprint width in bits per item.difference (A NOT B) is new, and invert() is deprecated.HllUnion.update(HllUnion) merges a union without first copying it into a sketch.CpcUnion.update(MemorySegment) merges a serialized sketch without heapifying it.UpdatableThetaSketch.compactTrimmed() compacts and reduces to k in one pass.JaccardSimilarity has new seed overloads, so sketches built with a custom seed can be compared.JaccardSimilarity now accept custom update seeds and check seed hashes, as Theta does.JaccardSimilarity.exactlyEqual no longer throws for small sketches. The Theta and Tuple versions now work the same way.getRSE() is fixed.getMinItem() and getMaxItem(). Each sketch's javadoc names the exception it throws when empty.totalRWeight on heapify and rejects infinite or NaN weights, as C++ does.CpcWrapper no longer throws on empty sketches.ArrayOfStringsSummary: fixed issue #699.BinomialBoundsN no longer divides by zero.Util.numDigits returns the correct count for 0, 1 and 10^18.QuickSelect picks its pivot as the median of three, so ordered input no longer takes quadratic time.BloomFilter.invert() is deprecated./tools/FindBugsExcludeFilter.xml, which no longer exists, and its Checkstyle note may be out of date. Both will be corrected in the next release.Contributors to this release:
Special thanks to @tisonkun, who took over the release from RC1, built release candidate RC2, and ran its vote.
Thank you!
@leerho
This release is a major release where we took the opportunity to do some significant refactoring that will constitute incompatible changes from previous releases. Any incompatibility with prior releases is always an inconvenience to users who wish to just upgrade to the latest release and run. However, some of the code in this library was written in 2013 and meanwhile the Java language has evolved enormously since then. We chose to use this major release as the opportunity to modernize some of the code to achieve the following goals:
None of these changes have affected the binary compatibility of the serialized versions of these sketches. The sketches in this library can still interpret serialized versions of the same sketch from earlier release versions of this library and from other language versions of this library.
Full Changelog: 6.0.0...8.0.0
This release:
Full Changelog: 6.1.1...6.2.0
This release runs on Java 17.
This release leverages the new Java Foreign Function & Memory (FFM) API capability (JEP 412), which is in incubation in Java 17. This means that Java 17 is required to compile this code and at runtime. The location of the FFM code was moved from jdk.incubator.foreign in Java 17 to its position for preview in Java 21 at java.base/java.lang.foreign. As a result, code compiled with Java 17 FFM will not run with Java 21, the next LTS. Java does not provide backward compatibility for older incubation code.
There are some changes in the DataSketches-Java API, due to the switch to leveraging the new FFM code, especially for methods that must deal with off-heap memory. These changes are a result of the DataSketches-Java-7.0.0 dependency on DataSketches-Memory-4.1.0, which manages all the direct memory.
DataSketches-Java 6.1.1 using DataSketches-Memory 3.0.2 (Java 8, 11)
try (WritableMemory wmem = WritableMemory.allocateDirect(4096)) { ... } //wmem is closed
where WritableMemory is a class of DataSketches-Memory.
In DataSketches-Java 7.0.0 using DataSketches-Memory 4.1.0 (Java 17)
try (ResourceScope scope = (wmem = WritableMemory.allocateDirect(4096).scope()) { ... } //wmem is closed
where ResourceScope is an FFM class.
Full Changelog: 6.0.0...7.0.0