LogicBlox 4.0.8

Executive Summary

LogicBlox 4.0.8 includes the following scalability, performance and functional enhancements:

Data loading has been benchmarked to load data up to TPC-H
scale factor 1000, with LogicBlox being 33% faster than
representative competitors on equivalent hardware.

Sorting is now incremental, where the work done to update a
sorted predicate is proportional to the amount of data
changed.

Transaction throughput continues to be tuned, with 4.0.8
showing at 2x throughput improvement through lb-web over 4.0.7.

Tabular Data Exchange is mostly functionally complete with
respect to TDX in LogicBlox 3.x. Th release notes highlight the
details that will be addressed in the next release.

What's New

Services Framework

LogicBlox 4.0.8 includes support for the following
Tabular Data Exchange features:

GET (export), POST (append) and PUT (replace) HTTP methods.

File formats with extensive checking
(including booleans and dates).

Error handling: rows that contain errors in
imports are sent back to the client with
cause columns added to the file.

Optional columns, bound to values of functional predicates.

Export functions (functions, that you can
use to transform values on export),
including constructors and inverse functions.

Auto refmode conversions.

Entity accumulation policies (NONE
and ACCUMULATE).

Retraction of empty optional values on POST.

Known Limitations and Workarounds

TDX support in LogicBlox 4.0.8 is still in beta release, with some limitations:

Auto primitive conversions:
format checking is currently very strict.
E.g., one can only bind an "integer"
column to an "int" value. In the future we will
allow certain conversions, such as binding a
"float" column to an "int" value.

Import functions (functions, that you can
use to transform values on import) are currently not supported.

Import filters: these are predicates bound to a
file to work as a filter for the import,
as opposed to being filled by the import.
The filter flag is currently ignored, so the filter
predicate will be modified by an import.

Ordered columns are currently not supported.

Abort on error: currently it is not possible
to abort a transaction when there is an error
in any of the import rows.

HTTP error codes: currently any error that prevents import
(such as missing headers) causes a status 500 to be
sent to the client. In 3.x, these errors would lead to a
status 4xx.

Performance Improvements

LogicBlox 4.0.8 has been tuned to load data up to TPC-H Scale Factor
1000. Figure 1 demonstrates data
load benchmarks of LogicBlox against a number of competitors on the
same hardware, up to Scale Factor 100. LogicBlox outperforms the
closest competitor by around 33%. We are in the process of
benchmarking competitors on equivalent hardware for scale factor 1000.

The transaction throughput performance of lb-web
has increased by ~2x from LogicBlox 4.0.7, using the TinyBank benchmark.

Figure 1. TPC-H Data Load

Other Improvements

Improvements in predicate sorting (list <<>>):

Added incremental maintenance to predicate
sorting. The cost of updating a sorted predicate is
now proportional to the size of the changes
rather than to the size of the predicate.

list <<>> exposes last (optional).

Example 1.

In addition to the old syntax (which is also supported):

first(y,x), next(y,x,u,v) <- list<<>> b(x,y).

it is now possible to write:

first(y,x), next(y,x,u,v), last(y,z) <- list<<>> b(x,y).

Autonumbered refmodes have been
removed from generated protobuf messages.

Example 2.

A protobuf message named Msg used to
have both an autonumbered refmode and a constructor. As of 4.0.8, only the constructor is available.
To get the auto-generated number for an individual message, you should now use the constructor:

MsgContructor[i]=m -> int(i), Msg(m).

The limit of strings stored in predicates is
increased to ~50% of a page size. In earlier 4.x releases,
the strings stored in predicates could at
most be 255 characters long.

Setting logdir_access in
lb-web-server.config to an
empty value now makes lb-web-server print access
logs to the standard error instead of to a file.

Corrected Issues

The issues listed below have been corrected
since the 4.0.7 release.

Resolved an issue with max
aggregations over negative values.

Removed some unnecessary stack traces
from the lb-compiler.log.

Installation and Upgrade information

Installing LogicBlox 4.0.8 is as simple as following
the steps outlined below: