LogicBlox 4.3.7

Release Date: March 1st 2016

What's New

Database

The performance of frame rules has significantly improved for
larger incremental data loads. There was a problem with the parallel
evaluation of frame rules, which would be used for larger deltas.
As part of this tuning we have also lowered the delta size
limit for the usage of parallel evaluation.
This will mean better core utilization for some applications.

Note

Please refer to the
Reference Manual
for an overview of all the integer bit manipulation functions.

modeler-js User Interface Components

Vega Charting Plugin:
The new Vega Charting plugin allows developers to configure
a sheet containing a chart. Chart configuration must be specified
using the Vega visualization language.
Data used to populate the chart can be specified using the sheet configuration
language already supported for modeler-js sheets.
Charts can be configured to interact with other visualization components,
such as grids. When linked, changing the data in grids will
cause chart displays to change and reflect the new data, as well.

Example 31.

The figure below illustrates how a Pareto chart can be displayed
on the same canvas as a grid. The grid and
the chart interact with each other, allowing
the user to immediately see changes in the
grid being reflected in the chart.

Locking:
This feature allows an end user to lock certain values,
such that they are not affected by the spreading that may
result from other edits. A lock applies to any edit performed
by the user, regardless of on which view the edit is performed.
Locks persists throughout the user’s browser session.
Locks will be reset upon closing of the browser tab.

Individual resizing of rows and columns:
Resizing can be now be done individually for the innermost headers
(i.e. those next to the data cells). The outer headers are resized based
on the inner headers. For cross producted headers
all instances of the resized header will be resized. To resize
all columns, users can hold down the <CTRL> key while resizing.

Developers can also configure the individual column/row sizes
via the pivot configuration.

Example 32.

The example below allows to apply custom height to
subclass1 and width for the Sales column.
The members object is used to define the
innermost headers sizes. The rollups object
is used to define sizes for aggregated heading:

Logout notification over multiple tabs:
The logout notification now also works over multiple tabs. The new configuration option
ignoreBroadcastingTime allows developers to configure
after how many seconds of idle time the processing of broadcasting messages should stop.

The notification around user inactivity and automatic logout can now be configured as follows:

Level mapping measures:
The measure model configuration utility distributed with
modeler-js now defines all level mappings to be measures.
Every level, consequently, is also declared to be an intersection.

Example 33.

For instance, if a Product hierarchy contains levels Product:Sku,
which maps to Product:Subclass, then a measure,
Product:Sku:subclass, is automatically configured,
backed by the predicate Product:Sku:subclass.

For level Product:Sku the corresponding intersection,
Product:Sku, is declared and can be used to define measures.

Improved Dimension Browser:
The new Dimension Browser is a more compact representation
of all the levels in the schema.

Warmups:
The new lbconfig.modeler python module can be used
for generating warmups for both measure rules and views.

Example 34.

To use this new module, the following needs to be added
to the config.py script:

from lbconfig import modeler
modeler.configure()

This will create targets to copy over all assets, JavaScript libraries as well as
scripts from the platform build installation. For warmups it will
create the following make targets:

rule-queries: generates queries for rule warmup

view-queries: generates queries for view warmup

warmup-rules: submits the generated queries to warmup rules

Note

Application developers are advised to using
this module instead of application specific scripts,
to be able to automatically benefit from future enhancements.

Services Framework

lb-web allows services to declare an inactive block
to be executed in the same transaction as a request.
It is now possible to declare multiple such inactive blocks,
which allows handling code to be better modularized and reused.

The raw file-mode is now supported for TDX imports.
It was previously only implemented for TDX export.
The raw file-mode does no escaping or unquoting whatsoever,
but does not support the delimiter in fields.

The lb-web server now prints a clear message when startup is done
and it is ready to serve requests.

New json_printing option is now available for protobuf
services to configure whether strict or
loose mode should be used.

It is now possible to configure the force_authn
flag on the SAML request that lb-web sends to IDPs when a user
attempts to log into a SAML realm. This forces the IDP to re-request
authentication even if the user has an ongoing session with the IDP.

lb-web services normally obtain the username for a service request
from request parameters that are imported into the database
using a protobuf message. In active logic, this populates the
pulse predicate lb:web:auth:username. We earlier introduced
an optional setting enable_pulse_auth on services,
which directly populates this predicate at stage initial.
This makes the username also easily accessible to inactive blocks.
This feature is now also available for TDX services
(earlier it only was supported by protobuf services).

New features to control lb web-server logs:

A new log_file config property can be used
to determine where to store the logs.

The new systemd property (also settable via the IN_SYSTEMD
environment variable) determines 1) whether logs will contain timestamps
(if in systemd timestamps are redundant so they are omitted)
and 2) if explicitly set to false, then even when
running as a systemd unit, the specified log_file will be used.
This can be convenient for experiments that produce very large logs.

The log_elapsed flag makes the logs contain
an additional column with the number of milliseconds since startup.

UTC is now the default timezone for timestamps
in lb-web server and client, including server access logs.
There's a special host timezone that can be used
to select the default timezone of the running host.

Measure Service

It is now possible to use a CubiQL expression as the input
to an update rather than a LiteralTypedRelation.

The new command lb workflow purge allows the removal of workflow instances
from the workspace.

Example 35.

You can use the following command to remove all workflow
instances of the daily workflow that are
older than 7 days:

$ lb workflow purge --name daily --older-than 7days

Example of removing a workflow instance with a specific ID:

$ lb workflow purge --root 10000000063

It is also possible to remove all workflow instances at once:

$ lb workflow purge --all

All duration values in lb workflow tasks can now be specified as strings
that contain the value plus a unit.
For backwards compatibility, tasks still accept integers,
which will be interpreted in the same unit as before
(seconds for most tasks; milliseconds for lb.tdx.* tasks).

Example 36.

The expected syntax is an integer (specifying the length) followed
by an optional resolution specification. The supported values
are as follows:

Note

It is now possible to list workflow instances using
lb workflow status --list-roots.
This will use the status service to lookup the roots
of workflow instances and list them.
It can be used together with the --name option
to select instances of a particular workflow.

lb workflow has now command line completion
like all other lb commands.

Tip

It is now possible to pass the --failed-restartables
option to the lb workflow status command
to get the status for the restartables that have a failed descendant.

The new workflow lb.wb.SetUsers can be
used to add and remove users that have access to workbooks
in one transaction. Previously, these activities had to
be performed via two separate tasks.

Workbook Framework

Cross-branch Commit/Refresh:
This feature supports an alternative method of performing
commit/refresh between workbooks.
Whereas the default method of commit/refresh currently is through
exchanging data between master and workbooks by
exporting/importing CSVs, the cross-branch commit/refresh
executes cross-branch logic: rules that refer to data in one branch
of the workbook, and derive it into another branch.

Cross-branch commit/refresh represent
performance wins, but are only applicable for workbooks
that are co-located on the same instance as their parent.

Developer Tools

The lb export-workspace command now also works
on secondary workspaces.

Various improvements to the profileDiskSpace command:

The --branch option can be used to
show disk usage only for a specific branch of the workspace.

The --versions option has been improved to show
the total number of reachable pages for every branch
(earlier it only showed differences between versions in the listing).

The command can now be used online using lb batch-script.
It does block write transactions.

lb print-rules (introduced in the 4.3.6 release)
now also shows delta rules when applied to EDB predicates.

All command line arguments that accept a log level now have
a better help message with some examples.

Several lb commands that accept a --csv
parameter to create a CSV file now support a
--delimiter parameter to configure the file's delimiter.
This includes lb exec, lb query,
lb execblock, lb workbook list
and list-templates.

lb info now supports the --json option
to print the information in JSON format.

Corrected Issues

The issues listed below have been corrected since the 4.3.6 release.

Resolved an issue introduced in 4.3.6 where some disjunctive
rules would not evaluate correctly.
The latest 4.3.6 patch releases also included this fix.

The lb batch-script command did not correctly
use security tokens and required the
LB_CONNECTBLOX_ENABLE_ADMIN environment variable to be set.
This is not the case anymore.

Resolved an issue where lb export-workspace
would hang if the lb-server was terminated unexpectedly during the export.

lb export-workspace now respects the port
configuration in lb-server.config.

A transaction abort in earlier releases would print an
ERROR line in the log, which complicated
monitoring for errors. This error line has been removed.

Resolved an issue with executing write transactions on
ws vs. ws@master,
which are both the same branch.
The transactions would occasionally fail with
an error that a concurrent write is detected.

The composition of configuration files in lb-web-server has been fixed.
Previously, configuration sections in lb_deployment
would only be able to send new options to handlers,
but never change an existing one.
Also, dynamically loaded jars and config files would
completely overwrite anything from lb_deployment.
This has been fixed such that lb_deployment files have precedence
over jar config files, and dynamically loaded config
files have precedence over lb_deployment.

modeler-js User Interface Components:

Resolved an issue that could cause a pivot grid to be rendered
empty for incorrectly selected slices on initial render
when the slice was set via a measure.

The scroll position is now correctly maintained when the
user switches between views.

Resolved an issue that could cause an incorrect sorting behavior
when editing a previously sorted column.

Resolved an issue that caused pending calculations not
to be marked as such when switching between views.

NOTE: this script will set all the necessary
environment variables. You might want to add this command to your
.bashrc.

Upgrade Information

Java 8:
LogicBlox requires Java 8 starting with the 4.3.7 release.
All Java code is compiled for Java 8,
so on older Java versions the components will not run (e.g. the compiler server).
If accidentally deployed on an older Java version,
then an error "Unsupported major.minor version 52.0"
should be reported by the JVM.

Workflow:

The lb-workflow command
has been deprecated in favour of lb workflow.

All duration values in lb workflow tasks should now
be specified as strings containing the value and their unit
(e.g., "300ms", "5s", "2h", etc). The use of integers
is discouraged and the compiler
and driver will raise deprecation warnings.

All uses of QueueTransport and SQS support
for transports has been removed: Only TCP is now available.

Deprecated Measure Service Protocol Changes:
To facilitate work on removing deprecated features from projects,
we re-announce in this release previously-announced deprecations,
accompanied with migration advice. We aim to remove them in the 4.3.9 release.

To check for absence of these deprecated features in a project,
set fatal_warnings=true in the project's
lb-measure-service.config.

ad hoc fields for textual CubiQL are deprecated (since 4.3.3):
Most string fields that have been variously available
to hold CubiQL expressions in textual form are now
deprecated in favor of replacement fields holding a MeasureExpr.

For example, expr_str in ExprSubst message is deprecated in favor
of the expr field and measure_str in
QueryRequest in favor of measure.

Since MeasureExpr now has the str string field for textual CubiQL,
the transition amounts to moving the content of the deprecated field
to the replacement field, after wrapping it as a MeasureExpr.

map_pred is deprecated in Edge
and Path (since 4.3.3):
The string field map_pred is deprecated in favor of the Predicate
field pred, in the Dimension.Edge and
Dimension.Hierarchy.Path messages.

Projects that use the config library do not need adjustments to deal with this
deprecation (the config predicates dimension_hasEdge
and levelMap have been adjusted to produce the pred field).
Clients that create protobuf directly should change, e.g.,

pred field in ExprSubst
is deprecated (since 4.3.3):
The field pred holding a Predicate within an ExprSubst
entry in a model has been deprecated.
It was used to specify a predicate in the workspace
to be used as a replacement of a CubiQL expression,
instead of any optimization that measure service would compute for it.
The preferred alternative in protobuf is the replacement_expr
field that holds a MeasureExpr, with an anonymous
Metric defined for the predicate.
In the configuration library, the alternative would
also require defining a named metric in the model based on the predicate.

metric field in
UpdateExpr is deprecated (since 4.3.4):
metric field in UpdateExpr is deprecated,
as it is now a special case of the target field
holding an UpdateExpr.Target message.

Example 39.

For example, an UpdateExpr

{"kind": "SPREAD",
"metric": "Sales",
//...
}

should now be written

{"kind": "SPREAD",
"target": {"metric": "Sales"},
//...
}

signature field in
UpdateExpr is deprecated (since 4.3.7):
The field signature in UpdateExpr
is now deprecated as it can be inferred from the source or
input fields of the enclosing UpdateRequest.

source field in UpdateRequest
is deprecated (since 4.3.7):
The field source in UpdateRequest
that holds a LiteralTypedRelation with input data
for the update is now deprecated. It is replaced with the new
field input, holding a MeasureExpr.
It is more general, in allowing the input data to be computable.
The same effect as source can be achieved by using
a ParamExpr as input, coupled with a
RelationBinding in the enclosing Request,
to hold a LiteralTypedExpression with the data.
A LiteralExpr (rather than a ParamExpr)
could be used as well, but in this case the data would be installed as logic.
Unless these data are used more than once, this would likely be undesirable.

Using LiteralTypedRelations as the input for updates
is deprecated:
To eliminate the use of LiteralTypedRelations as the input of updates,
use a ParamExpr as the input expression.