Manipulate and analyze big data with the full expression power of Rax - Relational Algebra with eXtensions.

a year ago

Rax 1.2 Released

Rax 1.2 with shiny new features is now available! Get your fee Rax for SQLite copy to try it out!

Regular Expressions

Rax supports now Perl-compatible regular expressions to simply matching text fields, for example URLs. Rax has two new match operators: =~ (match) and !~ (not match). For example, to select all web clicks on one of the Veronica websites, use the following expressions:

select [.Url =~ '.*https?:\/\/\w+\.veronica[\w\.]+.*'] WebClicks

When processing regular expressions, Rax takes advantage of the computing power of the underlying SQL backend. If the backed supports any sort of regex, Rax will parse the regular expression and translate it into a form understood by the backends. For lightweight backends, such as SQLite, Rax will process regular expressions internally.

New Temporal Operators

We added a few temporal operators that are particularly useful for processing time series:

@|- (bridge) computes all gaps between intervals in a set. For example: @|-[.PathID] ConversionPaths will compute gaps between intervals on each conversion paths.

@<:@ is similar to @&@ (temporal AND) but works on sets of timestamps on the left side, instead of sets of intervals. Example: Conversions @<:@ {[ConversionPeriod]} will return all Conversions falling withing the ConversionPeriod.

<< and >> (time-shift) on intervals - shift an interval by a given absolute duration. For example (|)[(@)"2012-01-01T12:00:00/P2D"] >> (^)"P1W" will shift the interval forward resulting in (|)[(@)"2012-01-08T12:00:00/P2D"].

:\ and :/ (floor and ceiling) operators on timestamps. This allows a point in time to be moved to the beginning or end of the enclosing week or day (or another absolute duration). Example: (@)"now" :\ (^)"P1W" will find ISO beginning of the current week.

<: and :> (element-of and contains) operators on time. These operators can be used to determine if a time @ value is inside a given interval | value. Example: myBirthday <: (|)[(@)"1980-01-01",(@)"1989-12-31T23:59:59"] will tell you if you were born in the nineties.

JSON support

Rax can now produce output in JSON format. For example `json ConversionPaths will output 'ConversionPaths' in a JSON format.

String-concatenation aggregate

A new string-concatenation aggregate has been added to fold. Example: fold [/cat(.#1||"/")] {"aap", "noot", "mies"} will return "aap/noot/mies".