It is possible to address multiple cells with a single rule. This is done by doing a so-called multi-cell reference. To show this I will introduce a new demonstration table with a composite primary key to be able to work with multiple dimensions. The table holds monthly sales numbers of two books in 2005 and 2006 and is created like this:

The book called “Bosatlas” has a record for every month, but only in the Netherlands. “The Da Vinci Code” has been sold in three countries, but not for every month. The columns book, month and country make up the primary key for this table. To show the months a little better, I change the date format mask like this:

SQL> alter session set nls_date_format = 'fmmonth yyyy'
2 /

The example below shows how all sales of “Bosatlas” after june 2006 are doubled:

Measure amount now has two dimensions. So we have to provide a book as well as a month, to refer to a cell in the model. With one rule six cells are being modified, due to using the expression month > date ‘2006-06-01’ on the left side of the rule. At the right side the function cv is used to refer to the corresponding value at the left side, where cv stands for current value. Instead of cv(month), I also could have used cv(); because there is only one multi-cell reference present. Even if more multi-cell references are used, cv() may be used, provided it is clear which dimension is being referred to. If this is not clear, you’ll get an ORA-32611:

FOUT in regel N:
.ORA-32611: onjuist gebruik van operator MODEL CV

When you want to refer to all dimension values, instead of a subset of the values, you use the word ANY:

In this rule the word ANY is used to refer to all books that have sales records for august 2005. Multi-cell references are also possible with a FOR keyword, for example: amount[‘Bosatlas’,for month from date ‘2005-03-01’ to date ‘2005-08-01’ increment 1], and with a BETWEEN, for example: aantal[‘Bosatlas’],month between date ‘2005-03-01’ and date ‘2005-08-01’], and with all other comparison operators.

Reference models

Reference models are sub models within a model. The main model is able to use all values provided in the reference model, but the values from the reference model are not accessible to the outer select list. We are dealing here with auxiliary data that is read only for the main model. To illustrate this I create an extra table storing prices of the two books:

So the reference model also contains dimensions and measures, but it cannot contain a partition clause nor its own rules. You also may have noticed the fact that both the main model and the reference model are now named: the reference model is called prices, prefixed by the keyword REFERENCE, and the main model is called result, prefixed by the keyword MAIN. The measure price from the reference model is being used inside the rule of the main model by the phrase price[cv(book)]. Also new here is the creation of an extra measure called turnover, initially filled with 0 for all existing cells.

Iteration

Iteration offers the opportunity to repeat the same rule multiple times on the same model. In the example below the number of sales of The Da Vinci Code is taken to the fourth power, without using the POWER function, just to show how iterations work. To do so we have to multiply the number of sales three times with the original number of sales, like this:

Directly after the RULES keyword, we place the keyword ITERATE, followed by the number of iterations. Measure a1 contains all original and unchanged number of sales. Measure a2 initially contains the numbers 5, 8, 3, 2 and 1; the same as a1. During each of the three iterations these numbers are multiplied with the numbers of measure a1. So after 1 iteration, a2 contains 25, 64, 9, 4 and 1, and after the second iteration 125, 512, 27, 8 and 1. And finally the numbers are as can be seen in the result set above.

Very handy in working with iterations is the keyword UNTIL followed by a boolean expression to be able skip unnecessary iterations. The word ITERATION_NUMBER can be used in the rule which holds the number of iterations. But be warned: it starts with 0. So in the example above, with iterate (3), it increases from 0 until 2.

To show how UNTIL and ITERATION_NUMBER work, I came up with an example where predictions of future sales are generated according to the rule that they will drop with 75% each year. So each year we will only sell 25% of the books as the same month previous year, where any fractions are truncated. The statement will have to generate sales records until there isn’t a single book sold in an entire year. This query will do that for you:

Now this is a lot more complex than what we have seen before. As you can see by the where clause, we’ll start with the 12 rows for the book “Bosatlas” in 2006. The month column is split into two dimensions: the year and the month number. This is done to be able to make 12 rows for an entire year in one shot. To prevent making a year with only 0 sales for all months, I created an auxiliary measure called max_monthly_amount. This measure is populated by an analytic function with the syntax max(amount)[2007+iteration_number, month_number between 1 and 12] and gives the maximum sales number for the current year. This measure is used in the UNTIL clause. The iterations should stop when this highest number is below 4. For if there is a sales amount of 4 or higher, this will lead to a non zero number in the next year. With the current data set, out of the specified 100 iterations only two iterations are executed leading to this result set:

In this query the UPSERT ALL mode is used. It is one of the three possible modes: UPDATE, UPSERT and UPSERT ALL, where UPSERT is the default and this mode was used in all previous examples. In UPDATE mode it is not possible to create new rows as can be done with UPSERT. The difference between UPSERT and UPSERT ALL is that the latter creates new cells if the left hand side of the rule contains non-positional references. For example: if the left hand side of the rule contains the ANY keyword, in UPSERT mode it would look at all existing cells and find all matches. Using the UPSERT ALL mode, it would create new cells, one for each match of the non-positional dimension values.

More functions and features are available with the model clause, which really are worth the study, but I will only mention them here briefly. First of all there are the functions PRESENTV and PRESENTNNV, the predicate IS PRESENT and the keywords KEEP NAV and IGNORE NAV. In short you can distinguish between missing cells and existing cells with NULL values with these. Then there is the PREVIOUS function to be able to refer to the previous iteration value inside the UNTIL clause during iterations. And finally, I mention the clauses AUTOMATIC ORDER and SEQUENTIAL ORDER, which gives the possibility to Oracle to decide for itself in which order the rules should be evaluated. All clauses and functions mentioned in this paragraph are quite clearly explained in the documentation.

In the last part I will try to explain what practical use the model clause can have. I will show some solutions for previously unsolvable questions and for common problems that can be solved more efficiently than with previously known techniques. Frequent visitors of the OTN Forums will undoubtedly know which problems I am hinting at here. I will try to get this last part published in a magazine first. If it succeeds, I will point to the online version of course. If it does not succeed, then I will post the third part here. So I’ll have to ask for your patience on this one.

Wednesday, October 24, 2007

A quick intermezzo before publishing part two of the SQL model clause tutorial.

Two and a half months after the Oracle11g Linux download became available, Oracle made the Windows version available two days ago. Last night I downloaded it and this evening I finished the install and created my good old SQL*Plus shortcut. I was happy to see the install did not impose any problems:

With the SQL model clause you build one or more matrixes with a variable number of dimensions. This is called the model. The model uses a subset of the available columns from your FROM clause. It contains at least one dimension, at least one measure and optionally one or more partitions. You can think of a model as a spreadsheet file containing separate worksheets for each calculated value (measures). A worksheet has a X- and an Y-axis (two dimensions) and you can imagine having your worksheets split up in several identical areas, each for a different country or department (partition).

The next figure shows a model of the well known EMP table, where deptno is a partition, empno a dimension and sal and comm are two measures:

Once your model is setup, you define rules that modify your measure values. These rules are the core of the model clause. With few rules you are able to make complex calculations on your data and you are able to create new rows as well. The measure columns are now arrays that are indexed by the dimension columns, where the rules are applied to all partitions of this array. After all rules are applied, the model is converted back to traditional rows.

My experience with the model clause wire diagram in the Oracle documentation is that it is quite complex and it tends to scare off people. That’s not what I’d like to achieve here, so I’ll use a different approach using lots of small examples using the EMP and DEPT table, starting with very simple ones, and gradually expanding on that. At the end of this piece, you’ll find a script you can download and run on your own database.

Here we have two measures, ename and sal, and one dimension being empno. The combination of all partitions and dimension columns have to be unique. This is checked at runtime, and if violated, you’ll get an ORA-32638. Because of the two measures, two one dimensional arrays are internally created, which are index by empno. At line 9 you see the keyword “RULES”, but without contents yet. This is the place where the rules will be defined doing the calculations and the creations of new rows. The keyword “RULES” is optional, but for sake of clarity I will always write it down. When done modeling, all partitions, dimensions and measures are converted back to columns in traditional rows, which means that you can only have columns in your select list that appear in your model. If, for instance, I did not include the column ename as a measure, I would get this error message

The rule at line number 10 shows how measure ename is expanded with dimension 7777 and gets the value ‘VAN WIJK’. If table EMP would have contained an empno 7777 already, then the existing ename value for empno would have been overwritten by this rule. But 7777 does not exist in EMP, so now a new cell has been created, which shows itself as a new row in the result set. Please note that the row is not being inserted into the table, but only to the result set of the query. By adding a second rule you can fill the sal column as well:

All calculations are being performed on each partition. We can see this happening when we drop the filter predicate “deptno = 10” and show the deptno column instead. Now the question becomes whether to include the deptno column as a partition or as a measure. First, let’s explore what happens if we define deptno as a measure:

A clear difference. In table EMP only the deptno values 10, 20 and 30 occur and therefore this model effectively has three partitions. The rule is applied to all three partitions, thus resulting in the addition of three new rows.

So far the examples will probably not have persuaded you to use the model clause. A lot of extra lines of SQL code were introduced in order to get something done which is quite simple: with the creative use the UNION ALL set operator and table DUAL we could have achieved the same results. The intention of this first part is only to show the basics. In the next part I will show multi cell references, references models and iterations. And this is where things become more interesting.