How to use Data on Demand with Markings in Spotfire

The Data on Demand feature in Spotfire lets you define conditions that must be met before data is loaded from the source into Spotfire. Instead of loading all data at once, data will only be loaded for the conditions that you specify.

In this video you’ll learn how to load Data on Demand from a large database by selecting, or marking, data points on a map chart, as shown in the screenshot below.

[ninja-popup ID=1396] Click Here to download the Spotfire file shown in the screenshots[/ninja-popup]

Note: The data tables will be empty when you download this file (because I’m connected to my SQL Server), but you can still see how I’ve configured the Data on Demand & Visualizations. Click “OK” when the Missing Data prompt pops up.

Data on Demand is ideal when you’re connecting to a large data source that would be time consuming to load, or reduce the performance of your Spotfire project.

How much Data can Spotfire Handle?
It’s hard to say exactly, because the speed and performance of Spotfire is largely dependent the RAM and processing speed of your computer. 10+ million rows may not be a big deal for a fast computer with 8+ GB of memory. But if your computer is a little older, has only a few GB of RAM, and you’re running a lot of applications at one time, then you’ll definitely notice performance issues with a just few million of rows.

How to: Load Data on Demand with Marking
This example uses two data tables containing natural gas production data for the entire state of Oklahoma. Both tables have a common column: API Number, which uniquely identifies each well record.

Table 1 is a summary table with 1 row for every well record – commonly called a “one-line table” or “well header table”. This table has about 17,000 rows.

Table 2 contains detailed monthly production information for each well record in Table 1. Each well has 400-500 records in Table 2, and therefore Table 2 has many more rows – over 8 million.

Because Table 2 has so much data, it would take a long time to load and might slow down my computer. Instead of loading all of the data at once, we can use Data on Demand to only load part of the dataset, based on conditions that we specify.

In this case, the condition is Marked rows in Table 1 – detailed monthly production data will load for any well that’s marked on the map chart.

Notes: Data on Demand is different than a Details on Demand or a Details Visualization. With Details on Demand and Details Visualizations, the data shown in a visualization is limited by Marking, but Spotfire is still holding all of the data in-memory. With Data on Demand, the in-memory data table is changing based on the conditions specified in the Details on Demand settings.

Check out the video (Also on YouTube) to learn how to use Data on Demand from a large database by selecting, or marking, data points on a map chart.

[ninja-popup ID=1396] Click Here to download the Spotfire file shown in the screenshots[/ninja-popup]

Note: The data tables will be empty when you download this file (because I’m connected to my SQL Server), but you can still see how I’ve configured the Data on Demand & Visualizations. Click “OK” when the Missing Data prompt pops up.

Questions / Comments
If you have any questions or comments, or if there’s something you’d like featured on this blog, feel free to email me at kyle@datafuel.co or leave a comment below.