Details

Description

There are have been several requests from users to have APIs to create and delete views from the Java (and other) SDKs.
This is a MUST have for some customers and needs to be a part of the Client SDK we release prior to Beta.

Activity

I'm in agreement. One question is do we also want this API to be involved in the view materialization process. I think the answer is probably yes, but this can be complicated since it can potentially take a very long time with little feedback.

Matt Ingenthron
added a comment - 08/Jun/12 1:42 AM I'm in agreement. One question is do we also want this API to be involved in the view materialization process. I think the answer is probably yes, but this can be complicated since it can potentially take a very long time with little feedback.

Dipti Borkar
added a comment - 08/Jun/12 1:57 AM When you say View materialization process? what exactly do you mean? given that index building happens at query time, I don't think so.
However, there are other APIs that may be more interesting to have. explicitly triggering compaction, configuring min time duration / min # of mutations to trigger index building.

What I mean by the view materialization process is that we, in the web console, have this flow where we advise people to take their "dev_aview" and run a full cluster dataset query on it before publishing it to production. This way, when it's published as "aview", the view will have been mostly materialized. It's a nice thing for updating a view.

The reason this is complicated is that it's an HTTP request that may go for a very long, long time. I can think of a way around it (poll it with a timeout until it's fast), but it's suboptimal unless we start querying view materialization progress.

Matt Ingenthron
added a comment - 24/Jul/12 1:38 AM What I mean by the view materialization process is that we, in the web console, have this flow where we advise people to take their "dev_aview" and run a full cluster dataset query on it before publishing it to production. This way, when it's published as "aview", the view will have been mostly materialized. It's a nice thing for updating a view.
The reason this is complicated is that it's an HTTP request that may go for a very long, long time. I can think of a way around it (poll it with a timeout until it's fast), but it's suboptimal unless we start querying view materialization progress.

Matt, what if we wait for it in a separate thread and return with a future when it's done? We just need to make the user aware that it can take a long time, and if they add a view for development that they'd have to publish it separately.

Another idea would be to make the "let's do a full dataset query" optional when the view gets published since someone may want to do that later or from the web-ui.

Michael Nitschinger
added a comment - 26/Aug/12 7:31 AM Matt, what if we wait for it in a separate thread and return with a future when it's done? We just need to make the user aware that it can take a long time, and if they add a view for development that they'd have to publish it separately.
Another idea would be to make the "let's do a full dataset query" optional when the view gets published since someone may want to do that later or from the web-ui.

Michael: that's effectively what we do right now since it's the query against the view that takes a long time. The design document management operations do not. The view requests are not quite done asynchronously, that's true.

The main point I was raising was that the Web Console provides for functionality to execute a view before pushing it in production and workflow to take it from "dev_thing" to "thing" knowing that the view has been materialized. In the SDK, I don't think we'll try to replicate that workflow, but the same functionality is definitely available.

Matt Ingenthron
added a comment - 26/Aug/12 12:59 PM Michael: that's effectively what we do right now since it's the query against the view that takes a long time. The design document management operations do not. The view requests are not quite done asynchronously, that's true.
The main point I was raising was that the Web Console provides for functionality to execute a view before pushing it in production and workflow to take it from "dev_thing" to "thing" knowing that the view has been materialized. In the SDK, I don't think we'll try to replicate that workflow, but the same functionality is definitely available.