Pentaho Community Forums - BI Platform Developerhttps://forums.pentaho.com/
Dev and build topics related to integrating Pentaho applications, creating platform components, Dashboards, Design Studio, Metadata, commons projects...enThu, 22 Feb 2018 04:48:53 GMTvBulletin60https://forums.pentaho.com/images/misc/rss.pngPentaho Community Forums - BI Platform Developerhttps://forums.pentaho.com/
What would it take to get a true step blocker in a transformation?https://forums.pentaho.com/threads/230887-What-would-it-take-to-get-a-true-step-blocker-in-a-transformation?goto=newpost
Sat, 17 Feb 2018 17:23:16 GMTI run into this all the time. I want to read some files. Do magic with the data in them . Then when all that is done. Do something with the...I run into this all the time. I want to read some files. Do magic with the data in them . Then when all that is done. Do something with the files/move/delete/rename. The trouble is spoon does not have any tools that work inside a transformation that can easily do this. "Blocking step" and "Block this step until steps finish" don't really block anything. It just seems silly to me, that to do such a simple thing I have to create a job and 2 transformations. I understand the parallel nature of the tool BUT there really should be a way to be able to say ..hey do all these steps until complete...then do this series of steps i.e. a blocking step. Or, failing that possibly a way to 'job' inside of a transform from one portion to another? I mean all of it, not just a row at a time blocking.

And this isn't just files either. I have many times run into a situation where i need one thing to finish before starting another and for various reasons don't want to do a separate transform/job or lose speed by single treading.

As an aside, i have had 'block step until all finish' work for what I'm taking about but that was 2 versions ago at least and i am really not sure how the heck i did get it to work, possibly single threading the whole transform ?

Attached is a simple non working transform of what I mean, not that this hasn't been brought up before:)