This forum is now a read-only archive. All commenting, posting, registration services have been turned off. Those needing community support and/or wanting to ask questions should refer to the Tag/Forum map, and to http://spring.io/questions for a curated list of stackoverflow tags that Pivotal engineers, and the community, monitor.

Problem with Spring Batch 2.0 ... ideas?

Jun 25th, 2009, 05:15 PM

So basically we haev a spring batch 1.0 project.

It basically has a job and steps. Now for the steps we actually are having the reader/writers multithreaded. So you actually have 3 read/writes going at a time. Added a bit of syncrhonization as to not clober ourselves.

In addition the same job can be run concurrently multiple times. So to accomplish that i used prototype scope for the jobs.

....> batch 2.0 ....>

using the job namespace i cant set the scope so it SEEMS like its being a singleton? IS that right? I say this because i noticed the reader/writer (in the beforeStep i actually set up the data i am going to use) seem to be sharing data.

So i said ok i'll try out this new "step" scope ... now perhaps i misunderstood its use but now it seems like everytime its accessed in ONE job its a different instance? Is that the expected behaviour?

Just running into a bit of a roadblock and any advice would be appreciated.

The job and step that you set up in the namespace are indeed singletons. That shouldn't be a problem. You need to use step scope for components that are stateful within a step.

One thing that occurs to me: step scope doesn't work if you create your own threads and the first access to the bean is in one of those threads. But without seeing the error or some configuration / code it would be impossible to say if that is the problem you are experiencing.

Also there was a bug that we fixed the other day (so not in 2.0.1) where multithreaded steps would create too many instances of the step scoped beans. Can you try a snapshot and see if that helps? Again it's impossible to tell if this is the same problem.

Comment

The job and step that you set up in the namespace are indeed singletons. That shouldn't be a problem. You need to use step scope for components that are stateful within a step.

One thing that occurs to me: step scope doesn't work if you create your own threads and the first access to the bean is in one of those threads. But without seeing the error or some configuration / code it would be impossible to say if that is the problem you are experiencing.

Also there was a bug that we fixed the other day (so not in 2.0.1) where multithreaded steps would create too many instances of the step scoped beans. Can you try a snapshot and see if that helps? Again it's impossible to tell if this is the same problem.

Comment

Also there was a bug that we fixed the other day (so not in 2.0.1) where multithreaded steps would create too many instances of the step scoped beans. Can you try a snapshot and see if that helps? Again it's impossible to tell if this is the same problem.

Still didn't seem to work with the snapshot from friday.

I basically have a step that is multithreaded, then i use the beforeStep to setup a global List object.

I did an output of "this" and i get different Ids back. It does going into the beforestep only once though. So thats good. But hten the other items just dont work.

Comment

public void beforeStep(StepExecution stepExecution) {
-- this is where i set a list set on the object.

The scope="prototype" is certainly not helping, but it might not be hurting. I would get rid of it asap if I were you. The reader and writer (assuming they are the stateful components) are the ones that you need to thing about the scope for, and scope="step" would be best, but "singleton" should be OK in this case of a proof of concept, since there's only one step and one job. My advice: get it working in an integration test first. Did you do that?

What list (and why)? What's a "list set"? What object? I assume either the item reader or writer is the listener?

(Please use [code][/code] tags to post code and stack traces.)

Comment

The scope="prototype" is certainly not helping, but it might not be hurting. I would get rid of it asap if I were you. The reader and writer (assuming they are the stateful components) are the ones that you need to thing about the scope for, and scope="step" would be best, but "singleton" should be OK in this case of a proof of concept, since there's only one step and one job. My advice: get it working in an integration test first. Did you do that?

What list (and why)? What's a "list set"? What object? I assume either the item reader or writer is the listener?

(Please use [code][/code] tags to post code and stack traces.)

yeah i've gotten rid of the prototype as i said didnt hurt or help. And yeah i am using scope="step" on the reader and writer.

No i havent gotten working with an integration test, mainly because it is integrated into Seam so its hard to run the outof the box tests. But i think i will go that way as well.

Comment

FWIW not sure if i had to do a clean or what have you. But i have "step" in there now and that part "works". However, if i run 2 of the same jobs at the same time it still seems to use the wrong object. Is this what you meant by injecting the reader?

Comment

Then it needs to be registered as a listener - the Step factry beans only detect listeners that are directly injected (as reader, writer, processor or tasklet). If you put a log statement or a breakpoint in your beforeStep you should be able to confirm that it is actually being called.