Scheduler for Java.
It is able to schedule execution of
1. java methods: public static int run(Job job)
2. any OS process
3. java method or process at remote host
It implements the following models of scheduling:
1. trigger: job can be started as the result of one of the following events:
- success termination of some other job
- failure of some other job
- queue overflow
2. permanent: job is always started when scheduler is activated
3. periodic scheduling: job is repeatedly started after specified interval of time
4. calendar bases scheduling: scheduler periodically reload calendar from JobCalendar
calendar table and run jobs according to the time specified in the calendar.
5. hour-of-day scheduling: it is possible to specify mask of hours of day and mask of day of week
when job should be started. For example day mask 0x41 and hour mask 0x400 will cause execution
of some job at weekend (Saturday + Sunday) at 10am
All information for the scheduler is specified through relational database.
Scheduler access this information through JDBC protocol.
Tables JobDesc and QueueDesc contains descriptions of jobs and queues registered for scheduling.
Tables JobCalendar and JobFlow are used to specify scheduling times for jobs (calendar scheduling) and
control flow transfer diagram (trigger scheduling).
Scheduler itself fills JobState and QueueState tables which represent the current state of
job execution. These tables can be used by any application for monitoring state of job execution.
Also JobState table is used to restart execution of running jobs in case of system fault.
Now few words about how to test and use current implementation.
It is currently tested with my own pure Java database JRacers but it can be used with any database
for which JDBC or ODBC driver is available.
I. Structure of catalogues:
bin - contains scripts for running scheduler and supplementary tasks:
- run: start scheduler
- proxy: start scheduler proxy process which allows execution of jobs at remote nodes
- jipc: start JIPC server
- jsql: JRacer SQL utility
lib - contains scheduler.jar
sql - contains SQL scripts to create and drop database:
- create-db.sql: create all tables used by scheduler and indices for these tables
- drop-db.sql: drop all tables created by create-db.sql
src - contains Java sources of scheduler. This directory contains compile.bat script which
build scheduler.jar
tst - contains basic tests for the scheduler. For each test there is correspondent init-dbN.sql file
which initialize database with the data for correspondent test.
- Test of spawning local and remote Java jobs and processes (init-db1.sql)
This test uses TestJob class which is spawned locally and remotely with different parameters.
Depending on the parameter value testJob is either successfully finished, either failed, either throw exception.
- Test of handling queue overflow (init-db2.sql)
This test use Produces and Consumer classes. Producer jobs is spawned more frequently causing job overflow and
execution of job overflow handler (TestJob with "overflow" parameter).
- test of job flow (init-db3.sql)
This test for trigger scheduling. Workflow for jobs is specified and tested.
-- Test for calendar scheduling (init-db4.sql).
Test of calendar scheduling.
II. How to run
To run scheduler the following processes are needed:
- JIPC server (modified version of JIPC server supporting polling of queue size)
- database server (any DBMS supporting JDBC interface)
- remote proxies. Proxies should be started at all nodes at which execution of remote jobs is planned.
Installation steps will be the following:
1. Extract scheduler.zip, jipc.zip and (optionally) jracer.zip. You can install them in any place but by
default in scripts it is assumed that them are installed in root directory, so paths will be
\scheduler, \jipc, \jracer. If you extract them to the alternative location, you will have to specify
correct PATHes before you start scripts (it can be done using environment variables).
2. Include in PATH \scheduler\bin directory
3. Create database. With JRacer it can be done in the following way:
jsql %SCHEDULER_HOME%\sql\create-db.sql
After completion execution of the create-db.sql script, JSQL will show ">" prompt.
Just type "exit".
4. Fill database with some data. If you want to run one of the existed tests, then
execute of one init-dbN.sql files, for example:
jsql %SCHEDULER_HOME%\tst\init-db1.sql
5. Start jipc server (just execute jipc.bat in separate window assuming that %SCHEDULER_HOME%\bin is in PATH)
6. Start remote proxies (execute proxy.bat assuming that %SCHEDULER_HOME%\bin is in PATH)
7. Start scheduler itself (execute run.bat assuming that %SCHEDULER_HOME%\bin is in PATH)
Scheduler can be stopped by typing "exit". Proxy can be stopped by Ctrl-C.
III. How to configure
Parameters for the scheduler are specified through Java properties mechanism.
Below is the list of valid properties:
verbose: Extra trace messages [false]
db.url: JDBC database URL [jdbc:odbc:LocalServer]
db.driver: JDBC driver class name [sun.jdbc.odbc.JdbcOdbcDriver]
db.user: database user name [sa]
db.password: database user password []
entry.point: name of the method which will be invoked by scheduler [run]
trace.dir: directory where trace files for local tasks will be created [.]
trace.append: whether to append or rewrite trace file for local tasks [false]
log.file.path: path to the scheduler log file [scheduler.log]
log.append: whether to append or rewrite scheduler log file [false]
queue.poll.interval: interval in seconds of polling state of queues [60]
queue.create: whether scheduler should create queues [false]
ipc.server.name: name of the host at which IPC server is running [localhost]
ipc.server.port: port listened by IPC server [6000]
Proxy properties are the following:
port: port at which proxy will accept connections [6001]
trace.dir: directory where trace files for remote tasks will be created [.]
trace.append: whether to append or rewrite trace file for remote tasks [false]
entry.point: name of the method which will be invoked by scheduler [run]
log.path: path to log file [proxy.log]
log.append: whether to append or rewrite log file [false]
IV. Source of possible problems.
Scheduler was tested with JRacer DBMS. Although it is using pure JDBC interface, there can be
problems with other database and JDBC drivers (caused by incompatibility with types used by Scheduler
and builtin types supported by DBMS). I tried to choose most portable SQL types (varchar, smallint, tinyint,
timestamp) and also use "create domain" clause to create-db.sql to be able to change used type only in one place
if it is needed. But still using other DBMS may need changing SQL files and may be (in worst case)
even Scheduler sources.
- lack or improper support if timestamp data type.