Spring, Quartz, and Persistence

Spring is great, Spring is fun, Spring is best when it’s one on one (props to George Michael).

No, really, Spring is a great architecture. Its easily configured, highly flexible, and comes with a ton of already coded utils for doing anything from REST based integration calls to quickly deployed Servlets.

One of the packages that you may eventually end up using it the Spring Scheduler. The Spring Scheduler is a Spring wrapper to the very nice scheduling library, Quartz. The Spring Scheduler is simple and easy until you have to worry about more than one server. As soon as you get into a clustered environment (especially one that you back the scheduling with a database) it starts to swagger like a frat boy during homecoming. Unless your careful it just may puke on you.

So you don’t waste 3 (yes 3) days on getting it setup and working I thought I’d go over what it takes to get Quartz to run in a Spring clustered environment with a db backend and actual usable Spring beans (autowiring, etc).

The first thing you will want to do it get the Quartz libraries. Careful now because the latest Quartz libs (2.x) don’t work with Spring. You have to use something earlier (at the time of this writing it’s at 1.8.6).
Here is the Maven dependencies for it:

	<dependency>
		<groupId>org.quartz-scheduler</groupId>
		<artifactId>quartz</artifactId>
		<version>1.8.5</version>
		<exclusions>
			<exclusion>
				<groupId>com.mchange.c3p0</groupId>
				<artifactId>com.springsource.com.mchange.v2.c3p0</artifactId>
			</exclusion>
		</exclusions>
	</dependency>

        <dependency>
            <groupId>org.quartz-scheduler</groupId>
            <artifactId>quartz-oracle</artifactId>
            <version>1.8.5</version>
        </dependency>

Please note that I excluded C3P0 because I already use it in most of my projects. If you don’t use C3P0 then your Maven dependencies will be:

	<dependency>
		<groupId>org.quartz-scheduler</groupId>
		<artifactId>quartz</artifactId>
		<version>1.8.5</version>
	</dependency>

        <dependency>
            <groupId>org.quartz-scheduler</groupId>
            <artifactId>quartz-oracle</artifactId>
            <version>1.8.5</version>
        </dependency>

I’m assuming your application already has a dataSource and a transactionManager defined. Something like:

	<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
		destroy-method="close">
		<property name="driverClass" value="${driver}"></property>
		<property name="jdbcUrl" value="${url}" />
		<property name="user" value="${username}" />
		<property name="password" value="${password}" />
		<property name="initialPoolSize" value="2" />
		<property name="minPoolSize" value="2" />
		<property name="maxPoolSize" value="5" />
		<property name="checkoutTimeout" value="100" />
		<property name="maxStatements" value="500" />
		<property name="testConnectionOnCheckin" value="false" />
		<property name="maxIdleTime" value="180" />
		<property name="idleConnectionTestPeriod" value="1000" />
	</bean>

	<bean id="entityManagerFactory"
		class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
		<property name="persistenceProvider" ref="hibernatePersistence" />
		<property name="persistenceUnitName" value="model" />
		<property name="dataSource" ref="dataSource" />
		<property name="jpaVendorAdapter">
			<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
				<property name="showSql" value="true" />
				<property name="databasePlatform" value="org.hibernate.dialect.Oracle10gDialect" />
			</bean>
		</property>
	</bean>

	<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
		<property name="entityManagerFactory" ref="entityManagerFactory" />
		<property name="dataSource" ref="dataSource" />
	</bean>

You will need an dataSource and transactionManager for Quartz to use. Quartz, in order to make sure that multiple servers are not firing the same event, uses the database to keep track of what has executed and what needs to be executed. This allows multiple servers that have the application on it to not fire the same event at the same time. Only one server will fire the job (the server that will do the job is random).

Next we need to set up Quartz to run under Spring. Now, I mentioned Spring Scheduler above. It has it’s one tags, <task>, that work great in single server environments. Anything other than that and you will be using regular Spring bean definitions. There’s quite a bit to the configuration so I’ll go over each piece and then present the whole configuration as one file.

First lets setup the scheduler that will run the jobs for us. This uses the Spring framework SchedulerFactoryBean to create the scheduler that wraps Quartz and runs our jobs for us.

<bean id="scheduler"
	class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">

Next we need to setup some basic configuration parameters to get the system working:

<property name="autoStartup" value="true"/> 
<property name="applicationContextSchedulerContextKey" value="applicationContext"/>
<property name="waitForJobsToCompleteOnShutdown" value="true"/>
<property name="overwriteExistingJobs" value="true"/>
<property name="dataSource">
	<ref bean="dataSource"/>
</property>
<property name="transactionManager">
	<ref bean="transactionManager"/>
</property>
  • autoStartup – Start the Scheduler as soon as it is instantiated
  • applicationContextSchedulerContextKey – The key in the map that will be passed to our jobs that has the Spring Application Context in it
  • waitForJobsToCompleteOnShutdown – Should we not shut down if a job is in the middle of executing
  • overwriteExistingJobs – If the configuration in this file changes from what Quartz has in the database, should the database values be overwritten
  • dataSource – The dataSource (already defined as a bean somewhere) that should be used for database connectivity
  • transactionManager – The transactionManager (already defined as a bean somewhere) that should be used to retrieve transactions

There are additional properties but they merit more individual attention:

		<property name="quartzProperties">
			<props>
				<!-- Job store -->
				<prop key="org.quartz.jobStore.misfireThreshold">6000000</prop>
				<prop key="org.quartz.jobStore.driverDelegateClass">org.quartz.impl.jdbcjobstore.oracle.OracleDelegate</prop>

				<prop key="org.quartz.jobStore.isClustered">true</prop>
				<prop key="org.quartz.jobStore.clusterCheckinInterval">20000</prop>

				<prop key="org.quartz.scheduler.instanceName">ClusteredScheduler</prop>
				<prop key="org.quartz.scheduler.instanceId">AUTO</prop>
				<prop key="org.quartz.scheduler.jmx.export">true</prop>
			</props>
		</property>
  • misfireThreshold – The the number of milliseconds the scheduler will ‘tolerate’ a trigger to pass its next-fire-time by, before being considered “misfired”
  • driverDelegateClass – The class to use to connect to your database instance. I use Oracle. This should be changed to your database type
  • isClustered – Is this a clustered instance. Yes because we are running on more than one server
  • clusterCheckinInterval – How often should things be checked
  • instanceName – The name of this cluster instance
  • instanceId – How will the instances be id’ed? AUTO lets Quartz deal with it
  • jmx.export – Should management beans be exported? I like MBeans so put it as true. This is not required for persistence
		<property name="triggers">
			<list>
				<ref bean="caseArchiveSuspendedJobTrigger"/>
		         </list>
		</property>
  • triggers – The triggers are a list of jobs that need to be executed. More on that in a bit

The last property that needs defined is the factory that will create the Jobs. Spring Scheduler ships with a version that works by creating a new instance of a job each time the job is to be executed. The issue with this is that the job is not in the scope of the Spring application because of this. The Job does have its parameters set but any autowiring or, more importantly, transactions, will not exists. This means that if you job does any database manipulation that requires a commit, it will not work. I have created my own SpringBeanJobFactory that gets around this by creating the job as a regular Spring bean and then using that instance of the bean to execute the job.

		<property name="jobFactory">
			<bean class="mycode.scheduled.SpringBeanJobFactory"/>
		</property>

And the code for the class:

package mycode.scheduled;

import org.quartz.SchedulerContext;
import org.quartz.spi.TriggerFiredBundle;
import org.springframework.beans.BeanWrapper;
import org.springframework.beans.MutablePropertyValues;
import org.springframework.beans.PropertyAccessorFactory;
import org.springframework.scheduling.quartz.SpringBeanJobFactory;
import org.springframework.web.context.support.XmlWebApplicationContext;

/**
 * @author sseaman - mostly a cut/paste from the source class
 *
 */
public class SpringBeanJobFactory 
	extends org.springframework.scheduling.quartz.SpringBeanJobFactory
{

	private String[] ignoredUnknownProperties;

	private SchedulerContext schedulerContext;


	@Override
	public void setIgnoredUnknownProperties(String[] ignoredUnknownProperties) {
		super.setIgnoredUnknownProperties(ignoredUnknownProperties);
		this.ignoredUnknownProperties = ignoredUnknownProperties;
	}
	
	@Override
	public void setSchedulerContext(SchedulerContext schedulerContext) {
		super.setSchedulerContext(schedulerContext);
		this.schedulerContext = schedulerContext;
	}
	
	/**
	 * An implementation of SpringBeanJobFactory that retrieves the bean from
	 * the Spring context so that autowiring and transactions work
	 * 
	 * This method is overriden.
	 * @see org.springframework.scheduling.quartz.SpringBeanJobFactory#createJobInstance(org.quartz.spi.TriggerFiredBundle)
	 */
	@Override
	protected Object createJobInstance(TriggerFiredBundle bundle) throws Exception {
		XmlWebApplicationContext ctx = ((XmlWebApplicationContext)schedulerContext.get("applicationContext"));
		Object job = ctx.getBean(bundle.getJobDetail().getName());
		BeanWrapper bw = PropertyAccessorFactory.forBeanPropertyAccess(job);
		if (isEligibleForPropertyPopulation(bw.getWrappedInstance())) {
			MutablePropertyValues pvs = new MutablePropertyValues();
			if (this.schedulerContext != null) {
				pvs.addPropertyValues(this.schedulerContext);
			}
			pvs.addPropertyValues(bundle.getJobDetail().getJobDataMap());
			pvs.addPropertyValues(bundle.getTrigger().getJobDataMap());
			if (this.ignoredUnknownProperties != null) {
				for (String propName : this.ignoredUnknownProperties) {
					if (pvs.contains(propName) && !bw.isWritableProperty(propName)) {
						pvs.removePropertyValue(propName);
					}
				}
				bw.setPropertyValues(pvs);
			}
			else {
				bw.setPropertyValues(pvs, true);
			}
		}
		return job;
	}
	
}

As you can see the code gets the applicationContext and then uses that to retrieve the bean from Spring. This bean will be in any defined transactions and will have all Autowiring already done.

The last steps are to define the trigger and the bean that represents the job. For this we will use more of the Spring Scheduler classes.

	<bean id="caseArchiveSuspendedJobTrigger"
		class="org.springframework.scheduling.quartz.CronTriggerBean">
		<property name="jobDetail">
			<bean name="caseArchiveSuspendedJob" class="org.springframework.scheduling.quartz.JobDetailBean">
				<property name="name" value="caseArchiveSuspendedJob"/>
				<property name="jobClass" value="mycode.job.CaseArchiveSuspendedJob"/>
			</bean>
		</property>
		<property name="cronExpression" value="0 0 3 * * ?"/>
	</bean>
	
	<bean id="caseArchiveSuspendedJob" class="mycode.job.CaseArchiveSuspendedJob">
		<property name="daysToArchiveSuspended" value="6"/>	
	</bean>

First, create a definition for the type of trigger. Again, mine is Cron based. The CronTriggerBean requires two properties:

  • jobDetail
  • cronExpression

The cronExpression is straightforward. The jobDetail requires some explanation.

The jobDetail uses an instance of the JobDetailBean but not in the way Spring Scheduler intended. The name is set to correspond to the bean id of the bean that represents the actual job. The jobClass is the class of the bean (same as defined in the bean class attribute). This allows the SpringBeanJobFactory to get the actual Spring instance of the bean by using the name while still satisfying the requirements of the JobDetailBean (jobClass is required).

Lastly, we just define our job bean as we would any other bean.

Here is a skeleton version of my job:

public class CaseArchiveSuspendedJob {
    @Override
	@Transactional(propagation = Propagation.REQUIRED, readOnly = false)
	public void execute(JobExecutionContext context)
		throws JobExecutionException
	{
            // do something with the db in a transaction
     }
}

And now, here is everything in one file:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	default-autowire="byName"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
	
	<bean id="scheduler"
		class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
		<property name="autoStartup" value="true"/>
		<property name="applicationContextSchedulerContextKey" value="applicationContext"/>
		<property name="waitForJobsToCompleteOnShutdown" value="true"/>
		<property name="overwriteExistingJobs" value="true"/>
		<property name="dataSource">
			<ref bean="dataSource"/>
		</property>
		<property name="transactionManager">
			<ref bean="transactionManager"/>
		</property>
		<property name="jobFactory">
			<bean class="mycode.scheduled.SpringBeanJobFactory"/>
		</property>
		<property name="quartzProperties">
			<props>
				<prop key="org.quartz.jobStore.misfireThreshold">6000000</prop>
				<prop key="org.quartz.jobStore.driverDelegateClass">org.quartz.impl.jdbcjobstore.oracle.OracleDelegate</prop>

				<prop key="org.quartz.jobStore.isClustered">true</prop>
				<prop key="org.quartz.jobStore.clusterCheckinInterval">20000</prop>

				<prop key="org.quartz.scheduler.instanceName">SgsClusteredScheduler</prop>
				<prop key="org.quartz.scheduler.instanceId">AUTO</prop>
				<prop key="org.quartz.scheduler.jmx.export">true</prop>
			</props>
		</property>
		<property name="triggers">
			<list>
				<ref bean="caseArchiveSuspendedJobTrigger"/>
			</list>
		</property>
	</bean>
	
	<bean id="caseArchiveSuspendedJobTrigger"
		class="org.springframework.scheduling.quartz.CronTriggerBean">
		<property name="jobDetail">
			<bean name="caseArchiveSuspendedJob" class="org.springframework.scheduling.quartz.JobDetailBean">
				<property name="name" value="caseArchiveSuspendedJob"/>
				<property name="jobClass" value="mycode.job.CaseArchiveSuspendedJob"/>
			</bean>
		</property>
		<property name="cronExpression" value="* * * * * * ?"/>
	</bean>
	
	<bean id="caseArchiveSuspendedJob" class="mycode.job.CaseArchiveSuspendedJob">
		<property name="daysToArchiveSuspended" value="10"/>	
	</bean>
</beans>

As you can see, there is quite a lot setting up Spring and Quartz to easily work within a transaction. I spent way too much time researching this. I hope this helps someone….

About sseaman

Connect with me on Google+
This entry was posted in Java, Programming and tagged , , . Bookmark the permalink.

118 Responses to Spring, Quartz, and Persistence

  1. Pingback: How to use Quartz 2.2 with Spring 3.2.x on WebSphere AS | Monobit Solutions

Leave a Reply

Your email address will not be published. Required fields are marked *