Apache Camel Processors Should NEVER Be Stateful

If you have experience with Apache Camel, this one might sound a little obvious.  But, it has recently come up a few times, so it’s worth mentioning. As an example, say you have a route that iterates over paged data and does something with it, and you therefore need to keep track of the pagination. You might use something like the following:

public class FooProcessor implements Processor {
	private int page = 0;
	
	public void process(Exchange exchange) throws Exception {
		List<String> results = new ArrayList<>();
		
		while (page < 5) {
			// Run *anything* involving pagination -- SQL, REST, etc.
			String result = ...;
			results.add(result);
			page++;
		}
		
		exchange.getIn().setBody(results);
	}
}

Spot the issue? At first glance, this looks like it’d work, and it will if the route never executes multiple times at once. But…

In Camel, a Processor is a singleton bean, meaning every flow through a route hits a single instantiation of the Processor object. So, if a route is executed, then executed again before the first flow finishes, “page” will set back to 0 in the middle of the original run. Even worse, both Processor runs then increment “page”, so neither one is able to retrieve the entire result set.  You wind up with something like:

  1. Processor run #1, page = 0
  2. Processor run #1, page = 1
  3. Processor run #2, page = 0
  4. Processor run #1, page = 1
  5. Processor run #2, page = 2
  6. Processor run #1, page = 3
  7. Processor run #2, page = 4

In the end, run #1 will contain pages 0, 1, 1 & 3, while #2 has 0, 2, and 4.  In high throughput applications, or contexts where the Processor execution can take a while, this certainly spells disaster for the end results.

The example is admittedly ridiculous and could be easily fixed by moving the “page” variable into the “process” method itself, rather than at the Class level.  However, the point is that any statefulness within a Processor, Bean, etc. must be avoided!

Why Systems Integration is Hard: Ashley, Bob, Cindy, Dave, Emily, Frank, Gina, and Hank

Imagine you manage a team of corporate event planners, responsible for overseeing each event and ensuring every single detail is arranged and executed.  Your team consists of 8 individuals.  The scale of each event is sufficiently large, requiring you to task each individual.  If any of them fail, the event will be unsatisfactory (at best) or completely unsafe (at worst).

Ashley speaks English.  Ashley refuses to listen to anyone but Gina.

Bob speaks Spanish.  If you ask him to do something, you won’t hear anything from him until he’s completely finished.  So, you’re not always sure if he heard you to begin with.

Cindy speaks Portuguese.  She can understand bits and pieces of Bob’s Spanish, but regularly needs help translating anyway.  If asked to do something, she will immediately respond with “yep, no problem at all” and run off to work on it.  But, she’ll frequently hit a roadblock and stop working on the task.  You won’t find out whether or not her work was completed until a few days later.  By that time, you’ve long since forgotten what you asked and have to go back to your notes to figure it out.

Dave can speak English, but he instead chooses to speak a language he made up entirely on his own and refuses to respond to anything else.  All you have is a set of hastily written notes he jotted down on a napkin.  Most of the ways you know to communicate with him were arrived at by trial and error on past projects, all of which required you to keep your own notes.  The COO firmly believes he’s absolutely critical to the business and frequently directs you to assign him important tasks.  But he’s only available 25% of the time you need him.  The other 75%, he’s drunk at the local bar.

Emily speaks English, but interlaced with words she made up purely because she thinks they’re superior to the standard language.  She refuses to speak verbally and will only communicate using written word on paper.

Frank is obsessed with the “old days” and sticks to Old English.  He’s perfectly capable of becoming effective with modern English, but he’s stuck in his ways.  These days, Frank can’t handle much on his own, but he’s been an employee for a solid 40 years.  He knows everything there is to know about local ordinances and codes, so you must frequently consult with him and pass the information off to the others.

Gina does not understand a single language, but she’s able to perfectly enunciate any sentence you tell her, repeating it to someone else.  However, she refuses to do so unless you first pat her on the back and gently shove her in the correct direction.

Hank spoke English when you first hired him, but he decides to speak nothing but his hobby language (Klingon) in the middle of the project.

thonk

Need help managing Ashley, Bob, Cindy, Dave, Emily, Frank, Gina, and Hank?  I’m all ears.

Apache Karaf on Vagrant (example Vagrantfile)

If you have a team developing OSGi applications for Apache Karaf, Vagrant provides an easy way to ensure everyone is testing local deployments in a consistent context.  Vagrant is a little like Docker, using a layered approach to build up virtual environments.  In this case, we create an Ubuntu “box”, running on a VirtualBox VM, and automatically set it up with everything necessary for Karaf testing. Continue reading

Find Transaction Leaks in Wildfly and JCA

Here’s a quick tip: the easiest way to find transaction leaks in Wildfly/JCA.  In standalone.xml’s JCA subsystem, change

<cached-connection-manager/>
to
<cached-connection-manager debug=”true”/>

That will automatically close un-closed JCA connections and spit out a nice warning/stacktrace, showing exactly where the transaction was opened.

Apache and MariaDB/MySQL Settings for Low-Memory Servers

Gone are the days of requiring large amounts of resources to adequately run a fast, enterprise-grade web server.  I currently run a single DigitalOcean instance (the 1GB memory plan) and host many web platforms with no performance issues, whatsoever.  I thought I’d share the settings that have been working really well in this low-memory environment.  Note that the server is running CentOS 7, but these settings should be applicable for any OS.

For what it’s worth, if you’re interested in a DigitalOcean account, click here to use my referral — you’ll gain $10 in credits when you sign up… Continue reading