Jekyll2019-08-25T22:31:27+01:00https://tomd.xyz/Tom D’s Tech BlogA blog on development and devops.Tom DonohueThere can be only one… properties file2019-08-25T09:00:00+01:002019-08-25T22:30:16+01:00https://tomd.xyz/blog/merging-properties-files<p>Two good things happened this week:</p>
<ol>
<li>
<p>Eclipse did not crash.</p>
</li>
<li>
<p>I discovered a new command to add to my <a href="http://kb.monodot.co.uk/text-manipulation-linux.html">Linux text manipulation</a> arsenal.</p>
</li>
</ol>
<hr />
<p>I’m sure I don’t need to explain any further about my first point above. We’ve all experienced it. I think I must just have been lucky this week.</p>
<p>But I’m not here to write about IDEs.</p>
<p>This week I made a nice new <strong>automation recipe</strong>, which is going in my cookbook.</p>
<p>And it’s down to a use case I saw this week, which is the use of multiple Java properties files to specify an application’s config.</p>
<p>For example, we might see a properties loader defined somewhere, which loads, in the following order:</p>
<ul>
<li><code class="highlighter-rouge">my-application-core.properties</code></li>
<li><code class="highlighter-rouge">my-environment.properties</code></li>
<li><code class="highlighter-rouge">my-special-overrides.properties</code>.</li>
</ul>
<p>The idea here is that properties defined in <code class="highlighter-rouge">my-environment</code> should override those in <code class="highlighter-rouge">my-application-core</code>. And the ones in <code class="highlighter-rouge">my-special-overrides</code>, well as they’re super-special, they should override everything.</p>
<p>In the world of Spring Boot, we could do this using <code class="highlighter-rouge">spring.config.location</code> property. Spring will use this property as a list of locations to search for <code class="highlighter-rouge">application.properties</code> files. It will load the ones it finds, in turn.</p>
<p>But I’m not a fan of this, because it’s not easy to see your application’s config at a glance (unless you <a href="/blog/spring-boot-env-endpoint/">left Spring Boot’s env endpoint enabled</a>, which probably isn’t a good idea).</p>
<p>In Kubernetes we have the <strong>ConfigMap</strong> object. Wouldn’t it be nice if we could specify <strong>just one ConfigMap</strong> for a Spring Boot application – e.g. using the Spring Cloud Kubernetes bridge – and then have all of our properties set in there?</p>
<p>What I want to do is merge the three <code class="highlighter-rouge">.properties</code> files together, removing duplicate keys and ignoring earlier entries.</p>
<p>We can use Linux text manipulation commands to get us there, specifically <code class="highlighter-rouge">tac</code> and <code class="highlighter-rouge">awk</code>.</p>
<ul>
<li><code class="highlighter-rouge">tac</code> will print a file backwards (it’s <code class="highlighter-rouge">cat</code> backwards, geddit?)</li>
<li><code class="highlighter-rouge">awk</code> will do almost anything with text</li>
</ul>
<p>The following will merge 2 properties files together, using <code class="highlighter-rouge">awk</code> as a text processor. <code class="highlighter-rouge">awk</code> processes the files as tabular-like, and, using the <code class="highlighter-rouge">=</code> sign as a column separator, takes the <strong>first occurrence</strong> of a <code class="highlighter-rouge">key=</code> as its accepted value. The special operator <code class="highlighter-rouge">seen</code> removes a line if this specific <code class="highlighter-rouge">key=</code> has been seen before:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cat props1.properties props2.properties | awk -F'=' '!seen[$1]++'
</code></pre></div></div>
<p><strong>BUT</strong>, if we want to take the <strong>final occurrence</strong> of a <code class="highlighter-rouge">key=</code> as its accepted value (i.e. overriding an earlier property definition with a later one), then we can just use <code class="highlighter-rouge">tac</code> to flip the list, de-duplicate, and then <code class="highlighter-rouge">tac</code> to flip it again:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cat props1.properties props2.properties |
tac |
awk -F'=' '!seen[$1]++' |
tac
</code></pre></div></div>
<h2 id="example">Example</h2>
<p>Take the file <code class="highlighter-rouge">dogs.properties</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>legs=4
colour=black
</code></pre></div></div>
<p>Take the file <code class="highlighter-rouge">cats.properties</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pet=cat
food=whiskas
colour=ginger
</code></pre></div></div>
<p>When running the command above, we get this merged properties file, where <code class="highlighter-rouge">colour=ginger</code> overrides <code class="highlighter-rouge">colour=black</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>legs=4
pet=cat
food=whiskas
colour=ginger
</code></pre></div></div>Tom Donohueaka giving Spring just enough config to runAdventures in debugging: Camel mock endpoints are real2019-08-15T09:00:00+01:002019-08-16T09:31:52+01:00https://tomd.xyz/blog/mock-endpoints-are-real<p>I made an interesting discovery the other week, which I want to share with you!</p>
<p>I was trying to set up <a href="https://docs.pact.io/getting_started">Pact contract testing</a> for a <a href="https://github.com/apache/camel/blob/master/docs/components/modules/ROOT/pages/spring-boot.adoc">Camel on Spring Boot</a> application, and kept running into issues with a mock endpoint.</p>
<p>The endpoint in question was <strong>camel-mongodb3</strong> (now renamed <a href="https://github.com/apache/camel/blob/master/components/camel-mongodb/src/main/docs/mongodb-component.adoc"><strong>camel-mongodb</strong></a>) component. Here’s what the original Camel route looked like - <em>edited for the purposes of this blog post!</em>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">from</span><span class="o">(</span><span class="s">"direct:get-orgs"</span><span class="o">).</span><span class="na">id</span><span class="o">(</span><span class="s">"get-organisations-route"</span><span class="o">)</span>
<span class="o">.</span><span class="na">to</span><span class="o">(</span><span class="s">"mongodb3:myMongoClient?database=&amp;collection=&amp;operation=findAll"</span><span class="o">)</span>
<span class="o">.</span><span class="na">process</span><span class="o">(</span><span class="k">new</span> <span class="n">ResponseMessageProcessor</span><span class="o">())</span>
<span class="o">.</span><span class="na">log</span><span class="o">(</span><span class="s">"Returning body - ${body}"</span><span class="o">);</span>
</code></pre></div></div>
<p>I wanted to mock the <code class="highlighter-rouge">mongodb3:</code> endpoint in a unit test, so that it didn’t attempt to connect to a real MongoDB instance. So I use <code class="highlighter-rouge">mockEndpointsAndSkip()</code> in the test class:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mockEndpointsAndSkip</span><span class="o">(</span><span class="s">"mongodb3:*"</span><span class="o">);</span>
</code></pre></div></div>
<p>When I ran the test, I saw the confirmation that the component had been mocked in the logs:</p>
<blockquote>
<p>Adviced endpoint [mongodb3://myMongoClient] with mock endpoint [mock:mongodb3:myMongoClient]</p>
</blockquote>
<p>But – despite mocking the Mongo component – I could see that it was still attempting to make a <strong>real connection</strong> to a Mongo instance on localhost, port 27018, and failing:</p>
<blockquote>
<p>java.lang.IllegalStateException: Failed to load ApplicationContext …</p>
</blockquote>
<blockquote>
<p>20:26:48.838 [cluster-ClusterId{value=’5d55b1f8717ae36ad8ceeddd’, description=’null’}-localhost:27018] INFO org.mongodb.driver.cluster - Exception in monitor thread while connecting to server localhost:27018
com.mongodb.MongoSocketOpenException: Exception opening socket</p>
</blockquote>
<p><strong>Why was the Mongo component still being instantiated even though I had mocked it?</strong></p>
<p><img width="200" class="align-center" src="https://media.giphy.com/media/kaq6GnxDlJaBq/giphy.gif" alt="Very confused" /></p>
<p>I spent a long time trying to figure out what was going on. I <em>may</em> have banged my head against the keyboard a few times.</p>
<p>Was the dependency for Pact testing playing havoc with Camel? Was it causing the mock endpoints to be ignored?</p>
<p>I turned to the trusty tool I use when I encounter situations like this: <strong>isolation</strong>.</p>
<p>To me, isolation goes something like this:</p>
<ol>
<li>
<p>Comment out blocks of code.</p>
</li>
<li>
<p>Comment out some more code.</p>
</li>
<li>
<p>Strip everything out of the application, until you’re left with almost nothing.</p>
</li>
<li>
<p>Hope you’ve found the problem.</p>
</li>
</ol>
<p>But that still didn’t give me an answer.</p>
<p>So I <strong>replaced the Mongo component</strong> with another fairly innocuous component: the JMS component.</p>
<p>That <strong>did work</strong>: the JMS component <strong>seemed</strong> to get mocked and skipped successfully.</p>
<p>So why was it that one component seemed to be mocked and skipped, but another was not?</p>
<p>I started <a href="https://www.jetbrains.com/help/idea/debugging-code.html">debugging the application in IntelliJ</a> (love the support for decompiling and breakpoints!) and stepping through the code, line-by-line.</p>
<p>(This is how I get when I <strong>refuse to be stumped by a problem</strong>. I don’t call it a day and find a workaround. No, I get a bit, erm, obsessed.)</p>
<p>I stepped through the code, right down to the deepest parts of the Camel source.</p>
<p>And I made this discovery:</p>
<p><strong>Original endpoints are still initialised, even if they have been mocked.</strong></p>
<p>Here’s the proof. When stepping through the code, I noticed that this line in <em>MongoDbEndpoint.java</em> gets hit when the Camel Context starts up in my unit test:</p>
<p><img src="/assets/img/mongo-mock-real.png" alt="Camel mongodb component createProducer is invoked even when mocked" /></p>
<p>This is a line which creates the <em>Producer</em> for an endpoint. In other words, even though you might mark an endpoint as <em>mocked and skipped</em>, the <strong>component is still there</strong>. It is still initialised. It will just be <strong>skipped when a message flows through the route</strong>.</p>
<p>This is important, as it means that <strong>components which need some initialisation, like the Mongo component, will still initialise themselves in your unit test</strong>.</p>
<p>And the same goes for the JMS component. Mocking it with:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mockEndpointsAndSkip</span><span class="o">(</span><span class="s">"jms:*"</span><span class="o">);</span>
</code></pre></div></div>
<p>…means it will still get initialised with a Producer. <strong>I didn’t notice this before, because the JMS component doesn’t initialise a connection when a Producer is created</strong>:</p>
<p><img src="/assets/img/jms-mock-real.png" alt="Camel JMS component createProducer is invoked even when mocked" /></p>
<p>So you’ve got to take this fact into account, if you need to mock a component.</p>
<p>. . .</p>
<p>What did I learn? Lots:</p>
<ul>
<li>
<p><strong>The createProducer() method is always called for endpoints, even when they have been mocked.</strong></p>
</li>
<li>
<p>If there is code in createProducer() which tries to create a connection to some third-party system, this code will still be executed, even when the component is mocked.</p>
</li>
<li>
<p>For mongodb3, <a href="https://github.com/apache/camel/blob/camel-2.21.0/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbEndpoint.java#L125-L129">there is a line in MongoDbEndpoint#createProducer which calls initializeConnection</a> – and this attempts to connect to MongoDb to check the database/collection exists, given that there is a MongoClient bean in the context.</p>
</li>
<li>
<p>Also, Camel Producers are <strong>created in the order that they appear in routes</strong>; so if the mongodb component appears before others, it will fail first and cause an exception which terminates the program.</p>
</li>
<li>
<p>So, for MOST components, mocking should work without issue. It’s just that mongodb is one exception to the rule, because it tries to initialise a connection to a database when the endpoint is created.</p>
</li>
<li>
<p>Sheer stubbornness on problem-solving wins again.</p>
</li>
</ul>
<p>And <strong>diving into the source code is a great way of understanding more about the tools I use every day</strong>.</p>
<p>My solution going forward is either to <a href="https://docs.spring.io/spring-boot/docs/1.5.10.RELEASE/reference/htmlsingle/#boot-features-testing-spring-boot-applications-mocking-beans">use a MockBean</a> to create a fake MongoClient object; or, I’ll replace the endpoint URIs in the Camel route with property placeholders, e.g.:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">.</span><span class="na">to</span><span class="o">(</span><span class="s">"{{some.uri}}"</span><span class="o">)</span>
</code></pre></div></div>
<p>…then, I can set the endpoint URI explicitly to <code class="highlighter-rouge">mock:....</code> in my unit test configuration, avoiding the Mongo component altogether.</p>
<p>Learning every damn day. :-)</p>Tom DonohueI discovered that mock endpoints are actually realS2I builds in OpenShift - an example with Java and Maven2019-05-27T10:00:00+01:002019-05-27T15:55:24+01:00https://tomd.xyz/articles/openshift-s2i-example<p>(Photo by Aaron Hermes on Unsplash)</p>
<p>You’ve kicked the tyres with OpenShift and <a href="/articles/spring-boot-kubernetes/">deployed an app or two</a>. Now you want to start building your container images inside OpenShift. But how do you do it? One way is using S2I, or <strong>source-to-image</strong> builds.</p>
<aside class="notice notice--secondary">
<h4 id="why-build-container-images-in-openshift">Why build container images in OpenShift?</h4>
<p>If you’re going to be deploying containers to production, you’ve got to be building them in an automated, repeatable, <em>predictable</em> way. You don’t want to be typing lots of manual Docker build commands.</p>
<p>With an S2I build in OpenShift, you can get OpenShift to compile your code from source, build a container image and push it to the <a href="/articles/openshift-nexus-docker-registry/">registry</a>, all without having to write a <em>Dockerfile</em>.</p>
</aside>
<p>OK, let’s take a look at S2I builds in OpenShift.</p>
<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fas fa-file-alt"></i> On This Page</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#why-build-container-images-in-openshift" id="markdown-toc-why-build-container-images-in-openshift">Why build container images in OpenShift?</a></li>
<li><a href="#what-exactly-is-a-buildconfig" id="markdown-toc-what-exactly-is-a-buildconfig">What exactly is a BuildConfig?</a></li>
<li><a href="#creating-a-buildconfig" id="markdown-toc-creating-a-buildconfig">Creating a BuildConfig</a></li>
<li><a href="#source-to-image-s2i-builds" id="markdown-toc-source-to-image-s2i-builds">Source-to-Image (S2I) builds</a> <ul>
<li><a href="#creating-s2i-builds-using-the-oc-new-build-command" id="markdown-toc-creating-s2i-builds-using-the-oc-new-build-command">Creating S2I builds using the oc new-build command</a> <ul>
<li><a href="#whats-a-builder-image" id="markdown-toc-whats-a-builder-image">What’s a builder image?</a></li>
</ul>
</li>
<li><a href="#example-building-a-java-maven-app-with-oc-new-build" id="markdown-toc-example-building-a-java-maven-app-with-oc-new-build">Example: Building a Java Maven app with oc new-build</a></li>
<li><a href="#an-example-s2i-buildconfig-yaml" id="markdown-toc-an-example-s2i-buildconfig-yaml">An example S2I BuildConfig YAML</a></li>
</ul>
</li>
</ul>
</nav>
</aside>
<h2 id="what-exactly-is-a-buildconfig">What exactly is a BuildConfig?</h2>
<p>A build in OpenShift is defined by a <strong>BuildConfig</strong>.</p>
<p>A BuildConfig is just a set of instructions which tell OpenShift how to build a container image. It defines things like:</p>
<ul>
<li>
<p>the <strong>strategy</strong> to use - this describes the workflow that will be used to build the image. There are a couple of different strategies, but for now I’m just going to talk about source-to-image</p>
</li>
<li>
<p>where the <strong>source</strong> for your image is located - for example, your application’s source code.</p>
</li>
<li>
<p>the <strong>image name and tag</strong> where OpenShift should push the image once it’s been built.</p>
</li>
</ul>
<p>A BuildConfig is like most other objects in OpenShift: it’s usually written as JSON or YAML and then applied to the cluster using the <code class="highlighter-rouge">oc</code> command.</p>
<h2 id="creating-a-buildconfig">Creating a BuildConfig</h2>
<p>Once you’ve developed an app, and you want to start building a container image for it inside OpenShift, you need to create a <em>BuildConfig</em>.</p>
<p>You can do this in a few different ways:</p>
<ul>
<li>
<p>Use OpenShift’s <code class="highlighter-rouge">oc new-build</code> command to create a new build.</p>
</li>
<li>
<p>Write your own BuildConfig by hand (uuuuuuuuuurrrrrgh)</p>
</li>
<li>
<p>Use the <a href="http://maven.fabric8.io/">Fabric8-Maven-Plugin</a>, which inspects your application and creates the necessary OpenShift objects for you</p>
</li>
</ul>
<p>For this quick intro to OpenShift S2I, I’m just going to focus on the first option, using <code class="highlighter-rouge">oc new-build</code>.</p>
<h2 id="source-to-image-s2i-builds">Source-to-Image (S2I) builds</h2>
<p>But first….what is S2I?</p>
<p>A <strong>source-to-image build</strong>, or <strong>S2I build</strong>, is where OpenShift builds a container image from your application’s source code or binary. This is done using the <a href="https://github.com/openshift/source-to-image">Source-To-Image feature</a> of OpenShift.</p>
<p><img width="500" class="align-center" src="/assets/img/s2i-process.png" alt="Visual illustration of the S2I build process in OpenShift" /></p>
<p>OpenShift can clone your source code from Git (or use an existing binary, like a Java JAR), inject that into a <strong>builder image</strong>, and use that to produce a final image for your app, which you can then run on OpenShift.</p>
<!-- Something about builder images -->
<h3 id="creating-s2i-builds-using-the-oc-new-build-command">Creating S2I builds using the oc new-build command</h3>
<p>To <strong>build an app on OpenShift using S2I</strong>, you can use the <code class="highlighter-rouge">oc new-build</code> command to create a <em>BuildConfig</em>.</p>
<p>The command to create a new build from the command line is:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>oc new-build &lt;builder-image&gt;~&lt;repo-url&gt; ...
</code></pre></div></div>
<p>In this example, I’m providing details of a <strong>builder image</strong> that I want to use, to build my code.</p>
<aside class="notice notice--secondary">
<h4 id="whats-a-builder-image">What’s a builder image?</h4>
<p>A builder image is a container image with a few extra bits and bobs. It’s a little bit different from a normal Docker <a href="/articles/java-docker-parent-image/">parent image</a>. It will usually have the build tool needed to build your application pre-installed (e.g. for Java, it’s Maven). It will also have some other hooks that allow it to hook in to OpenShift’s S2I process.</p>
<p>When the S2I build starts, OpenShift starts a container from the builder image, passing along your application’s source code. Your app is then compiled using the tools inside the container (if necessary), and a new container image for your application is built. Voila!</p>
</aside>
<p>You can get builder images for several different languages. For example:</p>
<ul>
<li>
<p><a href="https://github.com/sclorg/s2i-ruby-container">Ruby</a>.</p>
</li>
<li>
<p>Java applications: you can use <a href="https://hub.docker.com/r/fabric8/s2i-java">fabric8/s2i-java, which you can find at Docker Hub</a> (<a href="https://github.com/fabric8io-images/s2i">source code here</a>).</p>
</li>
</ul>
<h3 id="example-building-a-java-maven-app-with-oc-new-build">Example: Building a Java Maven app with oc new-build</h3>
<p>This quick example shows you how to create a S2I build for a Java application on OpenShift.</p>
<p>I’ve created an example Java application (using Spring Boot), so you can follow along with this example steps. My sample app uses Maven as its build tool.</p>
<p>Have a look at the example repo below. (You don’t need to clone the repo though, as all the commands below tell OpenShift to fetch the code directly from GitHub.)</p>
<p><a href="https://github.com/monodot/openshift-demos/tree/master/s2i-java-app" class="button btn btn--info btn--large">View the application source code</a> on GitHub</p>
<p>So, to create a new BuildConfig and start a build for my demo app:</p>
<ol class="tech-steps">
<li>
<p>Create a new build using the <code class="highlighter-rouge">oc new-build</code> command.</p>
<p>If you’re working with the <a href="https://www.okd.io/">community edition of OpenShift, <em>okd</em></a>, then you can use the <a href="https://hub.docker.com/r/fabric8/s2i-java">fabric8 s2i-java</a> builder image:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>oc new-build fabric8/s2i-java~https://github.com/monodot/openshift-demos \
--name=my-demo-app \
--context-dir=s2i-java-app
</code></pre></div> </div>
<aside class="notice notice--secondary">
<p><strong>If you’re using Red Hat OpenShift Container Platform</strong>, then you’ll want to use the Red Hat S2I Java image instead:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>oc new-build registry.redhat.io/redhat-openjdk-18/openjdk18-openshift~https://github.com/monodot/openshift-demos \
--context-dir=s2i-java-app \
--name=my-demo-app
</code></pre></div> </div>
</aside>
<p>This command will:</p>
<ul>
<li>
<p>pull the builder image (either from Docker Hub or Red Hat’s Container Registry)</p>
</li>
<li>
<p>create <em>ImageStreams</em> to point to the builder image, and your application’s container image, once it’s been built.</p>
</li>
<li>
<p>create a <em>BuildConfig</em> using <strong>source-to-image</strong>, pointing at the subfolder which contains my Java app (<code class="highlighter-rouge">s2i-java-app</code>)</p>
</li>
<li>
<p>start the build!</p>
</li>
</ul>
</li>
<li>
<p>Now we can check on the status of the build using <code class="highlighter-rouge">oc get builds</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ oc get builds
NAME TYPE FROM STATUS STARTED DURATION
my-demo-app-1 Source Git@845ad07 Running 30 seconds ago
</code></pre></div> </div>
<p>And shortly after…</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ oc get builds
NAME TYPE FROM STATUS STARTED DURATION
my-demo-app-1 Source Git@845ad07 Complete About a minute ago 1m1s
</code></pre></div> </div>
</li>
<li>
<p>Once the build is complete (as my example shows above), we can check the image has been built by getting all <em>ImageStreams</em>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ oc get is
NAME DOCKER REPO TAGS UPDATED
my-demo-app 172.30.1.1:5000/build-demo3/my-demo-app latest 2 minutes ago
s2i-java 172.30.1.1:5000/build-demo3/s2i-java latest 2 minutes ago
</code></pre></div> </div>
</li>
</ol>
<h3 id="an-example-s2i-buildconfig-yaml">An example S2I BuildConfig YAML</h3>
<p>Want to see what a basic BuildConfig looks like? If you go through the example above, then the <code class="highlighter-rouge">oc new-build</code> command will create and apply a <em>BuildConfig</em> that looks something like the example below.</p>
<p>This BuildConfig defines:</p>
<ul>
<li>
<p>a <strong>source-to-image</strong> build</p>
</li>
<li>
<p>a builder image, which is an <em>ImageStreamTag</em>, <code class="highlighter-rouge">s2i-java</code>, which points to the image we’ll use</p>
</li>
<li>
<p>an output image to build to, which is to an <em>ImageStreamTag</em> named <code class="highlighter-rouge">my-demo-app:latest</code></p>
</li>
</ul>
<p>Here’s the BuildConfig as YAML:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">build.openshift.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">BuildConfig</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">my-demo-app</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">failedBuildsHistoryLimit</span><span class="pi">:</span> <span class="s">5</span>
<span class="na">nodeSelector</span><span class="pi">:</span> <span class="no">null</span>
<span class="na">output</span><span class="pi">:</span>
<span class="na">to</span><span class="pi">:</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ImageStreamTag</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">my-demo-app:latest</span>
<span class="na">postCommit</span><span class="pi">:</span> <span class="pi">{}</span>
<span class="na">resources</span><span class="pi">:</span> <span class="pi">{}</span>
<span class="na">runPolicy</span><span class="pi">:</span> <span class="s">Serial</span>
<span class="na">source</span><span class="pi">:</span>
<span class="na">contextDir</span><span class="pi">:</span> <span class="s">s2i-java-app</span>
<span class="na">git</span><span class="pi">:</span>
<span class="na">uri</span><span class="pi">:</span> <span class="s">https://github.com/monodot/openshift-demos</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">Git</span>
<span class="na">strategy</span><span class="pi">:</span>
<span class="na">sourceStrategy</span><span class="pi">:</span>
<span class="na">from</span><span class="pi">:</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ImageStreamTag</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">s2i-java:latest</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">Source</span>
<span class="na">successfulBuildsHistoryLimit</span><span class="pi">:</span> <span class="s">5</span>
<span class="na">status</span><span class="pi">:</span>
<span class="na">lastVersion</span><span class="pi">:</span> <span class="s">1</span>
</code></pre></div></div>
<hr />
<p>Got any questions about building images in OpenShift? Looking forward to your feedback and comments below! Cheers!</p>Tom DonohueHow to create S2I BuildConfigs for OpenShiftUsing environment variables in Jenkins pipelines - with examples2019-05-04T09:00:00+01:002019-05-04T19:53:55+01:00https://tomd.xyz/articles/jenkins-env-vars<p><em>Photo by Tanvi Malik on Unsplash</em>.</p>
<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fas fa-file-alt"></i> On This Page</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#basic-usage-setting-environment-variables-manually" id="markdown-toc-basic-usage-setting-environment-variables-manually">Basic usage: setting environment variables manually</a></li>
<li><a href="#special-built-in-jenkins-variables" id="markdown-toc-special-built-in-jenkins-variables">Special built-in Jenkins variables</a></li>
<li><a href="#using-job-parameters-as-environment-variables" id="markdown-toc-using-job-parameters-as-environment-variables">Using job parameters as environment variables</a></li>
<li><a href="#setting-environment-variables-from-a-file" id="markdown-toc-setting-environment-variables-from-a-file">Setting environment variables from a file</a></li>
<li><a href="#sharing-pipeline-variables-between-stages" id="markdown-toc-sharing-pipeline-variables-between-stages">Sharing pipeline variables between stages</a></li>
<li><a href="#setting-global-environment-variables" id="markdown-toc-setting-global-environment-variables">Setting global environment variables</a></li>
<li><a href="#example---a-complete-jenkinsfile-with-environment-variables" id="markdown-toc-example---a-complete-jenkinsfile-with-environment-variables">Example - a complete Jenkinsfile with environment variables</a></li>
</ul>
</nav>
</aside>
<p>When you’re writing a Jenkins pipeline, sometimes you want to find a way to be able to inject some dynamic values into the pipeline, so that you don’t have to hardcode everything the pipeline.</p>
<p>This is especially useful for things like:</p>
<ul>
<li>
<p>runtime configuration for your job</p>
</li>
<li>
<p>additional job parameters which you might only know at runtime, but not at design time</p>
</li>
<li>
<p>sensitive data which should only be injected to the job at runtime, and not hardcoded into the pipeline or <em>Jenkinsfile</em></p>
</li>
<li>
<p>toggling certain parts of the pipeline - providing a parameter which describes a certain subset of tests you want to run.</p>
</li>
<li>
<p>IDs of any <a href="https://jenkins.io/doc/book/using/using-credentials/">credentials</a> which you might have defined in Jenkins</p>
</li>
</ul>
<p>Dynamic values like these are made available in your pipeline as <strong>environment variables</strong>. If you carry on reading then in this article I’ll show you how to use them.</p>
<h2 id="basic-usage-setting-environment-variables-manually">Basic usage: setting environment variables manually</h2>
<p>So, how do you declare environment variables inside a Jenkins pipeline?</p>
<p>First, you need to initialise your variables. In <em>declarative pipeline</em> syntax, you do this in an <code class="highlighter-rouge">environment</code> block in the pipeline definition (<em>Jenkinsfile</em>). You can do this:</p>
<ul>
<li>
<p>at the top level, to define environment variables that will be shared across all pipeline stages</p>
</li>
<li>
<p>at the stage level, if you want to define environment variables local <a href="https://jenkins.io/doc/pipeline/steps/pipeline-stage-step/">to a stage</a>.</p>
</li>
</ul>
<p>Here’s an example of the <code class="highlighter-rouge">environment</code> block (for a full example, <a href="#example---a-complete-jenkinsfile-with-environment-variables">see further below</a>):</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Set one environment variable named FAVOURITE_FRUIT</span>
<span class="n">environment</span> <span class="o">{</span>
<span class="n">FAVOURITE_FRUIT</span> <span class="o">=</span> <span class="s1">'tomato'</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Then, to use your variables, use this syntax:</p>
<p class="big text-center"><code class="highlighter-rouge">${VARIABLE_NAME}</code></p>
<p>For example, you might want to use environment variables <a href="https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#echo-print-message">with the echo step</a>, or <a href="https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script">the sh step</a>, to pass an environment variable as an argument to a command, e.g.:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Uses Jenkins's 'echo' step</span>
<span class="n">echo</span> <span class="s2">"I like to eat ${FAVOURITE_EGG_TYPE} eggs"</span>
<span class="c1">// Runs the shell command 'echo'</span>
<span class="n">sh</span> <span class="s2">"echo I like to eat ${FAVOURITE_FRUIT} fruit"</span>
</code></pre></div></div>
<h2 id="special-built-in-jenkins-variables">Special built-in Jenkins variables</h2>
<p>Jenkins also has a bunch of environment variables which it populates for you by default. This includes information about your job and the specific build which is running.</p>
<p>This means you can access things like the <strong>job name</strong> and <strong>build number</strong> inside your pipelines.</p>
<p>These variables are accessed like environment variables, so you can use them in your pipelines. For example, you might want to use the <strong>build number</strong> as a tag in your Git repository.</p>
<p>Here are some of the most common variables which you can access from your pipelines.</p>
<table>
<thead>
<tr>
<th>Variable name</th>
<th>Description</th>
<th>Example value</th>
</tr>
</thead>
<tbody>
<tr>
<td>JOB_NAME</td>
<td>The name of the job in Jenkins</td>
<td><code class="highlighter-rouge">my-job</code></td>
</tr>
<tr>
<td>JOB_URL</td>
<td>The URL to the job in Jenkins UI</td>
<td><code class="highlighter-rouge">http://localhost:8080/job/my-job/</code></td>
</tr>
<tr>
<td>BUILD_NUMBER</td>
<td>The number of the build in Jenkins</td>
<td><code class="highlighter-rouge">6</code>, <code class="highlighter-rouge">7</code>, etc..</td>
</tr>
<tr>
<td>BUILD_TAG</td>
<td>A unique tag for this job name and number</td>
<td><code class="highlighter-rouge">my-job-build-6</code></td>
</tr>
</tbody>
</table>
<p>To use these variables in a pipeline, just reference them in the same way:</p>
<p class="big text-center"><code class="highlighter-rouge">${JOB_NAME}</code></p>
<p>There are tons more built-in environment variables you can use. To see the full list, go to the (not very well-advertised) path <code class="highlighter-rouge">/env-vars.html</code> on your Jenkins server.</p>
<p><strong>For example:</strong> If you go to <code class="highlighter-rouge">http://localhost:8080/env-vars.html/</code> on a Jenkins installation, you should see a list like this:</p>
<p><img width="500" class="align-center screenshot" src="/assets/img/jenkins-env-vars-list.png" alt="Viewing the list of prepopulated Jenkins environment variables" /></p>
<h2 id="using-job-parameters-as-environment-variables">Using job parameters as environment variables</h2>
<p>If your job also has <a href="https://wiki.jenkins.io/display/JENKINS/Parameterized+Build">parameters</a>, then these will be available just like environment variables.</p>
<p>Take this example job, which has one parameter, <code class="highlighter-rouge">WORST_THRONES_CHARACTER</code>, with the value of the worst character from <em>Game of Thrones</em>. :)</p>
<p><img class="align-center screenshot" src="/assets/img/jenkins-env-parameter.png" alt="A build job parameter, expressing an opinion" /></p>
<p>If I want to access this parameter inside my <em>Jenkinsfile</em>, I can refer to it like this:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Build Parameters are set as environment variables in the shell.</span>
<span class="n">sh</span> <span class="s2">"echo The worst GoT character is: ${WORST_THRONES_CHARACTER}"</span>
</code></pre></div></div>
<h2 id="setting-environment-variables-from-a-file">Setting environment variables from a file</h2>
<p>You can also populate environment variables from a file.</p>
<p>You might have a file inside your code repository which contains some configuration properties for your app. Or, you might have a file on a fileshare somewhere.</p>
<p>The <a href="https://plugins.jenkins.io/pipeline-utility-steps">Pipeline Utility Steps plugin</a> has a step <code class="highlighter-rouge">readProperties</code> which makes it easy to read some properties from a file. You can then use these in your other build steps.</p>
<p>(I generally don’t like to install lots of Jenkins plugins, but this does have a lot of useful steps).</p>
<p>So firstly, <strong>install the Pipeline Utility Steps plugin</strong>. Then, if you define a properties file in Java <code class="highlighter-rouge">.properties</code> format, like this:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>WEATHER=rainy
TRAFFIC=heavy
</code></pre></div></div>
<p>Then you can read in the values from the properties file using the <code class="highlighter-rouge">readProperties</code> step, provided by the <a href="https://plugins.jenkins.io/pipeline-utility-steps">Pipeline Utility Steps plugin</a>. You’ll also need to copy any variables you want to use into the environment variable scope:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">stage</span><span class="o">(</span><span class="s1">'reading from a file'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="c1">// Here's an example of downloading a properties file</span>
<span class="n">sh</span> <span class="s1">'curl -OL https://raw.githubusercontent.com/monodot/jenkins-demos/master/environment-variables/extravars.properties'</span>
<span class="c1">// Use a script block to do custom scripting</span>
<span class="n">script</span> <span class="o">{</span>
<span class="kt">def</span> <span class="n">props</span> <span class="o">=</span> <span class="n">readProperties</span> <span class="nl">file:</span> <span class="s1">'extravars.properties'</span>
<span class="n">env</span><span class="o">.</span><span class="na">WEATHER</span> <span class="o">=</span> <span class="n">props</span><span class="o">.</span><span class="na">WEATHER</span>
<span class="o">}</span>
<span class="n">sh</span> <span class="s2">"echo The weather is $WEATHER"</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<h2 id="sharing-pipeline-variables-between-stages">Sharing pipeline variables between stages</h2>
<p>If you want to share the same variable between multiple stages of your pipeline, then you just need to define it at the top level of your pipeline.</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pipeline</span> <span class="o">{</span>
<span class="n">agent</span> <span class="n">any</span>
<span class="n">environment</span> <span class="o">{</span>
<span class="n">FAVOURITE_FILM</span> <span class="o">=</span> <span class="s1">'The Goonies'</span>
<span class="o">}</span>
<span class="n">stages</span> <span class="o">{</span>
<span class="c1">//...</span>
</code></pre></div></div>
<h2 id="setting-global-environment-variables">Setting global environment variables</h2>
<p>You can also set environment variables which will be passed to all pipeline runs. These are <strong>global environment variables</strong>. You can add these from the <em>Manage Jenkins</em> screen:</p>
<p><img class="align-center screenshot" src="/assets/img/jenkins-global-env-var.png" alt="Adding a Jenkins global environment variable" /></p>
<p>These are set as key-value pairs, and apply for every build that’s run from the master node. Just like above, you use <code class="highlighter-rouge">${key}</code> syntax, so for the example above you’d access the variable using <code class="highlighter-rouge">${PASTA}</code>.</p>
<h2 id="example---a-complete-jenkinsfile-with-environment-variables">Example - a complete Jenkinsfile with environment variables</h2>
<p>Finally, here’s a complete example <a href="https://jenkins.io/doc/book/pipeline/syntax/#declarative-pipeline">Jenkins declarative pipeline</a> which uses environment variables, ready for you to copy and start from:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pipeline</span> <span class="o">{</span>
<span class="n">agent</span> <span class="n">any</span>
<span class="n">environment</span> <span class="o">{</span>
<span class="n">FAVOURITE_FRUIT</span> <span class="o">=</span> <span class="s1">'tomato'</span>
<span class="o">}</span>
<span class="n">stages</span> <span class="o">{</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'echo env vars'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="c1">// This prints out all environment variables, sorted alphabetically</span>
<span class="c1">// You should see the variables declared further below</span>
<span class="n">sh</span> <span class="s1">'printenv | sort'</span>
<span class="n">sh</span> <span class="s2">"echo I like to eat ${FAVOURITE_EGG_TYPE} eggs"</span>
<span class="n">sh</span> <span class="s2">"echo And I like to drink ${FAVOURITE_DRINK}"</span>
<span class="n">sh</span> <span class="s2">"echo My favourite city is ${FAVOURITE_CITY}"</span>
<span class="c1">// Build Parameters are also set as environment variables in the shell.</span>
<span class="n">sh</span> <span class="s2">"echo The worst GoT character is: ${WORST_THRONES_CHARACTER}"</span>
<span class="c1">// We can also access some of the built-in environment variables</span>
<span class="n">sh</span> <span class="s2">"echo My hostname is: ${HOSTNAME}"</span>
<span class="c1">// Environment variables can be overridden within a certain block</span>
<span class="n">withEnv</span><span class="o">([</span><span class="s1">'FAVOURITE_CITY=Portland'</span><span class="o">])</span> <span class="o">{</span>
<span class="n">sh</span> <span class="s2">"echo My favourite city is temporarily ${FAVOURITE_CITY}"</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// This block is evaluated before executing the steps block</span>
<span class="n">environment</span> <span class="o">{</span>
<span class="n">FAVOURITE_EGG_TYPE</span> <span class="o">=</span> <span class="s2">"poached"</span>
<span class="n">FAVOURITE_DRINK</span> <span class="o">=</span> <span class="s2">"sauvignon blanc"</span>
<span class="n">FAVOURITE_CITY</span> <span class="o">=</span> <span class="s2">"London"</span>
<span class="n">FAVOURITE_FRUIT</span> <span class="o">=</span> <span class="s2">"satsuma"</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="n">stage</span><span class="o">(</span><span class="s1">'second stage'</span><span class="o">)</span> <span class="o">{</span>
<span class="n">steps</span> <span class="o">{</span>
<span class="c1">// This will echo tomato, because the env var was set at the global scope</span>
<span class="n">sh</span> <span class="s1">'echo My favourite fruit is ${FAVOURITE_FRUIT}'</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>You can also find this on GitHub! Check out the demo repository at the link below:</p>
<p><a href="https://github.com/monodot/jenkins-demos/tree/master/environment-variables" class="button btn btn--large btn--primary">See the example repository</a> on GitHub</p>
<hr />
<p>I hope this has helped you understand how to use variables inside your Jenkins pipelines!</p>
<p>Let me know what you think in the comments below.</p>Tom DonohueHow to use environment variables inside a JenkinsfileDeploying Spring Boot onto Kubernetes2019-03-03T19:00:00+00:002019-04-26T12:51:38+01:00https://tomd.xyz/articles/spring-boot-kubernetes<p>Kubernetes is great, but flippin’ heck, it is unbelievably complicated for new developers.</p>
<p>If you’re a developer who’s new to Kubernetes, you might be staring it in the face and thinking “how the hell do I work this thing”. Or you might be thinking “oh great, yet another step in my deployment process 😩”</p>
<p>So I wanted to put together a clear guide on how to deploy a Spring Boot application to Kubernetes. This will get you going running your first application on Kubernetes.</p>
<p><img width="500" class="align-center" src="/assets/img/spring-boot-and-kubernetes.jpg" alt="Spring Boot and Kubernetes, together forever" /></p>
<p>Here’s what I’m going to walk through in this tutorial:</p>
<ul class="checklist">
<li>
<p><strong>Run Kubernetes</strong> on the desktop using <a href="https://kubernetes.io/docs/tasks/tools/install-minikube/">Minikube</a></p>
</li>
<li>
<p>Use Maven to compile a Spring Boot application to a JAR, and also <strong>build a Docker image</strong> for it</p>
</li>
<li>
<p><strong>Deploy the application on Kubernetes</strong>, in a container</p>
</li>
</ul>
<p>LET’S GO. 🎈🎈</p>
<h2 id="get-yourself-a-friendly-local-kubernetes-cluster">Get yourself a friendly, local Kubernetes cluster</h2>
<p><img width="300" class="align-right" src="/assets/img/minikube-illust.png" alt="Minikube. It goes on your computer." /></p>
<p>For a novice like me, enterprise-grade Kubernetes is pretty difficult to set up. Fortunately, for developers, there’s <a href="https://github.com/kubernetes/minikube">Minikube</a>!</p>
<p>Minikube is a way to run Kubernetes on your local machine. It’s kind of like a scaled-down version of Kubernetes, which is way more suitable for doing local development work.</p>
<p>It lets you get a feel of Kubernetes without having to mess around doing a full install.</p>
<p>To set up your local Kubernetes environment with Minikube:</p>
<ol class="tech-steps">
<li>
<p>Go <a href="https://kubernetes.io/docs/tasks/tools/install-minikube/">grab Minikube from here</a>.</p>
</li>
<li>
<p>Then start minikube:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ minikube start
</code></pre></div> </div>
<p>This will start up a virtual machine (VM) and spin up a single-node Kubernetes cluster inside it.</p>
</li>
<li>
<p>You can check that Minikube is up and running by running <code class="highlighter-rouge">minikube status</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ minikube status
host: Running
kubelet: Running
apiserver: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100
</code></pre></div> </div>
<p>At this point, Kubernetes is now running inside the virtual machine. If you want to take a look at the inner workings of Kubernetes, then you can SSH into the VM and have a look around by typing:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ minikube ssh
</code></pre></div> </div>
</li>
<li>
<p>When minikube starts for the first time, it will create three <em>namespaces</em>. <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/">Namespaces in Kubernetes</a> are like projects within Kubernetes where we can deploy things:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ kubectl get namespaces
</code></pre></div> </div>
<p>We’ll deploy our Spring Boot app into the <code class="highlighter-rouge">default</code> namespace for now as it’s easiest.</p>
</li>
</ol>
<h2 id="build-an-image-with-the-fabric8-maven-plugin">Build an image with the Fabric8 Maven Plugin</h2>
<p>The next step in the process is building a Docker image.</p>
<p>I prefer to get the <a href="https://maven.fabric8.io">Fabric8 Maven Plugin</a> (FMP) to do the hard work. It’s a plugin for Maven which builds images, and does all the necessary stuff to deploy to Kubernetes or OpenShift.</p>
<p>FMP auto-detects your application, makes some reasonable assumptions about what you want to deploy, and sorts out the Docker build for you.</p>
<p>For this tutorial, I’m assuming that you’ve <strong>already got a Spring Boot application you want to deploy</strong>. (If not, go generate a new application using the <a href="https://start.spring.io">Spring Initializr</a>!)</p>
<ol class="tech-steps">
<li>
<p>First, we need to add the <a href="https://maven.fabric8.io">Fabric8 Maven Plugin</a> to our project’s POM. This will set Maven up for some Fabric8 goodness.</p>
<p><code class="highlighter-rouge">cd</code> to your project’s root (where your <code class="highlighter-rouge">pom.xml</code> is located).</p>
<p>Add the Fabric8 Maven Plugin into the plugin section of your <code class="highlighter-rouge">pom.xml</code>:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;plugin&gt;</span>
<span class="nt">&lt;groupId&gt;</span>io.fabric8<span class="nt">&lt;/groupId&gt;</span>
<span class="nt">&lt;artifactId&gt;</span>fabric8-maven-plugin<span class="nt">&lt;/artifactId&gt;</span>
<span class="nt">&lt;version&gt;</span>3.5.41<span class="nt">&lt;/version&gt;</span>
<span class="nt">&lt;configuration&gt;</span>
<span class="nt">&lt;enricher&gt;</span>
<span class="nt">&lt;config&gt;</span>
<span class="nt">&lt;fmp-service&gt;</span>
<span class="nt">&lt;type&gt;</span>LoadBalancer<span class="nt">&lt;/type&gt;</span>
<span class="nt">&lt;/fmp-service&gt;</span>
<span class="nt">&lt;/config&gt;</span>
<span class="nt">&lt;/enricher&gt;</span>
<span class="nt">&lt;/configuration&gt;</span>
<span class="nt">&lt;executions&gt;</span>
<span class="nt">&lt;execution&gt;</span>
<span class="nt">&lt;goals&gt;</span>
<span class="nt">&lt;goal&gt;</span>resource<span class="nt">&lt;/goal&gt;</span>
<span class="nt">&lt;goal&gt;</span>build<span class="nt">&lt;/goal&gt;</span>
<span class="nt">&lt;/goals&gt;</span>
<span class="nt">&lt;/execution&gt;</span>
<span class="nt">&lt;/executions&gt;</span>
<span class="nt">&lt;/plugin&gt;</span>
</code></pre></div> </div>
</li>
<li>
<p>Next, build a Docker image. To build Docker images, Fabric8 Maven Plugin uses a Docker daemon.</p>
<p>Fortunately, Minikube already contains one!</p>
<p>So we execute this next command. It sets up some <strong>environment variables</strong> which configure Fabric8 Maven Plugin to use the Docker daemon running in Minikube. It does this by setting the <code class="highlighter-rouge">DOCKER_HOST</code> variable, and a couple of others:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ eval $(minikube docker-env)
</code></pre></div> </div>
<p class="notice notice--accent">If you’re <strong>running Windows</strong>, just type <code class="highlighter-rouge">minikube docker-env</code> and it will tell you what you need to do.</p>
</li>
<li>
<p>Finally, we compile our Spring Boot application to a JAR (<code class="highlighter-rouge">package</code>) and <strong>build a container image</strong> using the <code class="highlighter-rouge">fabric8:build</code> goal:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ mvn package fabric8:build
</code></pre></div> </div>
<p>This will:</p>
<ul>
<li>
<p>Compile your Spring Boot app to a JAR</p>
</li>
<li>
<p>Pull down a suitable <a href="https://hub.docker.com/r/fabric8/s2i-java">base image</a> to use for your application, from Docker Hub</p>
</li>
<li>
<p>Use Minikube’s Docker daemon to build (or bake!) a Docker image, using the base image</p>
</li>
<li>
<p>Push the new image into Minikube’s local Docker registry</p>
</li>
</ul>
</li>
<li>
<p>Once the build has completed, you can take a look at it in the Docker registry:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
apps/my-spring-boot-application latest d7e9ee334e18 About an hour ago 481MB
apps/my-spring-boot-application snapshot-190303-181942-0375 d7e9ee334e18 About an hour ago 481MB
</code></pre></div> </div>
<p><strong>NB:</strong> Rather confusingly, each Docker image is technically listed in this command in the repository column (not to be confused with a registry, which is a place that stores and serves Docker images!)</p>
</li>
</ol>
<h2 id="run-the-application-in-kubernetes">Run the application in Kubernetes</h2>
<p>The next thing you’ll want to do is run the application.</p>
<p>Fabric8 Maven Plugin can help with that too.</p>
<p>At its most primitive level, Kubernetes is configured by writing a bunch of YAML files and applying them to the cluster using a REST API.</p>
<p>It is, quite frankly, a massive faff.</p>
<p><img width="250" class="align-center" src="/assets/img/yaml-ytho.png" alt="YAML. Thanks." /></p>
<p>Fortunately FMP can do all of the YAML stuff for you. It creates YAML files for your application and then applies them to the Kubernetes cluster.</p>
<ol class="tech-steps">
<li>
<p>To deploy:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ mvn fabric8:watch
</code></pre></div> </div>
<p>This will:</p>
<ul>
<li>Generate the configuration files (YAML) to be applied to Kubernetes</li>
<li>Create a Deployment and a Service for your application</li>
<li>Start a Pod with your application running inside a container</li>
<li>Tail the logs from your Spring Boot application</li>
</ul>
</li>
<li>
<p>Now you can visit your deployed application, in your browser.</p>
<p>Execute this command, which will fetch the <strong>URL</strong> to your application</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ minikube service your-application
</code></pre></div> </div>
<p>Fabric8 Maven Plugin should still be following the logs from Spring Boot, so that you can watch them from the command line.</p>
<p>You can also view the application deployed in the Kubernetes Dashboard, which is a web-based view onto Kubernetes:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ minikube dashboard
</code></pre></div> </div>
</li>
<li>
<p>When you want to stop the application, press <strong>Ctrl+C</strong>.</p>
<p>This will leave the Pod running. So, finally, to undeploy the application from Kubernetes:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ mvn fabric8:undeploy
</code></pre></div> </div>
</li>
</ol>
<p>Congrats, you’ve deployed your Spring Boot application to Kubernetes!</p>
<p>Any questions or comments? Post them below!</p>
<p><a href="https://unsplash.com/photos/q6n8nIrDQHE">Photo by Fré Sonneveld on Unsplash</a></p>Tom DonohueRunning a Spring Boot Java application on Kubernetes with MinikubeUsing Arquillian Cube to test containers in Kubernetes2019-01-30T09:00:00+00:002019-08-15T21:22:21+01:00https://tomd.xyz/blog/arquillian-kubernetes-test<p>This is a very quick note from a recent project where I wanted to run an integration test for an Apache Camel / Spring Boot application, running on OpenShift.</p>
<p>The question is - <strong>how do you test that a container is deployed succesfully into OpenShift or Kubernetes, especially if it has dependencies on other containers?</strong></p>
<p>You can do this by writing an integration test using <a href="http://arquillian.org/arquillian-cube/">Arquillian Cube</a>, the container testing framework. <a href="http://arquillian.org/arquillian-cube/">There are many ways to configure Cube (read the docs)</a>, including auto-scanning and hooking up with the fabric8-maven-plugin.</p>
<p>But, if like me, you want a bit less “magic”, and more explicit set-up for your test, then here’s what I used:</p>
<p>Add the following to your maven <code class="highlighter-rouge">pom.xml</code> - this will pull in Arquillian Cube:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- Arquillian for integration (black-box) testing --&gt;</span>
<span class="nt">&lt;dependency&gt;</span>
<span class="nt">&lt;groupId&gt;</span>org.arquillian.cube<span class="nt">&lt;/groupId&gt;</span>
<span class="nt">&lt;artifactId&gt;</span>arquillian-cube-openshift<span class="nt">&lt;/artifactId&gt;</span>
<span class="nt">&lt;scope&gt;</span>test<span class="nt">&lt;/scope&gt;</span>
<span class="nt">&lt;/dependency&gt;</span>
<span class="nt">&lt;dependency&gt;</span>
<span class="nt">&lt;groupId&gt;</span>io.fabric8<span class="nt">&lt;/groupId&gt;</span>
<span class="nt">&lt;artifactId&gt;</span>kubernetes-assertions<span class="nt">&lt;/artifactId&gt;</span>
<span class="nt">&lt;scope&gt;</span>test<span class="nt">&lt;/scope&gt;</span>
<span class="nt">&lt;/dependency&gt;</span>
</code></pre></div></div>
<p>Create a file <code class="highlighter-rouge">src/test/resources/arquillian.xml</code> - this will configure Arquillian to use an existing namespace, <em>my-itest-namespace</em>, deploy some Kubernetes definitions (YAML files) you’ve defined for your application in <code class="highlighter-rouge">myapplication.yml</code> and then deploy any dependent objects, also defined in <code class="highlighter-rouge">database.yml</code>, <code class="highlighter-rouge">message-broker.yml</code> and so on.</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;arquillian</span> <span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span>
<span class="na">xmlns=</span><span class="s">"http://jboss.org/schema/arquillian"</span>
<span class="na">xsi:schemaLocation=</span><span class="s">"http://jboss.org/schema/arquillian
http://jboss.org/schema/arquillian/arquillian_1_0.xsd"</span><span class="nt">&gt;</span>
<span class="c">&lt;!-- This file describes the resources that need to be created in OpenShift to run a successful test --&gt;</span>
<span class="nt">&lt;extension</span> <span class="na">qualifier=</span><span class="s">"kubernetes"</span><span class="nt">&gt;</span>
<span class="nt">&lt;property</span> <span class="na">name=</span><span class="s">"namespace.use.existing"</span><span class="nt">&gt;</span>my-itest-namespace<span class="nt">&lt;/property&gt;</span>
<span class="nt">&lt;property</span> <span class="na">name=</span><span class="s">"env.config.resource.name"</span><span class="nt">&gt;</span>myapplication.yml<span class="nt">&lt;/property&gt;</span>
<span class="c">&lt;!-- Add YAMLs for any other OpenShift/K8s resources in here --&gt;</span>
<span class="nt">&lt;property</span> <span class="na">name=</span><span class="s">"env.dependencies"</span><span class="nt">&gt;</span>
file:src/test/resources/database.yml
file:src/test/resources/message-broker.yml
file:src/test/resources/auth-server.yml
file:src/test/resources/another-microservice.yml
<span class="nt">&lt;/property&gt;</span>
<span class="nt">&lt;/extension&gt;</span>
<span class="nt">&lt;/arquillian&gt;</span>
</code></pre></div></div>
<p>Define the YAML files for Kubernetes/OpenShift resources for your application - e.g. DeploymentConfigs, Routes, Services and so on - inside <code class="highlighter-rouge">src/test/resources/myapplication.yml</code>.</p>
<p>Also define YAML files for any dependent containers, like database, message broker or auth server, and place them in <code class="highlighter-rouge">src/main/resources/xxx.yml</code>, as given in the example.</p>
<p>Finally, write a unit test class which checks that the deployment happens successfully:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="n">com</span><span class="o">.</span><span class="na">example</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">io.fabric8.kubernetes.client.KubernetesClient</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.jboss.arquillian.container.test.api.RunAsClient</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.jboss.arquillian.junit.Arquillian</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.jboss.arquillian.test.api.ArquillianResource</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.runner.RunWith</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">static</span> <span class="n">io</span><span class="o">.</span><span class="na">fabric8</span><span class="o">.</span><span class="na">kubernetes</span><span class="o">.</span><span class="na">assertions</span><span class="o">.</span><span class="na">Assertions</span><span class="o">.</span><span class="na">assertThat</span><span class="o">;</span>
<span class="nd">@RunWith</span><span class="o">(</span><span class="n">Arquillian</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
<span class="nd">@RunAsClient</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyIntegrationTestIT</span> <span class="o">{</span>
<span class="nd">@ArquillianResource</span>
<span class="n">KubernetesClient</span> <span class="n">client</span><span class="o">;</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testAppProvisionsRunningPods</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
<span class="n">assertThat</span><span class="o">(</span><span class="n">client</span><span class="o">)</span>
<span class="o">.</span><span class="na">deployment</span><span class="o">(</span><span class="s">"myapp"</span><span class="o">)</span>
<span class="o">.</span><span class="na">pods</span><span class="o">().</span><span class="na">isPodReadyForPeriod</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>You can now run your test using:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mvn clean test -Dtest=*IT
</code></pre></div></div>
<p>This test will wait for the Deployment <code class="highlighter-rouge">myapp</code> to run successfully and for the pods to come into the Ready state.</p>
<p>Then it’s just a matter of extending this test, to do things like:</p>
<ul>
<li>check whether the pod comes up correctly</li>
<li>fire requests to your application’s REST endpoint and see whether it behaves as expected</li>
</ul>
<p>…and adding this test into a CI/CD pipeline, to make sure your containers are integration tested automatically. Neat.</p>Tom DonohueHow to implement a basic integration test for a container using Arquillian CubeJBoss/Wildfly: Setting a custom resource adapter for an MDB2018-10-10T09:00:00+01:002019-08-15T21:22:21+01:00https://tomd.xyz/blog/jboss-mdb-custom-resource-adapter<p><strong>How do you configure a Message-Driven Bean (MDB) to use a different resource adapter/connection factory from the default one?</strong></p>
<p>This can be done in one of two ways.</p>
<p>This assumes that you’ve already defined a connection factory in your JBoss/Wildfly config, e.g.:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&lt;pooled-connection-factory name="my-connection-factory"
entries="java:/jms/myConnectionFactory"
user="..." password="..." connectors="..."/&gt;
</code></pre></div></div>
<h2 id="using-annotations">Using annotations</h2>
<p>Annotate your MDB class with the JBoss-specific <code class="highlighter-rouge">@ResourceAdapter</code> annotation, specifying the name of the resource adapter:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">javax.jms.MessageListener</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.jboss.ejb3.annotation.ResourceAdapter</span><span class="o">;</span>
<span class="nd">@ResourceAdapter</span><span class="o">(</span><span class="s">"my-connection-factory"</span><span class="o">)</span>
<span class="nd">@MessageDriven</span><span class="o">(</span><span class="n">name</span> <span class="o">=</span> <span class="s">"HelloWorldQueueMDB"</span><span class="o">,</span> <span class="n">activationConfig</span> <span class="o">=</span> <span class="o">{</span> <span class="o">...</span> <span class="o">})</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">HelloWorldQueueMDB</span> <span class="kd">implements</span> <span class="n">MessageListener</span> <span class="o">{</span>
<span class="c1">//...</span>
<span class="o">}</span>
</code></pre></div></div>
<p>If you need to send messages, just inject the connection factory, looking it up using its JNDI name:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">javax.inject.Inject</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.jms.JMSConnectionFactory</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyClass</span> <span class="o">{</span>
<span class="nd">@Inject</span>
<span class="nd">@JMSConnectionFactory</span><span class="o">(</span><span class="s">"java:/jms/myConnectionFactory"</span><span class="o">)</span>
<span class="kd">private</span> <span class="n">JMSContext</span> <span class="n">context</span><span class="o">;</span>
<span class="c1">//...</span>
<span class="o">}</span>
</code></pre></div></div>
<h2 id="using-xml-config">Using XML config</h2>
<p>Alternatively, to avoid changing the code, you can set this in <code class="highlighter-rouge">META-INF/jboss-ejb3.xml</code>:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?xml version="1.1" encoding="UTF-8"?&gt;</span>
<span class="nt">&lt;jboss:ejb-jar</span> <span class="na">xmlns:jboss=</span><span class="s">"http://www.jboss.com/xml/ns/javaee"</span>
<span class="na">xmlns=</span><span class="s">"http://java.sun.com/xml/ns/javaee"</span>
<span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span>
<span class="na">xmlns:mdb=</span><span class="s">"urn:resource-adapter-binding"</span>
<span class="na">xsi:schemaLocation=</span><span class="s">"http://www.jboss.com/xml/ns/javaee http://www.jboss.org/j2ee/schema/jboss-ejb3-2_0.xsd http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/ejb-jar_3_1.xsd"</span><span class="nt">&gt;</span>
<span class="nt">&lt;enterprise-beans&gt;</span>
<span class="nt">&lt;message-driven&gt;</span>
<span class="nt">&lt;ejb-name&gt;</span>YourEJBName<span class="nt">&lt;/ejb-name&gt;</span>
<span class="nt">&lt;activation-config&gt;</span>
<span class="nt">&lt;activation-config-property&gt;</span>...<span class="nt">&lt;/activation-config-property&gt;</span>
<span class="nt">&lt;activation-config-property&gt;</span>...<span class="nt">&lt;/activation-config-property&gt;</span>
<span class="nt">&lt;/activation-config&gt;</span>
<span class="nt">&lt;resource-ref&gt;</span>
<span class="nt">&lt;res-ref-name&gt;</span>jms/YourConnectionFactoryRefName<span class="nt">&lt;/res-ref-name&gt;</span>
<span class="nt">&lt;res-type&gt;</span>javax.jms.ConnectionFactory<span class="nt">&lt;/res-type&gt;</span>
<span class="nt">&lt;jndi-name&gt;</span>java:/jms/myConnectionFactory<span class="nt">&lt;/jndi-name&gt;</span>
<span class="nt">&lt;/resource-ref&gt;</span>
...
<span class="nt">&lt;/message-driven&gt;</span>
<span class="nt">&lt;/enterprise-beans&gt;</span>
...
<span class="nt">&lt;assembly-descriptor&gt;</span>
<span class="nt">&lt;mdb:resource-adapter-binding&gt;</span>
<span class="nt">&lt;ejb-name&gt;</span>YourEJBName<span class="nt">&lt;/ejb-name&gt;</span>
<span class="nt">&lt;mdb:resource-adapter-name&gt;</span>my-connection-factory<span class="nt">&lt;/mdb:resource-adapter-name&gt;</span>
<span class="nt">&lt;/mdb:resource-adapter-binding&gt;</span>
<span class="nt">&lt;/assembly-descriptor&gt;</span>
<span class="nt">&lt;/jboss:ejb-jar&gt;</span>
</code></pre></div></div>Tom DonohueConfiguring a Message-Driven Bean to use a custom resource adapter, rather than the default oneInvoking a SOAP service with Apache Camel2018-08-01T06:00:00+01:002019-04-19T23:04:53+01:00https://tomd.xyz/articles/camel-soap-service<p>When was the last time you worked with a SOAP service? For me, it was actually yesterday.</p>
<p>SOAP has been around for a long time. You might think that it’s <em>had its day</em>, but <strong>it doesn’t seem to be disappearing yet.</strong></p>
<p>I see SOAP services still in use at many companies. Some well-known enterprise apps still implement their API using SOAP (here’s looking at you, Workday! 👋).</p>
<p>But the problem with SOAP is that it is <strong>unbelievably complicated</strong>, and this complexity is unfortunately also reflected in the tools we use to work with them.</p>
<p>With that in mind, here’s a brief guide to invoking SOAP services in Camel, and the basics you need to know to get up and running.</p>
<h2 id="which-component-to-use">Which component to use?</h2>
<p>In Camel there are two main components you can use to work with SOAP services:</p>
<ul>
<li>
<p><strong>CXF component</strong> (<code class="highlighter-rouge">camel-cxf</code>) - This is a Camel wrapper for <em>Apache CXF</em>, a Java library for working with web services. The CXF component in Camel makes it easier to use CXF in your Camel routes. For example, you can use the CXF component to send and receive messages to/from a SOAP service.</p>
</li>
<li>
<p><strong>SOAP dataformat</strong> (<code class="highlighter-rouge">camel-soap</code>) - This is a Camel <em>DataFormat</em>, so it’s designed for reading and writing SOAP messages only; it doesn’t actually send requests. You might use this dataformat if you want to create a SOAP message (in XML), but send it over a different protocol (like JMS), or write it to disk.</p>
</li>
</ul>
<p>Since I want to invoke a SOAP service (at a SOAP endpoint), I’m going to use the <strong>CXF component</strong> in this article. So let’s go.</p>
<h2 id="introducing-cxf">Introducing CXF</h2>
<p>Apache CXF is a Java framework for working with web services. It primarily supports SOAP, but can also be used to access other types of services, such as REST and even CORBA.</p>
<p><img class="screenshot" src="/assets/img/cxf-homepage.png" alt="Apache CXF website home page" /></p>
<p>You can download Apache CXF from <a href="http://cxf.apache.org/">the official web site</a>. But, when you use CXF in Camel, you only need to pull in the Maven dependency below, and CXF will be downloaded as a <em>transitive dependency</em> to your project:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;dependency&gt;</span>
<span class="nt">&lt;groupId&gt;</span>org.apache.camel<span class="nt">&lt;/groupId&gt;</span>
<span class="nt">&lt;artifactId&gt;</span>camel-cxf<span class="nt">&lt;/artifactId&gt;</span>
<span class="nt">&lt;/dependency&gt;</span>
</code></pre></div></div>
<p>CXF can be a complicated beast; but that’s because it has to support a lot of the advanced features of SOAP. Also, the terminology can be <em>really</em> confusing for beginners.</p>
<p>The best thing I’ve learned about working with CXF, is to just start very simple. Get a simple example up and running, and then add all the bells and whistles.</p>
<p><a href="https://github.com/monodot/camel-demos/tree/master/simple-tests/src/main/java/com/cleverbuilder/cameldemos/scenarios/invokesoapservice" class="button btn btn--info btn--large">Get the example code</a> on GitHub</p>
<h2 id="invoking-a-soap-service-in-camel">Invoking a SOAP service in Camel</h2>
<p>Like most things in Java, there’s lots of different ways to invoke a SOAP service in Camel. But when I’m doing it in my projects, I do these 3 things:</p>
<ol>
<li>
<p><strong>Create client classes from the WSDL.</strong> This is essential if you want to be able to work with the SOAP messages as plain old Java objects (POJOs). Fortunately there’s a plugin to do this for you - to generate JAXB classes from your WSDL.</p>
</li>
<li>
<p><strong>Create your request payload.</strong> You can do this by creating a Java object from the classes you created in step 1, and manipulate the data using <em>getters</em> and <em>setters</em>.</p>
</li>
<li>
<p><strong>Set up the CXF (SOAP) endpoint in Camel, and invoke it from your route.</strong> This is where you implement your logic in Camel to send your SOAP message.</p>
</li>
</ol>
<p>So here’s how to invoke a SOAP service from Camel using the CXF component:</p>
<ol class="tech-steps">
<li>
<p>Add <code class="highlighter-rouge">camel-cxf</code> as a dependency in your project.</p>
</li>
<li>
<p><a href="http://cxf.apache.org/docs/maven-cxf-codegen-plugin-wsdl-to-java.html">Use the <code class="highlighter-rouge">cxf-codegen-plugin</code> for Maven</a> to create Java classes from your WSDL file for you.</p>
<p>Get a hold of the WSDL file from the web service, and copy it into a directory <code class="highlighter-rouge">src/main/resources/wsdl</code>.</p>
<p>Then, set up the <code class="highlighter-rouge">cxf-codegen-plugin</code> in your Maven <code class="highlighter-rouge">pom.xml</code>, like this:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;plugin&gt;</span>
<span class="nt">&lt;groupId&gt;</span>org.apache.cxf<span class="nt">&lt;/groupId&gt;</span>
<span class="nt">&lt;artifactId&gt;</span>cxf-codegen-plugin<span class="nt">&lt;/artifactId&gt;</span>
<span class="nt">&lt;version&gt;</span>3.2.4<span class="nt">&lt;/version&gt;</span>
<span class="nt">&lt;executions&gt;</span>
<span class="nt">&lt;execution&gt;</span>
<span class="nt">&lt;id&gt;</span>generate-sources<span class="nt">&lt;/id&gt;</span>
<span class="nt">&lt;phase&gt;</span>generate-sources<span class="nt">&lt;/phase&gt;</span>
<span class="nt">&lt;configuration&gt;</span>
<span class="nt">&lt;wsdlOptions&gt;</span>
<span class="nt">&lt;wsdlOption&gt;</span>
<span class="nt">&lt;wsdl&gt;</span>src/main/resources/wsdl/BookService.wsdl<span class="nt">&lt;/wsdl&gt;</span>
<span class="nt">&lt;/wsdlOption&gt;</span>
<span class="nt">&lt;/wsdlOptions&gt;</span>
<span class="nt">&lt;/configuration&gt;</span>
<span class="nt">&lt;goals&gt;</span>
<span class="nt">&lt;goal&gt;</span>wsdl2java<span class="nt">&lt;/goal&gt;</span>
<span class="nt">&lt;/goals&gt;</span>
<span class="nt">&lt;/execution&gt;</span>
<span class="nt">&lt;/executions&gt;</span>
<span class="nt">&lt;/plugin&gt;</span>
</code></pre></div> </div>
<p>This plugin integrates itself into your Maven build process, so whenever you run a Maven build of your application, it will create and compile Java classes from the WSDL file.</p>
<p>Alternatively, to generate the classes whenever you want, type <code class="highlighter-rouge">mvn generate-sources</code>. The default location for the generated source files is in <code class="highlighter-rouge">target/generated-sources/cxf</code>.</p>
</li>
<li>
<p>Write a Java method to build your request message.</p>
<p class="notice notice--secondary"><strong>Need to generate the classes?</strong> If you haven’t generated the classes from your WSDL, you’ll probably want to do that now, so that you can use code autocompletion in your IDE. Run <code class="highlighter-rouge">mvn generate-sources</code>. In IntelliJ, right-click the <code class="highlighter-rouge">target/generated-sources/cxf</code> directory and choose <em>Mark As Generated Sources Root</em>.</p>
<p>Since <a href="/articles/bean-return-value/">Camel works well with Java beans</a>, it makes sense for us to write a simple Java method that creates a new Object using the classes created by the <code class="highlighter-rouge">cxf-codegen-plugin</code>.</p>
<p>You should start by creating a Java object of the class that represents your service’s request type, e.g. <code class="highlighter-rouge">GetBookRequestType</code>, or <code class="highlighter-rouge">AddBookType</code>.</p>
<p>Then it’s a simple case of writing a Java class, with a single method to create and return an object of that type:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">GetBookRequestBuilder</span> <span class="o">{</span>
<span class="kd">public</span> <span class="n">GetBook</span> <span class="nf">getBook</span><span class="o">(</span><span class="n">String</span> <span class="n">id</span><span class="o">)</span> <span class="o">{</span>
<span class="n">GetBook</span> <span class="n">request</span> <span class="o">=</span> <span class="k">new</span> <span class="n">GetBook</span><span class="o">();</span>
<span class="n">request</span><span class="o">.</span><span class="na">setID</span><span class="o">(</span><span class="n">id</span><span class="o">);</span>
<span class="k">return</span> <span class="n">request</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div> </div>
<p>Now that this class is written, it can be called from a Camel route.</p>
</li>
<li>
<p>Invoke your <a href="/articles/bean-return-value/">Java bean from a Camel route</a>, to create your request message.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">from</span><span class="o">(</span><span class="s">"direct:start"</span><span class="o">)</span>
<span class="o">.</span><span class="na">setBody</span><span class="o">(</span><span class="n">constant</span><span class="o">(</span><span class="s">"12345"</span><span class="o">))</span>
<span class="o">.</span><span class="na">bean</span><span class="o">(</span><span class="n">GetBookRequestBuilder</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
<span class="c1">// rest of the route goes here...</span>
</code></pre></div> </div>
<p>This code first sets the message body to <code class="highlighter-rouge">12345</code>. Then, when the <code class="highlighter-rouge">GetBookRequestBuilder</code> bean is invoked, Camel uses <em>bean parameter binding</em> to pass the value <code class="highlighter-rouge">12345</code> as the argument to the <code class="highlighter-rouge">build(String id)</code> method.</p>
<p>The method is invoked, returning an Object of type <code class="highlighter-rouge">GetBook</code>, for a customer ID of <code class="highlighter-rouge">12345</code>.</p>
</li>
<li>
<p>Finally, invoke the SOAP service using the CXF component.</p>
<p>You first need to configure which SOAP operation you’re invoking, by setting the value in a Header:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">.</span><span class="na">setHeader</span><span class="o">(</span><span class="s">"operationName"</span><span class="o">,</span> <span class="n">constant</span><span class="o">(</span><span class="s">"GetBook"</span><span class="o">))</span>
</code></pre></div> </div>
<p>Then, you can use the CXF component using the <code class="highlighter-rouge">cxf://</code> endpoint, like this:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">.</span><span class="na">to</span><span class="o">(</span><span class="s">"cxf://http://localhost:8423/test/BookService"</span>
<span class="o">+</span> <span class="s">"?serviceClass=com.cleverbuilder.bookservice.BookService"</span>
<span class="o">+</span> <span class="s">"&amp;wsdlURL=/wsdl/BookService.wsdl"</span><span class="o">)</span>
</code></pre></div> </div>
<p>Where <strong>serviceClass</strong> is the class that the <code class="highlighter-rouge">cxf-codegen-plugin</code> created that represents your SOAP service, and <strong>wsdlURL</strong> is the path to your WSDL.</p>
</li>
</ol>
<h3 id="example">Example</h3>
<p>If you’ve followed the steps above, your final Camel route should look something like this:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">from</span><span class="o">(</span><span class="s">"direct:start"</span><span class="o">)</span>
<span class="o">.</span><span class="na">setBody</span><span class="o">(</span><span class="n">constant</span><span class="o">(</span><span class="s">"12345"</span><span class="o">))</span>
<span class="o">.</span><span class="na">bean</span><span class="o">(</span><span class="n">GetBookRequestBuilder</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
<span class="o">.</span><span class="na">setHeader</span><span class="o">(</span><span class="n">CxfConstants</span><span class="o">.</span><span class="na">OPERATION_NAME</span><span class="o">,</span>
<span class="n">constant</span><span class="o">(</span><span class="s">"GetBook"</span><span class="o">))</span>
<span class="o">.</span><span class="na">setHeader</span><span class="o">(</span><span class="n">CxfConstants</span><span class="o">.</span><span class="na">OPERATION_NAMESPACE</span><span class="o">,</span>
<span class="n">constant</span><span class="o">(</span><span class="s">"http://www.cleverbuilder.com/BookService/"</span><span class="o">))</span>
<span class="c1">// Invoke our test service using CXF</span>
<span class="o">.</span><span class="na">to</span><span class="o">(</span><span class="s">"cxf://http://localhost:8423/test/BookService"</span>
<span class="o">+</span> <span class="s">"?serviceClass=com.cleverbuilder.bookservice.BookService"</span>
<span class="o">+</span> <span class="s">"&amp;wsdlURL=/wsdl/BookService.wsdl"</span><span class="o">)</span>
<span class="c1">// You can retrieve fields from the response using the Simple language</span>
<span class="o">.</span><span class="na">log</span><span class="o">(</span><span class="s">"The title is: ${body[0].book.title}"</span><span class="o">);</span>
</code></pre></div></div>
<h2 id="full-working-demo">Full working demo</h2>
<p><strong>See a demo Camel route which invokes a SOAP service using a WSDL.</strong> Click on the button below to see a full example:</p>
<p><a href="https://github.com/monodot/camel-demos/tree/master/simple-tests/src/main/java/com/cleverbuilder/cameldemos/scenarios/invokesoapservice" class="button btn btn--info btn--large">Get the example code</a> on GitHub</p>
<hr />
<p>Got feedback about this article, or questions about working with SOAP services in Camel? Post them in the comments below! Cheers!</p>
<p><em>Photo by <a href="https://unsplash.com/photos/vF9AtbfvFoM?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Letizia Bordoni on Unsplash</a>.</em></p>Tom DonohueHow to use Camel's CXF component to invoke a SOAP serviceThe Guide to Java Frameworks2018-06-28T10:00:00+01:002018-06-28T17:43:00+01:00https://tomd.xyz/java-frameworks<p>Which frameworks and libraries do you need to know as a Java developer?</p>
<p>Are you just about to graduate? Maybe you might be wondering which Java frameworks you should learn.</p>
<p>Here’s my guide to the most popular Java frameworks in use today.</p>
<p>This guide will help you decide which frameworks to study, to kickstart your growth as a modern Java software developer.</p>
<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fas fa-file-alt"></i> On This Page</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#you-cant-learn-everything" id="markdown-toc-you-cant-learn-everything">You can’t learn everything</a></li>
<li><a href="#the-frameworks" id="markdown-toc-the-frameworks">The frameworks</a> <ul>
<li><a href="#spring" id="markdown-toc-spring">Spring</a> <ul>
<li><a href="#spring-boot" id="markdown-toc-spring-boot">Spring Boot</a></li>
</ul>
</li>
<li><a href="#play" id="markdown-toc-play">Play</a></li>
<li><a href="#grails" id="markdown-toc-grails">Grails</a></li>
</ul>
</li>
</ul>
</nav>
</aside>
<p>But first, I want to share a personal story!</p>
<h2 id="you-cant-learn-everything">You can’t learn everything</h2>
<p>When I first started as a Java developer, I wanted to learn absolutely everything going.</p>
<p>I experimented with most of the frameworks in the list below, and I also tried some frameworks that no longer exist.</p>
<p>At the start, I thought I needed to know everything in order to get access to the most jobs.</p>
<p>But I didn’t get far beyond the simple “Hello world” examples, and so I knew only a little of each framework.</p>
<p>I soon found out that <strong>you don’t need to know everything</strong>. Java has a huge ecosystem of frameworks and tools, and you can never possibly know them all.</p>
<p>In fact, when you look for Java developer jobs, you will usually find that they are focused on a certain set of frameworks.</p>
<p>This is because most companies settle on a set of tools and frameworks that they use in-house. This helps standardise their applications, and makes them easier to maintain (instead of using a whole bunch of different frameworks all the time).</p>
<p>In this guide, you’ll learn about the top Java frameworks and how you can start to master them.</p>
<h2 id="the-frameworks">The frameworks</h2>
<p>Here’s the top Java frameworks and their vital stats:</p>
<table>
<thead>
<tr>
<th> </th>
<th>Created</th>
<th>GitHub stars</th>
<th>Supported by</th>
<th>Licence</th>
</tr>
</thead>
<tbody>
<tr>
<td>Spring</td>
<td>2005</td>
<td>21,945</td>
<td>Pivotal</td>
<td>Apache 2</td>
</tr>
<tr>
<td>Play</td>
<td>2007</td>
<td>10,545</td>
<td>Lightbend</td>
<td>Apache 2</td>
</tr>
<tr>
<td>Grails</td>
<td>2005</td>
<td>2,337</td>
<td>OCI</td>
<td>Apache 2</td>
</tr>
</tbody>
</table>
<h3 id="spring">Spring</h3>
<p>Spring isn’t just a framework. It’s an entire ecosystem of tools and libraries for Java developers. You can build lots of different styles of Java applications with Spring.</p>
<p>It starts with Spring Framework. This is a framework for developing Java applications which follow an approach called <em>dependency injection</em>. This is a way of structuring your application so that the components (your Java classes) can be developed and tested independently, and easily swapped. <!-- (TODO - definition) --></p>
<!-- TODO - illustration of components being wired/injected to each other) -->
<p>Spring Framework incorporates <em>Spring MVC</em> (Spring’s framework for web applications), plus many other core technologies, for things like: data validation, internationalisation, testing and database support.</p>
<p><strong>Spring facts:</strong></p>
<ul>
<li>Backed by: <a href="https://pivotal.io/">Pivotal Software</a></li>
<li>Licence: <a href="https://choosealicense.com/licenses/apache-2.0/">Apache 2 Licence</a></li>
<li><a href="https://github.com/spring-projects/spring-framework">GitHub stars</a>: 21,945</li>
</ul>
<h4 id="spring-boot">Spring Boot</h4>
<p>Spring Boot is a sub-project of Spring. It’s a <em>Java application in a box</em>. It’s a template and components that allow you to get started building Spring applications quickly, without you having to write a lot of code.</p>
<p>Spring Boot calls itself an <em>opinionated framework</em>. This means that Spring Boot provides a standard way of using different libraries together. It pre-configures many things for you, using a default approach.</p>
<p>It is very easy to jump into Spring Boot, and there are <a href="https://spring.io/guides">tons of tutorials online</a>. Thanks to its tight integration with lots of Java libraries, Spring Boot is probably the fastest way to get started building a web application in Java.</p>
<h3 id="play">Play</h3>
<p>Play is a <em>reactive</em> framework for building web applications for the Java Virtual Machine (JVM).</p>
<p>Like Spring, it has a Model-View-Controller (MVC) architecture, but it doesn’t use Java Servlets to serve web applications. Instead, it uses its own system for handling requests, which is built on top of the <em>Akka HTTP</em> project.</p>
<p>Play is also stateless. This means that no <em>session data</em> is stored about the user on the server (unlike other, classic Java web architectures).</p>
<p>It’s a little bit more complex for beginners to understand, because <a href="https://blog.redelastic.com/what-is-reactive-programming-bc9fa7f4a7fc">reactive programming</a> isn’t taught everywhere (yet). But because it’s asynchronous, <a href="https://www.lightbend.com/blog/why-is-play-framework-so-fast">it can be really fast</a>.</p>
<p>Play is available for both Java and Scala.</p>
<p><strong>Play framework facts:</strong></p>
<ul>
<li>Since: 2007</li>
<li>Backed by: <a href="https://lightbend.com/">Lightbend</a></li>
<li>Licence: <a href="https://choosealicense.com/licenses/apache-2.0/">Apache 2 Licence</a></li>
<li><a href="https://github.com/playframework/playframework">GitHub stars</a>: 10,545</li>
</ul>
<h3 id="grails">Grails</h3>
<p>Grails is a web framework that uses the <a href="http://www.groovy-lang.org/">Groovy language</a>. (It was originally called <em>Groovy on Rails</em>, in <em>homage</em> to the Ruby framework <em>Ruby on Rails</em>).</p>
<p>The Groovy language is a lightweight scripting language that’s pretty easy for non-Java developers to pick up. But since Grails is built on Java, you can use existing libraries or any custom components that you wish.</p>
<p>There’s also a range of <a href="http://plugins.grails.org/">plugins available for Grails</a>.</p>
<p>Since version 3, Grails runs on Spring Boot. So you’ll need to understand a little of Spring Boot first.</p>
<p><strong>Grails facts:</strong></p>
<ul>
<li>Around since: 2005</li>
<li>Backed by: <a href="https://objectcomputing.com/products/grails/">OCI</a></li>
<li>Licence: <a href="https://choosealicense.com/licenses/apache-2.0/">Apache 2 Licence</a></li>
<li><a href="https://github.com/grails/grails-core/">GitHub stars</a>: 2,337</li>
</ul>Tom DonohueA guide to the most common Java frameworks in 2018 - what they're for, how to use them and how to get started.Infinispan Query DSL: quick example2018-06-13T09:00:00+01:002019-08-15T21:22:21+01:00https://tomd.xyz/blog/infinispan-query-dsl<p><strong>How do you use the Query DSL in Infinispan? How do you search for objects in the in-memory database that have certain properties?</strong></p>
<p>First add the <code class="highlighter-rouge">infinispan-query</code> Maven dependency:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;dependency&gt;</span>
<span class="nt">&lt;groupId&gt;</span>org.infinispan<span class="nt">&lt;/groupId&gt;</span>
<span class="nt">&lt;artifactId&gt;</span>infinispan-query<span class="nt">&lt;/artifactId&gt;</span>
<span class="nt">&lt;version&gt;</span>${infinispan.version}<span class="nt">&lt;/version&gt;</span>
<span class="nt">&lt;/dependency&gt;</span>
</code></pre></div></div>
<p>Then you need to add annotations to the fields that you’d like to index on your object. You can add annotations on <strong>both fields and methods</strong>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">org.hibernate.search.annotations.Field</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.hibernate.search.annotations.Indexed</span><span class="o">;</span>
<span class="nd">@Indexed</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">Worker</span> <span class="o">{</span>
<span class="kd">private</span> <span class="n">Address</span> <span class="n">workAddress</span><span class="o">;</span>
<span class="kd">private</span> <span class="n">Address</span> <span class="n">homeAddress</span><span class="o">;</span>
<span class="nd">@Field</span>
<span class="kd">public</span> <span class="n">String</span> <span class="nf">getHomeAddressLine1</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">homeAddress</span><span class="o">.</span><span class="na">getLine1</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>By default, this field will now be searchable using the property <code class="highlighter-rouge">homeAddressLine1</code> (yes, getter methods are resolved to camelCase names):</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">QueryFactory</span> <span class="n">qf</span> <span class="o">=</span> <span class="n">Search</span><span class="o">.</span><span class="na">getQueryFactory</span><span class="o">(</span><span class="n">cache</span><span class="o">.</span><span class="na">getCache</span><span class="o">(</span><span class="s">"yourCacheName"</span><span class="o">));</span>
<span class="n">Query</span> <span class="n">q</span> <span class="o">=</span> <span class="n">qf</span><span class="o">.</span><span class="na">from</span><span class="o">(</span><span class="n">Worker</span><span class="o">.</span><span class="na">class</span><span class="o">)</span>
<span class="o">.</span><span class="na">having</span><span class="o">(</span><span class="s">"homeAddressLine1"</span><span class="o">).</span><span class="na">eq</span><span class="o">(</span><span class="s">"1 High Street"</span><span class="o">)</span>
<span class="o">.</span><span class="na">or</span><span class="o">(</span><span class="n">qf</span><span class="o">.</span><span class="na">having</span><span class="o">(</span><span class="s">"contractType"</span><span class="o">).</span><span class="na">eq</span><span class="o">(</span><span class="s">"Full_time"</span><span class="o">))</span>
<span class="o">.</span><span class="na">build</span><span class="o">();</span>
<span class="n">List</span><span class="o">&lt;</span><span class="n">Worker</span><span class="o">&gt;</span> <span class="n">list</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="na">list</span><span class="o">();</span>
</code></pre></div></div>Tom DonohueA quick example of using the Query DSL in the Infinispan in-memory cache