Monday, May 19, 2014

It's out!

I am pleased to announce, that my screencast: Building Web Applications with Clojure has been released by Packt Publishing. It is an excellent resource to get you started with developing Clojure web applications or just to refresh existing knowledge if you got your feet wet already, but still feel like some general understanding is needed.

Example codes & sample video

Screencast contents

As there is a number of web frameworks/approaches in Clojure and usually it is up to a programmer to combine them into one working application, this course introduces viewers to basic mechanisms and techniques used when developing web applications with Clojure - such as Ring handlers, requests, responses and middleware.

Then, we learn how to access external resources, such as PostgreSQL and MongoDB. After that, ClojureScript is briefly covered and finally some popular web libraries: Compojure, Liberator, Hiccup and Enlive are covered.

There's also another new library referenced - crypto-random. It is a useful tool when it comes to generating secure random identifiers - something that will come in very handy when creating new meetups or invitations.

The get-mongo-url function returns mongo db url either from environment or from default development setup. In real-life scenario, the connection options will be much more complex, including username&password, failover options, etc.

It is also noteworthy, that we don't have to manage MongoDB connections ourselves - the underlying libraries are taking care of that for us.

The initialize function is referenced in lein-ring plugin configuration and will be invoked upon ring application initialization.

Accessing the data with MongoDB

To separate our application from underlying persistence layer, we will create seperate service namespaces for basic data integration. Fortunately, most of the work is done by congomongo library and we can use native Clojure data structures in all layers.

New meetup widget

First of all, we have widget definition. As you can see below, it is a standard Clojure function, and it is invoked explicitly from Clojure code - ganelon.tutorial.pages.routes/meetup-layout to be exact.

We use Hiccup to generate HTML form, styled by Bootstrap. One significant difference is that we don't use [:form] HTML tag, but rather ganelon.web.widgets/action-form function. This function allows us to reference a Ganelon action to be invoked upon form submission - client side.

Please observe, that the main meetup details widget is referencing sub-widgets as functions: meetup-edit-form-widget, meetup-times/meetup-times-widget and meetup-invitations/meetup-invitations-widget. This way, we can decompose our web application into a set of widgets with clearly defined dependencies.

As we have more possibilities here than just meetup creation, we can make the form more dynamic. For example, we will capture :onchange JavaScript events for meetup title and place to update the fields in MongoDB without save button. Even more, we will display a confirmation label next to a saved field.

In meetup edition widget definition, we will add standard HTML attributes, referencing our Ganelon action (meetup-title-update) - but client-side! As Ganelon has information about defined actions, there is an interface available as a standard Compojure route (by default under /ganelon/actions.js), publishing all these functions to be referenced in JavaScript client-side:

You are welcome to browse the operations available in ganelon.web.ui-operations namespace - but please note, that it is also very easy to define your own, more fitting for you apps specific needs!

Meetup times widget allows us to add new invitations and to list and manage existing ones - including invitation confirmation status for each time and invitation. The whole implementation is available in ganelon.tutorial.widgets.meetup-times namespace.

Please observe, that we are using widgets in loop here - as ganelon.web.widgets/with-div macro binds widget each time to a unique and random value, we can tons of instances of the same widget side by side.

We can also define a function which will return Ganelon UI operation refreshing times list - to be invoked externally (e.g. from invitations list widget):

One thing worth noticing is that we put operation refreshing meetup times widget upon invitation creation or deletion. Instead of accessing widget functions for meetup times directly, we call a wrapper function which encapsulates all the necessary logic - e.g. DIV identification and returns ready to use Ganelon UI operation.

Invitation widget

Invitation widget is used by meetup attendees and allows them only to confirm or reject their presence at the meetup at given time. Whole implementation is provided withing ganelon.tutorial.widgets.invitation-details namespace.

Please note, that we are re-using meetup-times/meetup-times-list-widget to provide meetup times and to allow the attendee to confirm or negate his/her presence at the meetup. We do so by providing additional parameter to function invocation, limiting displayed invitations only to the currently used.

Summary

In this blog post we have learned, that it is very easy to build dynamic, AJAX-oriented web application entirely in Clojure, server side. Even better, such approach allows us to re-use existing components in different scenarios. We don't have to stick with our application being single-page and we are totally 100% SEO friendly in such approach, as widgets are used both in standard HTML page rendering as in updates to the page itself.

In the next post I will show how to add middleware for security handling and how to define custom Ganelon UI operations in JavaScript.

Creating Ganelon project

Ganelon is just a standard Leiningen/Maven/Gradle/etc. dependency, so you can just use lein new to start a new project or set up the project.clj file and src/, script/ and resources/ directories manually.

In either way, it is important that the project.clj file contains a dependency on ganelon 0.9.0 library. What is worth mentioning, is that ganelon comes with dependencies on lib-noir, Compojure & hiccup, so in simple cases, it is enough to just add this one dependency when it comes to Clojure web apps.

Ganelon 0.9.0 is deployed to clojars, so it will get fetched alongside other Clojure libraries.

As we can see, there is only one dependency other than Clojure 1.5.1 - it is for ganelon 0.9.0. There is also a lein-ring configuration with a pretty standard settings - an initialize function, which at this moment does nothing but will establish a connection pool for MongoDB and a handler reference.

Handler definition

Our handler will be using Ganelon's simple mechanism for establishing routes - and we will mix app-handler from Ganelon with standard Ring middleware.

You can add your own handlers and middleware here and it will just work. If you don't want to or cannot use
Compojure routes, you can just skip this step altogether and use raw Ring handlers from ganelon.web.actions package.

Please also note, that we are using a convienent ring.middleware.reload/wrap-reload middleware, which will reload all the changes to Clojure files in specified directories - src/ganelon/tutorial/pages in this case.

Defining the Hiccup templates

The simplest way (but sometimes not the most effective) to achieve common layout and HTML code for pages with Hiccup is to define them as functions, accepting page content as parameters:

The ganelon.web.dyna-routes/defpage macro simply creates compojure.core/ANY route and adds it to a list maintained in the ganelon.web.dyna-routes namespace. The routes can be grouped for easier management and middleware can be defined in a similar way. More information is available on the Routing page of Ganelon project website.

Running the project

After that, we have two general ways to run our project and there is nothing specific to Ganelon about them:

Monday, August 19, 2013

What is Ganelon?

Ganelon is a microframework of mine, built to ease the pain of using AJAX in Ring/Compojure based Clojure web applications.

The microframework is not intended as an end-to-end framework, solving all the problems of building web applications in Clojure, but rather as a tool that solves one particular problem in an elegant and simple way.

Other problems, such as routing, templating or integration with underlaying HTTP(S) server architecture have their own, well established solutions. For example there is Compojure or Mustache for routing, Hiccup or Clabango for templating, etc.

For a full list, you can go to http://www.clojure-toolbox.com/. Important thing to note is that by using Ganelon you are still free to apply any other Ring-compliant or client-side libraries - Ganelon simply enhances your application's capabilities, not forces you down some narrow path.

How does Ganelon work?

Ganelon is also different from modern, JavaScript/ClojureScript based web frameworks, as it gives a control over user interface behaviour directly to a code running server-side. JavaScript layer is very thin and provides a set of generic operations; the layer can of course be easily extended as we will see in the tutorial. More information is available in the documentation.

As to my knowledge, there are no other Clojure frameworks built this way, but similar principle is used for example in Vaadin or Wicket (Java) or in Weblocks (Common LISP). The main difference is that Ganelon is not keeping the information about widget state.

Thursday, April 18, 2013

Ganelon - a micro-framework supporting server-side oriented AJAX web applications in Ring/Clojure has been released in version 0.9.0 - first publicly available non-SNAPSHOT.In addition to that, an interactive tutorial app (meetings management with MongoDB) has been launched at http://ganelon-tutorial.tomeklipski.com. Source codes for it can be found in GitHub repository: https://github.com/tlipski/ganelon-tutorial. In the following weeks, this tutorial app will be used as a base for a tutorial blog series.

Using Ganelon

To use Ganelon in your Compojure/Ring/Clojure web application, simply add the following dependency to your project.clj file:

Production use

Ganelon is used to build user interface for Daily Social - social news aggregator delivering top stories to Pocket or Readability. So it is used in production for almost half a year. The demo and interactive tutorial sites are written using Ganelon as well.

I think that the best thing to do now - is to give a Ganelon a try and see how it works out for yourself.

Thursday, April 11, 2013

In the following blog post, I will show how to install La Clojure plugin, import project from leiningen to IntelliJ IDEA, how to interact with REPL and finally how to use IDEA's debugger to debug Clojure code run from REPL. These features make Clojure development with IDEA a real pleasure.

Installation of La Clojure plugin

To use Clojure with IntelliJ IDEA, we have to add La Clojure plugin. To do that, we open File / Settings / Plugins from a menu, than click on Browse repositories... button at the bottom of the Settings window (highlighted in red).

Then, we select La Clojure from plugins list (we can use filter in the right top corner of the window), right click on the item and select Download and Install command:

Upon leaving the Settings window, IDEA will ask if we want to restart it - the plugin will not work unless we do.

Importing leiningen project

With the La Clojure plugin ready and active, we can import a leiningen project into IntelliJ IDEA. All we have to do is to generate appropriate Maven pom.xml file:

One thing that has to be done manually is adding Clojure Facet to project modules. To do that, we simply select File / Project structure from a menu, then navigate to our module in Modules tab, click on a + sign and finally select Clojure:

Starting the REPL

With Maven project loaded successfully, IDEA will fetch dependencies which aren't already in Maven local repository. With these libraries fetched, we can start Clojure REPL for our project by selecting Tools / Start Clojure Console from a menu.

We can also use the keyboard shortcut - Ctrl-Shift-D by default in the newest La Clojure version.

The REPL will set a classpath for all of our libraries and sources referenced in a current module:

Interacting with REPL

To load a current Clojure file to REPL by a load-file function, all we have to do is to select Tools / Clojure REPL / Load file to REPL or use a keyboard shortcut - Ctrl-Shift-L by default.

We can also use Tools / Clojure REPL menu to:

Run selected text in REPL

Execute last S-Expression in REPL

Run top S-Expression in REPL

Debugging with REPL

With IntelliJ IDEA we can connect a remote debugger to a REPL, allowing us to debug our Clojure code.

Step 1: we have to create a Remote Debugger profile using Run / Edit Configurations... from a menu.
To add a Remote Debugger, we have to click on the plus '+'sign and select 'Remote' configuration type:

We can adjust the settings or leave them as default. Most importantly, we have to copy command line arguments for running remote JVM, for example:

-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005

The default value for port is 5005, but it can be adjusted in case there is another process already listening on that port.

It is also convienient to name this Remote Debugging session as 'Clojure REPL Debugger' for example.

Ganelon is fully Ring compatible and based on Compojure and lib-noir. Being a AJAX focused, it is not a direct replacement for Noir. It can rather ease the pain of handling dynamic page requests in any Ring-based web application. The execution model is heavily influenced by Weblocks or Vaadin, but without session-statefulness and out-of-the-box rich user interface:

Certain parts of the page can be scoped as Widgets (for example using id attribute in HTML or ganelon.web.widgets.with-div utility macro) and referenced by id in Actions, and updated by Operations. Widgets can reference Actions to be invoked.

Actions are invoked as XHR requests and return a set of Operations to be performed client-side. Actions are in fact simple Ring handlers.

Operations provide abstraction layer over client-side JavaScript execution - e.g. using Bootstrap Modal or just updating part of DOM tree (a Widget or any other) or open certain URL in a browser window - or just anything else that has a simple integration layer provided.

Monday, March 18, 2013

In the following blog post, I have highlighted the matter of embedding Activiti BPMS inside a Liferay Portal. With Activiti Explorer built in Vaadin and Spring, it is fairly easy to use entire components in a totally different environment. Proof of concept fork with Tasks and Admin portlets is available in https://github.com/tlipski/Activiti/tree/master/modules/activiti-portlets.

Porting Activiti Explorer to Liferay Portal

Activiti is a popular Open Source, BPMN2.0-compliant BPMS that provides a great overall package. Not only there is a library that can be embedded inside your application, but Activiti also comes with an Explorer web application, which provides an ergonomic user interface for basic operations:

But, Activiti is "just" a BPMS - you can run your processes with it, maybe even connect to Active Directory, Alfresco or ESB - but if you want to embed it in your intranet/extranet portal, you are almost on your own.

Please note, that embedding Activiti in portal is not the same as using it to manage portal assets - for example, as a replacement for Liferay's Kaleo Workflow. The matter discussed here is more focused on using all of Activiti features, including user interface.

Luckily, this is when frameworks used in Activiti pay off - Activiti Explorer is written in Vaadin 6 and Spring. With component oriented approach from Spring and UI modularity encouraged by Vaadin, it is fairly easy to port this application to Liferay. We can extract entire components from Activiti Explorer (e.g. Tasks panel) and use them in a whole new environment.
Extracting and re-using entire components requires much more work in a classical MVC approach and even may not always be possible. Also, portlet support in Vaadin makes a whole task much more pleasant.

Necessary steps

"Fairly easy" does not mean without any effort. There are differences between a standard Vaadin application and a portletized one:

Portlet descriptors are necessary

GUI should be divided into separate portlets

Spring-Vaadin integration is different due toa portal-specific request life-cycle

Navigation can be provided by portal - including friendly URLs, which makes for a more standard user experience.

In addition to that, some mechanisms used by Activiti are already provided by Liferay Portal and we need to bridge them:

Users and Groups should be managed by Liferay Portal

Authentication data should be taken from a portlet container

Mail notifications could possibly use Liferay Mail API

Some other things have to be taken into account as well:

Distribution and management of Vaadin widgetset and version - we should use Vaadin Control Panel portlet and Liferay to manage Vaadin dependencies. This requires special handling, since Activiti Explorer 5.13 utilizes dCharts Vaadin Add-on.

Vaadin theme used in portlets - which is also customized by Activiti, but should be coherent with portal's look & feel.

Current state of work

Activiti Tasks portlet, which works in a user's context as if the user would log in to Activiti Explorer app. The portlet provides full functionality, including operations as starting a new task, fetching it from a task list, displaying task's events, assignment/ownership transfers, file attachments and so on.

Activiti Admin portlet in Liferay Control Panel - most tabs seem to work fine - except Users and Groups, which should be disabled as we are using Liferay Portal to do that.

Custom portlet applications and main Window class. The interesting detail here is that with Vaadin portlets, you should not set width of components to 100%, as it will result in 0px height. Generally speaking, the height in Vaadin portlet components should be set to undefined - null, or a preset value in pixels.

Next steps?

Connecting Liferay with Activiti brings a myriad of new possibilities, but before that, some basic things need to be done:

More portlets! Reporting and Processes tabs need their respective portlets, but also "start process" portlet might be worth considering since we can manage UI contents in a more flexible way.

More testing! If anyone is interested in testing Activiti portlets in Liferay, let me know and I will provide prebuilt applications. So far, I've used Liferay 6.1 GA2 CE and H2 in-memory database for Activiti.

Friendly urls - especially for tasks resolved by id, and for interportlet communication - e.g. when starting a process.

Sunday, February 24, 2013

As there are numerous materials on how to integrate Vaadin with Spring, Spring with Liferay (using portlet dispatcher) and Vaadin with Liferay, the integration of all three technologies at once is not documented well.
In such approach, there are two major differences between Vaadin with Spring in a standard web application and portlets-based one. This post is based on Vaadin 6, but the rules itself apply
to Vaadin 7 as well.

The setup

The solution requires that we will overwrite two methods defined in com​.vaadin​.terminal​.gwt​.server​.AbstractApplicationPortlet2.
To achieve that, we have to create our own subclass:

As you can see, they are all mixed up and we cannot rely that during invocation of method javax.portlet.Portlet.init(PortletConfig config) the spring context is available.
If we want to refer to our application as a Spring bean, we have to overwrite method com​.vaadin​.terminal​.gwt​.server​.AbstractApplicationPortlet​.getNewApplication​(​PortletRequest request), and attempt to retrieve Spring context here:

But, in JSR286 portlet, the listener is not invoked as it should and portlet specification does not provide the ability to add portlet request listeners as above.
Luckily, we can overwrite the method com​.vaadin​.terminal​.gwt​.server​.AbstractApplicationPortlet​.handleRequest​(​PortletRequest request, PortletResponse response) with a code, that will set appropriate ThreadContext values:

Summary

It is fairly easy to use Spring in Vaadin portlets - but there are some differences between standard Vaadin-Spring application approach and portletized one. We can easily overcome
them by overwriting two methods in com​.vaadin​.terminal​.gwt​.server​.AbstractApplicationPortlet.

Sunday, May 6, 2012

When using congomongo library to access MongoDB in Clojure, it is sometimes required to remove a collection of items. As the example on GitHub page shows only how to remove one item, I have pasted the code here.

Monday, April 23, 2012

One big change that Activiti has introduced over jBPMv4 was to forego Hibernate ORM and use MyBatis. MyBatis is simpler and cleaner to use, but - it does not provide simple interface to perform queries that are not defined in annotations or XML files.

And even though Activiti provides great query interface, so you can do things like this:

with relative ease, there are still some limitations to the API provided by Activiti Service classes (e.g. you cannot add IN clause for task assignees).

Possible solutions

One thing that you can do, is to just skip Activiti interface, and access database directly. Surely it will work, but such approach has many downsides - you will be fetching information about Activiti processes in two different ways, you will have to map data to Activiti objects yourself, and so on.

What we have managed to do for Aperte Workflow <-> Activiti integration (in version 2.0, where API is a bit more demanding), is indeed a hack. It is not as elegant as constructing query for jBPM, but still is quite simple and manageable. It propably can be done with Spring in a similiar fashion, or with SelectBuilder - but the principle is similiar.

The hack

1. Enhance Activiti's MyBatis mapping file - by adding our own version in a different package.

The enhanced file looks almost like the original - with one exception:

As MyBatis runs inside of Activiti internals, we have to access and alter Activiti configuration mechanisms. To do that, we simply override one of the methods with our copy (which points to a new mapping file):

Activiti uses wonderful, clean and simple query objects, that provide structure to MyBatis query parameters. To introduce new parameters, all we have to do, is to enhance TaskQueryImpl class with our own:

Please note, that we are also exposing CommandExecutor object instance to access DbSqlSession.

Summary

The technique presented here would be unnecessary or much simpler if Activiti would provide external means to configure MyBatis. Maybe that is a thing, that will be available in the future versions of Activiti.

Still, even at this moment, it is fairly easy to enhance/alter Activiti's behaviour. I haven't seen any final classes (a common sight in Hibernate - e.g. org.hibernate.impl.SessionImpl) and the internals of Activiti are quite simple to understand.

on the example of Aperte WorkflowVaadinVaadin is an Open Source widget-oriented, Java Rich Internet Application web framework, which allows programmers to develop browser-based user interface entirely in Java - server-side.
That's it. This is what makes Vaadin different and this is what is game-changing in comparison to almost any other popular web framework.

100% JVM

Because everything is written in Java and runs server-side, you can do things, that were almost impossible or very painful for web frameworks before:

Have one, coherent application, with its logic seperated according to your needs - not to what the framework designers had in mind

Vaadin still allows programmers to access the browser side - but in a controlled way, using Add-ons - Vaadin Add-on Directory provides ecosystem for those. And for most applications - you can easily build them using just the standard ones, provided by Vaadin out-of-the-box.

...and more

JVM itself would be enough, but Vaadin is not only great in the idea itself, but also in implementation. The applications have eye-friendly look out of the box, you can also apply custom CSS and themes easily.
The UI is well-optimized even in comparison to client-side only frameworks - and it works on all modern browsers out of the box (IE6/7 - with some downsides too!).
There is even mobile device support with Vaadin TouchKit, Vaadin also supports JSR168/286 portals (e.g. Liferay Portal) and even Google App Engine.

Vaadin & Aperte Workflow

Aperte Workflow is an Open Source BPMS - the application that eventually is built during the introduction on BPM-class system to an enterprise. Aperte Workflow is designed with strong emphasis on modularity, re-usabiliti and extensibility of solutions created.
User interface in Aperte Workflow is built on Vaadin. The approach for BPM task UI concept in Aperte Workflow is that for each human task, user interface is built from reusable blocks or components (in Aperte Workflow terminology: widgets). Such components can be process-aware (e.g. display process instance data) or just organize other components.
With Vaadin, creating these components is a blast, even for someone new to the concept. You just code your component in Java - implementing some interfaces or extending ready-to-use classes. Vaadin widgets are there - so, for example - you don't have to worry on how to implement your own or incorporate existing suggesting combo box or grid - they are already here, well designed, optimized to work together with other combo boxes.
After coding - you just deploy/update them to a plugin mechanism. Being written entirely in Java, we can manipulate them in our plugins very easily. We don't have to build a set of JavaScripts to provide RIA experience to the users - but neither we have to refresh web page after every user action.
So the code->deploy->test->code->deploy->... cycle is very short - and still we write our application in plain old Java. I wouldn't even attempt to try to implement such thing with pure GWT or JSF.
Plugin mechanism provides integration with an Aperte Workflow web-based modeler - which is based on Signavio core, but more complex elements specific for Aperte Workflow were written in Vaadin too.
Use of Vaadin in modeler allowed us also to engineer preview mechanics to some components (more to come) - for example, Advanced Process Data widget, which provides generic-form feature (it's just one plugin, not an obligation or part of the Aperte Workflow core - you can use, you can replace it, or you can use other implementation, or just go with your own plugins) generates preview in real-time. And the preview is the same code and configuration that will be run in the real user interface for a real BPM task.
What else Vaadin has allowed us to do with relative ease, is that Advanced Process Data Widget provides JSR223 scripting support - for validation and manipulation of the configured form.

Summary

In Aperte Workflow, we have chosen Vaadin as our UI framework, not only to save ourselves from spending too much time coding the user interface. We have chosen it mainly because with Vaadin, it is possible to create much more modular RIA web application than with any other framework we have seen.
Will Aperte Workflow be possible without Vaadin. Sure - but with much bigger effort and budget. And the result would be underwhelming without Vaadin - or we would have created our own, imperfect implementation of Vaadin.More information: