Sending formatted email with inline images from #sapnwcloud

Hello, and welcome to another blog in the series Joanna Chan and I are writing about our experiences and learning building a mobile application using the SAP NetWeaver Cloud as a platform. Sorry for the hashtag or should I say mot-dièses in the title, it just means linking to it in twitter becomes easier 🙂

Long live the whiteboard but the code’s in the blog!

As has been the case with my other blogs in this series, I’ve thrown together a little video which goes through the key points of the blog and tries to explain them with the help of a few whiteboard drawings. This time however, I’ll leave my code walkthrough to the blog – so please excuse the rather copious amounts of code below.

Problem:

So, we want to be able to send a user an email confirming that they have successfully sent their time sheet entries to our cloud repository. We want this email to be nicely formatted. An image or two in the body/sig block of the email would be nice. We’d like to group all the time sheet entries for a given user into one email. So if the user sends 10 time sheet entries through, only 1 email should be sent. But if they only post 1, then only 1 should be sent.

Hundreds of emails

The first issue here is how on earth to only send one email, given that the trigger for the emails is a POST or a PUT to the time sheet entry resource?

What we need to do somehow is to gather up all the triggers/posted entries into a bucket and send the bucket-load over. What we need is some way for the first time sheet entry to put up a flag that any subsequent entries can see and gather around. Then we need some way that the flag can be brought down after a certain amount of time has passed and send the email with the details of all the entries collected to that point. Sounds reasonable (although not simple) but how on earth does one achieve that? 😕

Threads, synchronised methods and delayed action

In Java, just as in ABAP, it is possible to have asynchronous processes started from the same thread. and communicate with each other. In ABAP this in practice proves to be tricky as it means calling an RFC and communicating via polling shared memory areas. In Java, it’s a bit simpler than that, and a decent part of the language deals with just this point. Whereas in ABAP the easily addressable memory space of a program is constrained to a single process (outside of using shared memory construction), in Java this is not the case.

I have a class – UserMailSender that has a private constructor, this means that it cannot be instantiated from outside of methods within the class itself. (It also mucks up subclassing it, but that’s a different issue). That means a static method of the class must be used to create an instance. This design pattern is somewhat similar to the singleton design pattern that I love so much to hate. However, in this use case it makes sense – even if it isn’t actually a singleton. (I’m sure that this design pattern should have a name – factory was closest I could find, but it doesn’t really fit.) (Leave a comment if you know what the design pattern should be called!)

By specifying that the method is synchronized we can be sure that only one call to it will occur at a time, other calls will wait until the previous call is finished. (This does result in a bottleneck in our code, but so far this doesn’t seem to be a problem.) By further specifying within the code that a synchronised block exists we can “enqueue” , to use a more ABAP term, the handling of the list so that only one thread at a time may have access to add, retrieve or delete instances from the global list of email handlers.

It is this method that is called from the servlets (see my previous blog for more details:

When the method is called, it checks through a list of existing instances to see if a class has been instantiated for the current user id. (A HashMap or other lookup would probably be better code here than just looping through the array, but given the number of users I’m guessing would concurrently access my application, it’s probably not worth fretting about). If I find an instance, then I call the addEntry method on it. If I can’t find an instance I create one and add it to the list of existing instances.

Now, an interesting thing need to happen when we create that instance. It needs to shut itself down after a certain period of time, and remove itself from the list of instances available for users.

I create a new “thread” or asynchronous task. This thread is started using the start() method. The constructor for the thread requires that you pass it an instance that implements the Runnable interface. So I have created a private inner class which does this:

Note how the constructor of this class takes a reference to the instance of UserMailSender class that called the thread. Another important point to note is that a reference to ServletContext is being passed through to all of these methods – this is so that we can eventually pass it to the sendMail method that is called in the implementation of the run() method. The run() method tries to sleep for however long I like (I’ve used 60 secs as I think this is quite reasonable). This is possible because we have a separate process that is not blocking anything else from occurring. Just that one thread is paused. Once it has finished sleeping, the thread then (and this is quite important) removes the reference to the instance from the global list.

Like the method that fetched/created the instance in the global list, this method is synchronised both by definition and by the user of a synchronized block to ensure that only one thread at a time adds or removes entries.

Once we are sure that no more entries will be added to the current list (it is “finalised”) then we start the process of actually sending the email.

Emails using NWCloud – setting it up

Before I could send an email from NWCloud there is a bit of a work to do in order to set it up. You firstly will need an email account to send the emails from. The online documentation states that you can “ integrate your own e-mail provider (currently subject to restrictions)” so whilst I do know that Google mail users (including apps users) do work, if you’re using anything else, it might not! The online doco is pretty good, so I won’t repeat it here. But will just clarify a few things that were perhaps not as straightforward.

Testing locally

You’ll need to paste a copy of your email set-up xml into your local cloud set-up. And you’ll need to re-do this every time you patch to a new SDK level! So it is worth keeping a copy with your application.

I store a copy in my WEB-INF folder. It might be an extra piece of info to deploy to the cloud, but it’s a nice easy place to keep track of it. In the version 2.x of the NetWeaver Cloud this process is supposed to become easier I understand. Good thing too!

You’ll need to copy this file to a folder of the “Servers” project in Eclipse:

(Now wasn’t that simpler than the doco 😉 ).

Deploying to the NWCloud Trial system

The next step is to deploy also to the cloud. You’ll need to use the command line interface for this:

One thing the doco doesn’t mention is that you’ll need to add the –host parameter with a link to the trial system, otherwise it defaults to the productive NWCloud. BTW – I’ve upgraded to a later SDK since I took the screenshot if you were wondering! You’ll notice also that the content of the parameter values differ somewhat from the suggestions in the doco. All I can say is that you have proof in that there screenshot – what I entered worked!

Multipart MIME Messages

So now we have a collection of data that we want to send as an email, we have the required email account, and we uploaded that info to the cloud – how do we actually send the email? Well, before we can do that, we need to understand how to build that email.

The emails that you receive today are generally formatted as Multipart Multipurpose Internet Mail Extensions (MIME) Messages . In order to programmatically send one of these messages it helps to have a little understanding of what they are and how they are put together.

A multipart MIME message allows us to send the nicely formatted emails that most email clients support, along with images. Unlike HTML pages where the browser fetches each image individual and the user agent string can be used to redirect the user to an appropriate rendering of the content, in MIME messages all the content, including alternate renderings must all be sent at once.

For our purposes there are two important multipart subtypes – Alternative and Related.

We need to create a message that has Related parts (that is to say the formatted text of the message and an inline image) but we also need to allow for the case were the email client that is used to receive our email does not know how to display HTML formatted documents, so we need to drop back to just using plain text. This requires us to use the Alternative subtype.

There a many different MIME types that we could use to convey our formatted message, but I’ll use HTML as it’s pretty darn simple and has lots of formatting options.

Building the HTML message

I threw together some code to build the message from the user object and the list of entries that should be sent:

There was a slightly amusing (for me) exchange between our client relations person (great bloke is Leigh) and me when he offered to help with the formatting of the email, and I sent him an early version of this code and told him to go for it 😉 . Afterwards we refined terms a little – he sent me a mock-up and I did the formatting 🙂 and we think it’s not a bad result!

One of the things to note is that HTML in emails does not render the same way as HTML in your browser in every case, you may have very little control over things like margins between divs etc. So it is worth while testing and doing more testing. Don’t forget that most people read email via mobile devices, so make sure you test those too. In this example code we’ve got a very simple alternating background colour for the time sheet entries to help distinguish one from the other.

Using the javax.mail services in NWCloud – adding inline images

Please excuse the gratuitous code in this section – but I think it’s the best way to explain it! Here’s the code that I use to send my emails.

The only public method of the email sending class is what was called from our time sheet entries aggregation logic. It get a reference to the email setup that we have loaded onto our server and then creates a MIME message using the private createMimeMessage method. It then uses the references to the email setup to send the message.

Up to here it’s pretty simple really – just creating the message and setting who it’s from, to and the subject. It’s the next bit that gets tricky! I even included a picture in my code to help me understand what the heck was happening 🙂 ! Of course the inline Java formatting used in SCN doesn’t use a fixed width font, so the pictures kinda screwed up…

The content of the alternative multipart is then inserted into the body part that is then added to the main related multipart. Another body part is created to hold the footer image that I use in my email.

This is where I really had some fun! Unfortunately in NWCloud 1.x the email libs don’t quite work as they should 🙁 , so I had a bit of fun trying to get the javax.mail.internet.MimeBodyPart setDataHandler method working. End result it doesn’t 😛 and I had to try something else. Fortunately the attachFile method does work 🙂 so I found some way of getting the image file into the MIME message. However, in order to do that, I needed to get a reference to the image file itself.

Fortunately there happens to be a way to get real file references from virtual paths (the ones used in your eclipse project) through the ServletContext method getRealPath. This was why all the email aggregation logic kept a reference to this servlet context. So that the context could be used in the final email handling to get a file handle for the image file to be sent.

Some other stuff that’s worth noting, is that when calling the setContentID method the id needs to be inside carets <> otherwise it doesn’t work.

I’ve wrapped the HTML that I got for the entries with a simple <html> and <body> tags and added the attached image into the HTML with a reference to “cid:” or content-id and the GUID I created to reference the associated image.

Wrap up – key learnings, what else I’d implement

Sending an email with an image attachment, a company logo or a picture of the item you just ordered, whatever 🙂 is quite possible using the SAP NWCloud. This generates some very interesting possibilities. I certainly see a much more interactive solution in the future. (That is if we’re still using email!)

One key learning I had here is to re-examine every design choice/restriction that I’ve used/had with ABAP code. It is likely that it is either not relevant or is possible when using the NW Cloud – I just have to figure out how. That’s not to say it’s not possible with ABAP – it’s just quite often easier in the cloud.

I’m very chuffed with my solution, especially as I had not found anyone else writing about using SAP NWCloud to send emails with inline images (and even blogs on using javax.mail are relatively few)! However, something that I should have implemented, but didn’t – was some way of verifying that the email address that the user has entered is their own. A simple email verification link would have been simple to build using SAP NW Cloud, and is certainly a requirement for any productive solution like this where the user can enter their own (or if they wanted to, someone else’s) email address. (I did default the email address when the user signs in using Google – but what would be good would be to also default a “validated email address” flag.) There is all sorts of legislation regarding spam emails which I should probably try to be very careful about. In Australia, for example, any “commercial electronic message” must have a means for the recipient to unsubscribe and can be backed up by court action and fines! (Better get working on implementing that one!)

As per my previous blog, I’d like to implement some decent JUnit tests here too. If for nothing else than to give me some exercise in understanding how they could help with the development or potentially be a PITA.

Disclaimer, ramblings and what’s next

As usual, even if I make outrageous claims like “here is the code I used”, it’s all for use at your own risk, I don’t promise that any of it works or won’t completely bugger up your system. Likewise, none of my ridiculous ramblings should be taken to be indicative of the opinion of my wonderful and very patient employer (who has had to deal with me not being able to come into the office again today as my poor little girl isn’t feeling too well!) If you do reuse my code, some attribution would be appreciated but really what is code development if not elaborate copy/paste. These are my opinions, ramblings and mistakes. I’ve tried not to offend, imply anything which might offend or eat fresh raspberries with whipped cream whilst putting together this blog. I’m pretty sure I’ve failed on one point there.

Next blog will likely be the use of CORS to allow successful connections from mobile devices to the cloud. But we’ll see! (might be working on that unsubscribe button! 😉 )

In the meantime, and in my own time, I’ve put together a little page on www.wombling.com/BunkBed which shows that I don’t just live, eat, sleep and dream SAP stuff – there is another side! (so there Jo! I know it’s not quite the same as frocking up and writing a fashion blog, but it counts? cc John Moy.)

@jwenna @wombling Yes, but in between his technical #SCN blogs did @wombling find time to frock up and write a fashion blog? I think not!

If you liked my blog and might even have found the code useful, please take a sec, log in and rate it and even click on the like button. So far Jo is getting far more likes than me and I’m nothing if not competitive. So if you want to have a little fun at my expense, please go read her blogs, rate them and like them and don’t bother with mine 😉 .

In no event may Customer resell the Service or any component of the Service to any third party, except that Customer is entitled to allow End Users to remotely access the Customer Applications on the Platform as permitted by this Supplement. Customer may not use the Service or any Customer Application to generate, distribute, publish or facilitate unsolicited mass email, promotions, advertisings or other solicitations (“spam”).

Thanks D, (if I use the longer shortened version of your name I see stars!)

yes it certainly is important not to be sending spam mails out – even accidentally. One thing that is interesting about the Australian law is that even one single unsolicited email is considered spam – there is no need for it be a “mass email”.

In the link I posted above, Tiger Airways were fined $110,000 for having a un-register link that didn’t work – although in most cases businesses are just warned and told to fix their systems. Still, it’s not the kind of publicity that you’d want!

Thanks for reading and sharing your knowledge of the T&C – I remember that series of blogs you did about all the Supplemental Terms and Conditions and your thoughts on them well.

Might be worth noting after some feedback from my testers, I added the following code to ensure that the timesheets being sent to the users were actually sorted – as the logic I have above doesn’t actually sort them before putting them into the HTML – it’s whichever one gets there first.