Tricks and Tips on Paypal Integration

After releasing the DoubleClou ICE, I finally got some time to write what I had experienced with Paypal as a user and developer. Because the DoubleCloud ICE is productivity tool that sells within $100, I want people to buy it online quickly and easily. Given Paypal’s #1 position in online payment, I turned to Paypal without much thinking. After that I had gone through all the integration process from initial research to bring the site to production.

Overall the experience is not as smoothly as I had expected, but in the end it worked. I think it may be worthwhile to share with you what I had learned along the way. Hope it can save you research and development time.

Time to learn how to "Google" and manage your VMware and clouds in a fast and secure

Before we get into the technology part, let’s first take a look at what I wanted to do with Paypal. Unlike normal digital goods that can be downloaded and consumed, my software is free to download and install. The limitation is on the scale and certain feature. To unlock the full strength of the software, a license key needs to be imported. This license key depends on the user’s email address therefore cannot be generated before hand. I guess it can be generated as a generic key, but I chose to have a personal key for each user. The license key must be generated upon payment. In other words, after a user pays for the software, she or he should get a license key via email right away.

I thought about using online store to sell software, but the above process from real-time license key generation cannot be easily supported. If it’s just an electronic book, I think most online software stores can handle well. Even not, you can probably use normal Paypal button for this in a fairly straight-forward way.

After exploring a few sites, I decided to do the integration by myself – it’s a good chance to try something new anyway.

Choice of SDKs/APIs

To get what I wanted, I need to use a PayPal feature called IPN (Instant Payment Notification). On a very high level, it works like this. On the Paypal button, you can specify a URL to which Paypal can post transaction details after a user’s payment goes through. The service behind the URL has to acknowdge it in Paypal’s defined protocol, or Paypal will keep trying it until a limit is reached.

Althogh there isn’t much secret how the whole process works, it does not make sense to re-implement it because I believed it’s such a common case there must be something already. I was correct, but not fully.

It turned out that Paypal has a couple of APIs/SDKs: the so called classic SDK, and the REST APIs. The latter one is supposed to be overtaking the former one which would be deprecated. But the REST APIs has less features than the classic one. In the end, I settled down with the classic SDKs. My guess is that it won’t be deprecated any time soon at least before it’s caught up by REST APIs on feature set.

Classic APIs

The classic SDK is hosted at GitHub (https://github.com/paypal/merchant-sdk-java). You can download it or use the following dependency in Maven pom.xml file:

What is really needed is the paypal-core-1.5.1.jar (the version might be different). If you don’t use Maven, just copy the jar file to wherever your builder can find.

All these are easy, but the real problem came in when I tried to use it. What APIs to use and how? In the IPN case, a Java Servlet is needed for receiving information from Paypal and confirming it.

I was thinking there should be a code sample that could be easily modified for my case. After searching for a while, I could not find a complete sample. There are some information and code snippet here and there, but there is no sample that can run out of box. So I had to spend extra hours to weave different pieces together for a complete servlet.

Then, a potentially even bigger problem came in: how should I test it without really paying the money? Even you move money from left hand to right hand, you have to pay Paypal transaction fees. You could cancel a transaction, but a small amount cannot be returned.

Sandbox Testing and IPN Simulator

That is where Paypal Sandbox comes in to help. They actually have IPN simulator (https://developer.paypal.com/developer/ipnSimulator) which made life a lot easier. In the simulator, you can modify whatever value you want to use and upon clicking a button, the Sandbox will send a fake request to the servlet. Relieved: no need to pay. But there is another limitation with the simulator which I would discuss soon.

When I was done with IPN simulator, I tried to create a test button and hit a Paypal button. Somehow I could not create test button – whenever I tried that, it brought me to the live production system where real money is moved. Not what I wanted. After searching the forum, there is actually a workaround per this thread:

I did pay attention on the timestamp – the bug was reported in 2012 and now is 2015. The bug has probably never been fixed since 2012. Anyway, the workaround worked just fine except I spent extra hour to figure out why and how. Changing a few links on a Web page shouldn’t be hard, but why didn’t Paypal do it?!

Passing Extra Fields

I wanted to collect some information on my buy page, for example, name and email address. This name and email address are different from what a buyer registers with Paypal. Let’s get more specific on this: one buy may purchase the license on behalf on the company so she wants to use company email address, but she needs to use personal credit card or account for the transaction. Whatever account information registered is different from business information.

This is actually quite easy to do with hidden inputs in the original button form (something copy and paste from Paypal button page). Here is a pretty good introduction on how to use it. Overall it’s a pretty flexible way to extend what’s available out of box.

The challenge is how to get the value of these fields in the IPN servlet? It’s not that straight forward. After looking for code sample, I could not get it in Java, but PHP. Without much difficulty, I wrote something similar in Java as follows:

Now back to the limitation of IPN simulator: it does not support the option fields. It has to be tested with a button.

It worked in the end. The programming isn’t tricky at all, but the naming is very much so indeed. Somehow Paypal changes the names of the fields from os0 and os1 to option_selection1 and option_selections2.

I don’t know what the design intention was but this inconsistency definitely made it VERY hard to use. Without documentation, no one can guess out the new names. Also, they changed the starting index from 0 in HTML to 1 in passed out data to IPN. Based on that, what I can guess is the engineer for the HTML used to be C programmer who still remembered that an array starts from index 0; but the server engineer used to be COBOL programmer who remembered that an array starts from index 1.

Anyway, the simpler yet correct thing to do is to allow usage of meaning names like email, and name, and use them consistently across HTML, IPN, etc.

Extra Field Validation

Although you can validate the field value in the last IPN page, you want to do it as soon as possible. Why? One reason is that it costs less to do client side validation. More importantly, once form is submitted, it passes the point of no return. Finding the problem in IPN leaves you not much choice. Say, the user enters a wrong email address and my IPN servlet generates a license but fails to email it. I don’t have an easy way to notify user if she paid using Credit card, but wait for a frustrated customer email me.

To do it is pretty simple. On the same page, I added a simple Java script that listens to the event that escape from the email field. Once it’s detected, it starts to validate the email address with a well know regular expression. It rules out the format problem. Does it solve all issues? No and impossible. If someone types an address with valid format but wrong ID, the JavaScript doesn’t help much. It can be further improved by adding two email fields, but I figured that is probably too much.

Should you be interested in the JavaScript for email validation on Paypal buy page, check out the buy page by yourself. JavaScript is open sourced by default.

NEED HELP?

My company has created products like vSearch ("Super vCenter"), vijavaNG APIs, EAM APIs, ICE tool. We also help clients with virtualization and cloud computing on customized development, training. Should you, or someone you know, need these products and services, please feel free to contact me: steve __AT__ doublecloud.org.

Me: Steve Jin, VMware vExpert who authored the VMware VI and vSphere SDK by Prentice Hall, and created the de factor open source vSphere Java API while working at VMware engineering. Companies like Cisco, EMC, NetApp, HP, Dell, VMware, are among the users of the API and other tools I developed for their products, internal IT orchestration, and test automation.