Sunday, November 18, 2012

Foreword

I bought a Raspberry Pi a few months ago with the intent to have a toy web server at home. It's cheap (~$30), low-power (~3W), and doesn't need a cooling fan, so it's the ideal toy server that I won't feel bad about running 24/7. It has an ARM processor instead of the more common x86 processor, so you can only install certain OSes on it. Windows, Mac OSX, or most distributions of GNU Linux are normally compiled to create binaries that execute on x86 processors, so these can't be used. The Linux kernel can be compiled to create ARM binaries, so certain Linux distributions are compatible with this small computer. The OS that was custom-made for this small computer is called Raspbian, which is what I am using, but you can also install other OSes, including Android, Fedora, and XBMC.

I don't have an HDMI computer display, nor an HDMI adapter, so I will be setting up my web server by sending terminal commands via SSH. Some people may be off-put by the terminal, but I'll try to stay organized so we don't get lost.

I spent a lot of time reading about the details of networking, remote administration, and web server setup and best practices. This project is certainly the entrance to a rabbit hole of other interesting details you don't normally think about on a daily basis. Even though I spent a terrible amount of time, I gained a great deal of satisfaction.

Prerequisites

I performed a bit of setup before starting this project, so you may need to:

Download and install the Raspbian OS onto an SD card

Play around a bit on the OS, read the provided introductory Linux documentation

Install some packages you like using apt-get, such as Git

Connect power to the Raspberry Pi and connect it to your home network

Connect your laptop via ethernet to same network as Raspberry Pi

Before we can begin, we must connect our Raspbery Pi to our network, as stated in the prerequisites.

Our first step is to ensure we can interface with our Raspberry Pi's OS. I will not be using a keyboard, mouse, or monitor directly connected to the device, instead I will be interfacing with it by sending terminal commands to it by using SSH. The device declares a default hostname of raspberrypi, which the local network can use to uniquely identify your device. This is very convenient because we don't have to find the IP address that the router assigned to it, we only have to request raspberrypi from the network to send it requests. If you want to change the device's hostname, there are ways to this.

Connect to your Raspberry Pi with SSH

From your laptop, open a terminal and enter ssh pi@raspberrypi, where pi is the name of the user account you want to use.

Enter your password, which is raspberry by default.

Your current directory is the pi user's home directory. Any subsequent commands will be executed on the Raspberry Pi.

Type exit to quit your SSH session.

Install a Web Server

There are a number of web server applications out there, such as Apache, Nginx, lighttpd, and Jetty. Apache is by far the most popular web server, so you may want to try that, but I'll be using Nginx because I like to feel unique. Actually, I can think of a good reason: Rumors say that Nginx has a small memory footprint. This is a good match for a low-memory computer like the Raspberry Pi.

Install the Nginx package from the default Raspbian repositories.

Just to verify we don't already have Nginx on this device, type which nginx into the SSH terminal. It should give us no result.

Update our apt-get sources by typing sudo apt-get update

Just to verify that Nginx is in the default repositories, type apt-cache search nginx. We should see several results, including the simply-named 'nginx' package.

Install the Nginx package by typing sudo apt-get install nginx.

Validate that Nginx is installed

Type service --status-all. We should see an entry called 'nginx'.

Web Server Security

We just set up an application on our computer that we intend to welcome requests from the rest of the internet, which can be quite dangerous, so let's add some security. Like a good parent, we need to tell our web server to not talk to strangers. Because this is one of my first web servers, I will refer to the web server security guide on Linode for all security advice, mostly because it seems to be well-written.

Because some people make their careers as penetration specialists, I have always been curious about server protection best practices. Some of the protection I will set up may be redundant in a home server situation, but it is never redundant if new knowledge is gained! If I'm missing some important steps, please tell me - I would love to know!

Secure User Accounts

My Raspberry Pi had a default user named 'pi' which I have been using. One concern with web servers is that if a hacker gains control of a web request process, it can run under the same user permissions as the process owner. I want to ensure that Nginx request handling processes be owned by a limited-permissions user.

Before that, we have one more important change: Change the default password of the default user. We will later expose this server's SSH port to the public internet, and an stranger on the internet might try a set of default username/passwords.

Change the default password of the default user

Open an SSH terminal into your Raspberry Pi

Type passwd pi to change the password for the device's default user account.

Enter the existing password, the new password twice, and hit enter to save the change.

Nginx is a master process that spawns worker processes to handle multiple web requests. The Apache community says that the 'www-data' user should handle web requests, so we'll do the same. While the Nginx master process runs as 'root' user, each worker process runs as the 'nobody' user by default. The worker process owner can be customized in the /etc/nginx/nginx.conf file.

Ensure web request processes run as limited-permission user

Open an SSH terminal into your Raspberry Pi

Enter the following command to check the Nginx default user: nano /etc/nginx/nginx.conf.

Ensure the first line of this configuration file is user www-data;.

Use SSH key pair authentication instead of passwords

On your laptop, open a terminal

Generate an SSH key pair by typing ssh-keygen

Set up a Firewall

We should also set up a firewall to protect our computer from port scanners and other malicious programs. A firewall is basically a set of rules that limits or blocks incoming or outgoing web requests. After a bit of research, it seems that a tool called iptables is the most popular solutions for this. It is also the solution proposed Linode's server security guide.

The Raspberry Pi firmware version that I have isn't compiled with iptables support, so you may have to upgrade your firmware first. Luckily, it was is simple as a few terminal commands if we use the rpi-update tool that a user named Hexxah has created.

Defend Against Brute-force Attacks

One more thing we should be worried about is internet users attempting to access our SSH account by trying a dictionary attack against our password. There is a handy utility called Fail2Ban that monitors your log files for failed login attempts and temporarily blocks offending users.

You can do different customization, but I just followed the recommendations of these code snippets to add Nginx monitoring to Fail2Ban.

Secure SSH

I told my firewall to allow traffic through port 22, which is SSH. This means anybody, not only I, can try to SSH into my Raspberry Pi. To better secure this, we have already took two good steps: 1) changed the password for the default user, and 2) protect from brute-force attacks. But I want to do one more thing.

Restrict root for SSH

sudo vi /etc/ssh/sshd_config

Change the PermitRootLogin line like this:

PermitRootLogin without-password

Update the Server's Software

A best practice for server admins is to ensure all server software is kept up-to-date. This is really easy in a Linux system like this, so there's no excuse. You should do this about once a month, or whenever you think of it.

Update all installed packages

Open an SSH session with your Raspberry Pi.

Update your index of available packages and versions by running sudo apt-get update.

Request a Page from Your Private Web Server

I am fine with the default setup. I just changed it to listen on port 8080.

Start the server

Open an SSH terminal and run this on the Raspberry Pi: sudo service nginx start.

Check the web server

The default static web directory is /usr/share/nginx/www/.

Open a browser on your laptop and navigate to raspberrypi in a web browser. You should see the default index.html file created by Nginx, which says something like "Welcome to Nginx!".

Request a Page from Your Public Web Server

The final hurdle is to make your Raspberry Pi accessible to the rest of the world. My router is not forwarding requests to the Raspberry Pi, so we will need to do some port forwarding. I added the DD-WRT firmware to my router quite a while ago, so you may need to find a more specific guide for adding port forwarding for your specific router.

Make Server Visible by Public IP Address

Even with the port forwards, my web server still wasn't accessible by its public IP. With a friend's help, I tried putting it into my router's DMZ, which fixed the problem. It seems the purpose of a DMZ machine is to be the middle ground between your trusted/local network and the enemy/public network. This makes the DMZ machine almost wholly visible to the public internet. You may also want to try this on your router if you are having problems.

Put your server in your router's DMZ.

I have DD-WRT firmware on my router, so your steps may be different.

Open the web UI for your router by navigating to its IP address in a web browser.

Go to the NAT/QoS tab, then the DMZ tab.

Set Use DMZ field to Enable.

Set the internal IP of your web server in the DMZ Host IP Address field.

We now have our Raspberry Pi visible to the rest of the world on ports 80, 443, and 8080. Congratulations. Try sending a request to your web server by its public IP. You can find your public IP by using an internet site like IpChicken. If your external IP address is 123.45.67.890, then enter 123.45.67.890:8080 into your web browser to get the default Nginx index.html page.

Add Dynamic IP Solution: No-IP

My public IP address changes quite often, so it's impossible for me to SSH into my Raspberry Pi from work. To solve this, I wanted a domain name that points to my changing IP address. A technique called Dynamic DNS can help me. This involves registering a domain name with a third-party and periodically updating my server's registered IP address with a simple app. The third-party I chose to use is called No-IP. The No-IP updater app is built-in to some routers, so you should check there first. My router has this capability, which is filed under a category called DDNS, but it isn't working. So, I want to try installing the app on my Raspberry Pi. Let's give it a try.

During compilation, you will be prompted for your noip.com username and password, as well as a refresh interval (I chose 30)

Conclusion

That was quite an educational project. I have a new appreciation for managed web hosts, because I believe they are responsible for managing the server's security, network visibility, kernel upgrades, etc.

I will be following this tutorial, which was presented at Dreamforce 2012. This training session was wonderfully presented by Anand B Narasimhan @anand_bn and Richard Vanhook @richardvanhook. I wanted to condense this 2+ hour session into just the steps required to make a Spring app on Heroku.

(Note: This article was my first attempt at using markdown as a formatting engine. I grabbed the HTML and pasted it into Blogger, then fixed some spacing. Markdown limits you to six choices for headings, bullet point and number lists, and horizontal rules, so it's kinda restrictive. At this time, I'm not sure how to format this article better, so any tips would be nice.)

Project templates are gold, and save hours of frustration and configuration.
I have been horrified by the time required to go from an empty project to a working application in Java.

Embedded web container seems like a great idea. If system/environment admins don't have to set up the web server, there is less risk for failure when deploying to different environments. Moving one thing is so much easier than moving two things and ensuring they cooperate.

The project template uses a Java library called RichSobjects for talking to Salesforce. I haven't heard of this library before, but I'm making a mental note to check it out later if I need a Salesforce API library.

To set up the plugin, go to Window > Preferences and find the Heroku section on the left.

You will need a Heroku account, which is free. I made an account by using this wizard.

Get a Heroku API key by entering your Heroku credentials in the Email and Password fields and clicking Login.

The Heroku Plugin found my SSH key, because it is in a default location. If it's empty, follow the Heroku guide above to generate a public/private RSA key pair. Then return to the Heroku settings in Eclipse to generate an SSH key.

Create new Heroku app

Note: You can import an existing Heroku app by going to File > Import and selecting Heroku.
However, I will be creating a new app through Heroku.

Tell Eclipse to set up a new Heroku project for you by going to File > New > Project...
and select the Create Heroku App from Template.

Select Force.com connected Java app with Spring,OAuth and leave Application Name blank, as this name must be unique across all Heroku apps. If it's blank, Heroku will create a cool name for you.

This sends a request to Heroku to set up an app for you and puts all the code in a git repository that Heroku manages.

Eclipse will clone this Git repo locally and expose it as an Eclipse project.

Inspect what we have

This is a Maven project, so look at pom.xml to see dependencies:

Spring

spring-context

spring-webmvc

jstl

standard

javax.servlet-api

Salesforce

richsobjects-core

richsobjects-api-jersey-client

richsobjects-cache-memcached

force-oauth

force-springsecurity

Tomcat

webapp-runner

Logging

jcl-over-slf4j

slf4j-simple

There are many code files and settings files in this project, so it's hard to see what's going on. This is different from Force.com applications, in which only code is exposed to the developer, and settings are normally in the environment and changed in the UI. So, instead of looking at each component of this Java application, we are going to look at code at just the highest-level.

To see code that calls Force.com, see ContactController.java:

Class annotations (@Controller and @RequestMapping) are part of the Spring framework. These
instruct the framework where to inject framework code at runtime. This keeps code clean.

This class uses the RichSobjects library to interact with Sobjects in a Salesforce database
by using the Partner API.

To see the HTML page template we will request, see contacts.jsp:

Pretty simple: it's HTML which has JSP tags, which the HTTP request handler will resolve into HTML for the user.

To see global variables for the Spring app, see applicationContext.xml:

Most values are hard-coded in this file. However, look at lines 29-33 to see yet-unresolved values. These values will be drawn from the Heroku environment, I believe.

Heroku uses idea of an embedded web container. Instead of running a web deamon as an OS process, and the Java app as a separate OS process, we unify the two pieces. We can instantiate the Tomcat web server from our Java program, using a Java wrapper called webapp-runner.

webapp-runner makes app deployment and app start very simple.

Jetty is another web server that is popular to use as an embedded server.

Find the name of your app, which we allowed Heroku to decide. If it was funny-name-1234,
navigate to this URL to see that the app is already running on Heroku:

funny-name-1234.herokuapp.com

Set up local build for OAuth

This application is currently set up to use OAuth to gain access to data in a Salesforce org. By default, Salesforce will not allow OAuth applications to request access, so we have to add an exception, that is, define an accessible application.

To allow an external application to request access to a Salesforce org:

Choose any name in the Application field for this record. I chose Heroku Local.

Choose any email for Contact Email field.

Use http://localhost:8080/_auth for the Callback URL field.

Click Save.

Navigate to the detail view of this Remote Access record to see that Salesforce
generated a Consumer Key and a Consumer Secret.

To set up our project to run locally in Eclipse, instead of only on Heroku:

Set up a Run configuration by going to Run > Run Configurations...

Select Java Application from the list on the left and click the plus icon at the top of this list.

Choose a name for this Run configuration. I chose 'web app runner'.

Enter your project name in the Project field, which looks like funny-name-1234.

Choose the main class for the Main Class field, which is webapp.runner.launch.Main for this project.

Go to the Arguments tab, and enter src/main/webapp in the Program Arguments field.

This application uses environment variables for OAuth. We will specify this at run-time by going to the Environment tab. This app is expecting two keys, "SFDC_OAUTH_CLIENT_ID", and "SFDC_OAUTH_CLIENT_SECRET".

Click New.

Name = SFDC_OAUTH_CLIENT_ID

Value = (value of Consumer Key field from Remote Access record we just created)

Save this and click New.

Name = SFDC_OAUTH_CLIENT_SECRET

Value = (value of Consumer Secret field from Remote Access record we just created)

Save this.

Click Run.

After performing these steps and pressing Run, the application should be running locally on port 8080.
We can see this by navigating to the http://localhost:8080 URL in a web browser. To check that our OAuth has been set up correctly, navigate to http://localhost:8080/sfdc/Contacts URL. The app will redirect to a Salesforce authentication page, where you should click Allow. You will then be redirected back to the same URL in your authentication.

Setup Heroku app for OAuth

We added run-time variables to our environment by adding them to the Run Configuration in Eclipse.
To add these variables to our app when it is running on Heroku, we must add them to another settings
location in Eclipse that gets pushed to Heroku.

Choose any name in the Application field for this record. I used the name of
Heroku app, Heroku Funny Name.

Choose any email for Contact Email field.

Use https://funny-name-1234.herokuapp.com/_auth for the Callback URL field.

Click Save.

Navigate to the detail view of this Remote Access record to see that Salesforce
generated a Consumer Key and a Consumer Secret.

Open the Heroku settings for this Heroku project:

In Eclipse, click Window > Show View > Other...

Choose My Heroku Applications.

Right-click on your application in this view, and select App Info.

To add new environment variables to this Heroku app, choose the Environment Variables tab
from this file.

Click the + button on the right.

Key = SFDC_OAUTH_CLIENT_ID

Value = (value of Consumer Key field from Remote Access record we just created)

Save this and click New.

Key = SFDC_OAUTH_CLIENT_SECRET

Value = (value of Consumer Secret field from Remote Access record we just created)

After adding these environment variables, the Heroku app should be immediately updated to
reflect the values. If you navigate to the Contacts URL of your Heroku app, funny-name-1234.herokuapp.com/sfdc/Contacts, you can see that the OAuth is now working.

Add New Feature to Your App

To show how to add a new feature to the app, we will be adding a link to each contact's Twitter
handle. Follow these steps to add this new feature.

1) Add a new custom field to the Contact object:

Login to the same Salesforce org.

Go to Setup > Customize > Contacts > Fields. Click New and create a new text field
called TwitterHandle__c.

Save this.

2) Query this field in our local copy of the Java project:

In Eclipse, open ContactsController.java.

In the listContacts method, add TwitterHandle__c to the Select clause of the query.

Save this file.

3) Expose this field value in the JSP page:

In Eclipse, open contacts.jsp.

Add a new header column to this table by adding <th>Twitter Handle</th> to line 13,
underneath the Email header.

Expose the field value in the table cells by adding <td>${contact.getField("TwitterHandle__c").value}</td>
to line 27, underneath the similar row for Email.

Save this file.

4) Test this change in the local build of the project:

Stop the server by opening the Console view in Eclipse and clicking the red stop button.

Run a new build, which will have our Twitter handle column in the Contacts page by clicking
the Run button underneath the toolbar, which looks like a green arrow.

Navigate to the URL for this local page, which should be http://localhost:8080/sfdc/contacts.

5) To commit these changes locally:

Right-click on your project in the Package Explorer in Eclipse, select Team > Commit.

Add a commit message which describes the changes, and click Commit.

6) To push these local changes to our Heroku repository:

Right-click on your project in the Package Explorer in Eclipse, select Team > Push to Upstream.

Managing Your App

Check status of app

Go to My Heroku Applications view in Eclipse.

Right-click on one of your apps, and click View Logs to see last 1500 log lines
for your app in production.

Heroku made a thing called "Logplex". All messages that your app produces can
be accessed here. You can find third-party apps to derive information from these logs.

Scale your app up and down

Free dev accounts have only one dyno.

To scale app to >1 dynos, must tie money to account, for example, by joining
app to Heroku org with money.

Right-click on one of your apps in My Heroku Applications, click Scale and choose 3.
Your app is now on a 3-node, load-balanced cluster.

Add collaborators

Go to My Heroku Applications in Eclipse and click App Info.

Go to the Collaborators tab to see all other users in your Heroku organization.

Either select one of these users or click the plus button on the right to add by email.