Good enough practices

Sunday, October 25, 2015

It's been almost a year since I wrote a guide to setup processing using maven. This year, the guys at processing released their needed 3.0 version that leaves AWT behind and all applet related stuff, so I decided to update the tutorial for you to use the latest stuff.
If you don't know what processing is, it's an amazing Java project that allows you to work with visual arts. It's a really simple and powerful engine for doing multimedia stuff.

Maven

Yesterday, I took the latest stable version of processing (3.0.1), mavenized it and put it online so you can use processing in your projects with your IDE of choice

Declare this pom property
Add this third party repo to your pom:
And these dependencies:
You will need at least processing core that contains the processing classes.
If you want 3D (OpenGL support) you will need gluegen and jogl-all dependencies, with their native binaries. See the following list to pick the best for you according to your hardware(I used natives-windows-amd64 in my case)

Native libraries classifier

MacOS

natives-macosx-universal

Windows

natives-windows-amd64

natives-windows-i586

Linux

natives-linux-amd64

natives-linux-i586

natives-linux-armv6hf

Example Script

After creating the project and setting up the pom file you can create a class like this one:
Notice that some of the code has changed to be processing 3.0 compatible

This should give you a nice spinning 3D cube:

If you have any errors (specially with 3d stuff), check that you have the correct drivers, that you are not inside a virtual box (there's an issue for their 3d support), and if everything fails, check the web for common errors

Sunday, September 7, 2014

In case you don't know it processing is an amazing project in Java that allows you to do graphics.
(yeah, basically whatever you think of). Check the site for examples.

Maven

I took the latest stable version of processing (2.2.1), mavenized it and put it online so you can use processing in your projects with your IDE of choice

Declare this pom property
Add this third party repo to your pom:
And these dependencies:
You will need processing core that contains the processing classes.
And if you want 3D (OpenGL support) you will need gluegen and jogl-all dependencies, with their native binaries. See the following list to pick the best for you (I used natives-windows-amd64 in my case)

Example Script

After creating the project and setting up the pom file you can create a class like this one:
That will give you a nice spinning 3D cube:

Check the web for in case of error.
(You will need working video card drivers for this example to run)

To make our application environment aware we need to follow the following steps:
1.- Use Spring 3.1
2.- Add the profile-detector dependency to our pom
3.- Modify the web.xml file so we can initialize Spring prior anything else happens
4.- Add integration classes so we can declare environments and sensors
5.- Extend integration class to define you own environment parameters
6.- Select which sensors we are going to use
7.- Define known environment profiles with its sensor values
8.- Create different configurations for our needs
9.- Map each configuration with corresponding environment
10.- Optionally you can add beans for different configurations

Dissecting the example web app

Let's see what's inside the example web app while we follow the steps so you can use the code in it for your own projects.
I will assume you know how to use maven, how to add Spring 3.1.0.RELEASE as dependency, and how to install the profile-detector project on your repository. (Otherwise you should chek: Spring and maven, Install a jar to your repository). So let's skip step 1.

This way we tell Spring that it should look for this class, instantiate and use it for its own initialization,. We will use this hook to add the environment detection step.

4. Add integration classes

The classes I added over profile-detector project and integrating it with spring are:

AmbientBasedContextInitializerSupport

AmbientConfiguration

SpringBasedConfigurationSupport

These classes enable the feature of detecting the environment in the initilization phase of Spring.
You should take a look at the core of AmbientBasedContextInitializerSupport. This class does the magic of putting it all together:

As you can see, it makes a call to detect the current "AmbientProfile" which would represent the current environment using the profile-detector code. Then it asks its subclass for an "AmbientConfiguration" which represents the knowledge to adapt the current configuration to match the corresponding AmbientProfile.

5. Extend AmbientBasedContextInitializerSupport

If you payed attention to the web.xml snippet we declared a "com.tenpines.example.ambients.ExampleSpringContextInitializer" class. This class extends the "AmbientBasedContextInitializerSupport" class and enables the application to define its own characteristics.

This class will be instantiated by Spring to auto initialize itself.

6. Select which sensors to use

For the sake of the example I used only one sensor, the "HostnameSensor". This sensor takes the current machine's hostname and is probably the simplest way to detect different environments.
Implementing the "getUsedSensors" you tell which sensors to use:

Every sensor you declare in this method will be used any time the application starts to detect which environmet is current.

There are more predefined sensors you can use, and you can define yours just implementing the interface AmbientSensor. All the sensor have to take a snapshot of some environment variable and express it as a String so it can be compared later:

AvailableFileRootsSensorIt gives you all the file roots available as a single String

AvailableProcessorsSensorIt tells you the number of available processors on the current machine

CurrentSytemUserSensorTells the name of the current user as reported by the operating system

CurrentWorkingDirSensorThe name of the current working dir

HostIpSensorThe current ip address

MaxMemorySensorThe max memory available to the running virtual machine

OperatingSystemSensorA description of the current operating system

SystemArchitectureSensorA description of the current machine architecture

VmDescriptionSensorA decription of the running java virtual machine

7. Define your known environments

After selecting the sensors you should teach the application which sensors values should expect for each known environment.
You define different known environemnts (AmbientProfiles) to
discriminate where is your code running. This is intentionally separated
from the possible configurations (AmbientConfiguration) as a
configuration could be shared by different environments (probably in
development).

The "HOME_PROFILE" is my home machine profile. You should change the value to your own hostname in the "ALIEN_PROFILE" otherwise you will get and error indicating that the current environment is not known to the application when running the example.

[The last part of the init method declares a map, so we know which configuration goes with each profile]

8. Create different configurations

In this example there are two classes:

ExampleHomeAmbientConfigurationThis will be the configuration for my machine

ExampleAlienAmbientConfiguration This should be the configuration for your machine

They represent different configurations and also they use different spring profiles for the beans. When one configuration is elected according to the AmbientProfile, Spring is initialized with a specific profile:

Each configuration defines the Spring profile to use when activated:

Build your own configurations according to your needs. I my own project I change the logging options so in development mode I see everything on the STDOUT, but in production it goes to a rolling file.

9. Map each profile with its configuration

This is done in the method "getConfigurationFor" overrided from the "AmbientBasedContextInitializerSupport" class. You should use any criteria. I used a map.

The init method initialized the map for each profile.

10. Use different set of beans for each configuration

Finally and depending on your needs you can define different sets of bean to use for each configuration.
I mapped a bean profile for each configuration. The beans are declared in the main spring configuration file web-application-context.xml.

The syntax is pretty self-explanatory. You can see more on the Spring 3.1 tutorials.

Friday, February 17, 2012

Suppose you are developing a web application (I'm doing it right now). You use Spring, Hibernate, Maven and all those good old friends from Java world.

Now you have a production environment and a development environment, and you have 2 developers, each one with different operating systems. So you probably end up with 3 possible configurations (different file locations, connection urls and logging settings) for each of the possible environments (production, developer1, developer2).

How do you manage your configuration settings so any time you run the application the correct configuration applies?

Previous solution

Initially, in previous projects, we used what Maven calls "maven profiles" and "resource filtering".
Basically this means that our configuration files had variables that maven was able to replace with correct values based on the current profile.

For example, we could define a variable "v" that had different values for development, and production environments. Our configuration files used that variable "v" and running maven with the correct profile resulted in v replaced by "A". (You can see more details here)

This approach had 2 main drawbacks:

It's purely manual. We had to run this filtering process by manually specifying which maven profile was adequate.

You have different builds for different environments, so you have to quality test every one of them.

You have an additional build step. You had to make sure that whenever the application is started, the "filtering process" was correctly executed. This is not a problem in production environment because the war is assembled by maven, but it is an annoyance when developing.

New idea

What if the application were aware of its different running environments so it could change its configuration automatically based on the current?

If we wanted to do that, we had to solve 2 issues:

Fortunately for the second issue we have Spring 3.1 which introduced the concept of profiles to bean definitions. This is similar to what maven does to files but applied to beans. (more details)

Now, we can define different set of beans according to the current spring profile. If we structure our configuration in different sets of beans for each desired configuration we can change it easily.
The problem remains in how to detect the current environment to tell which spring profile to use.

Detect current environment

If we were able to determine which spring profile corresponds to the current execution environment we could let the application auto-configure itself.

So how do we detect the current execution environment?

We need to add environment sensors to the application, and define which possible environment profiles exist. Then if you know how each sensor measured on each environment profile, when you read current values, you can determine which environment profile is currently valid.

About the decision strategy

To decide which profile is best, I used afuzzy-like criteria.
Since environments change and it's likely that sensor values do change with them it's necessary that the application tolerates some level of change. I used a threshold approach for this.

When the application has to decide which profile is best, it evaluates each profile against all sensors assigning a score value to the profile. The best scored profile is the candidate for current environment profile, however there is a minimum value that any profile has to score to be accepted.

The rationale for this is that the environment can change, but if it changes a lot, it is another environment.

New Solution

I implemented this ideas in a very small maven project for the environment detection which you can freely use in your own projects. The "profile-detector" project is independent from Spring and can be used in any Java project to detect the current environment. (just the detection part)

Additionally you will want the integration classes that enable a web application with spring framework to auto-change its configuration based on the detected environment. (the configuration part)

Tuesday, September 13, 2011

As programmer one thing you do is to communicate with the computer so it does whatever you want.
However the computer isn't the only one you interact with. Your code is is something you tell to the computer and the team. Is communication of actions and intentions what makes the computer do things.
And this is the key part of the communication: intentions.

While the language of your choice may be able to communicate some intentions very explicitly, some others don't. This is where you should put your effort. Communicating your intentions is essential for your team and for your future self. Yes, you think you will remember but you won't. The more you code, the lesser the probabilities are you will remember what was the intention of each line.

So, how do you manage to communicate all the implicit decisions and knowledge that is used on the code? You don't! Because you cannot reflect all the implicit decisions. But at least, you try to express as much as is useful for you and your team.

Use annotations to communicate expected behavior with nulls

I use two extra annotations just for the purpose of communication. They don't change the behavior of the program nor are compiled. They are discarded on compilation but they help me and the team know what we should expect when using a method or passing parameters.

As you can see these annotations are only retained on the source level and can be used on parameters, methods, variables and class fields.

Of course, you can live without these, or make your own. The idea here is to communicate what your code expects as input and how it will behave as output. Communicating this information on the code has proved to be valuable for others and myself after months of coding.

Don't overuse
Remember the goal: communication. If you use them everywhere and without a second thought they will become noise, and people ignore noise. So all your extra effort will be in vain. Just use it when the code isn't clear or can be prone to errors.
If you use Hibernate or other frameworks that have similar annotations just use those. Don't add these mechanically as they are redundant in presence of similar annotations.

Uses
These are the main use cases

On methods: Indicate whether is safe to use the return value as null-impossible, or you should do extra checks

On method arguments: Indicate whether the method can handle null values or you may provoke a NullPointerException if careless.

On class fields: Indicate whether the class is designed to handle null on that field, or some parts of the class rely on the assumption of not null.

On local variable: This case is rare and I don't use it very much because I try to make short methods. But just in case, you can indicate whether or not a local variable can be made null during a loop or similar.

Ok, not so difficult.
First one is a Hibernate annotation for mapping a one-to-many relationship indicating wich field is the owner. The second one is how you obtain a reference to a class Object for String class. And the third one is how you obtain the "name" field of the class "Person" using reflection.

If you have been around Java long enough you should be familiar with of these forms of reflection.

Now. Can you tell what's wrong with them? (hint: the tittle of this article can help you)

Yes! There are using string literals to refer to class structure elements.

And why is that wrong?
Because doing so you loose the relationship with the referred element. Your code looses the information about structure dependencies.

Using string literals you cannot know where all the reflections references to some class field are, prior to a refactor.You cannot change the name of the field in one place and expect to automatically update all the references (remember DRY?).

In other words you are making implicit an existing dependency relationship between referred and referral code and you start to loose control over changes.

The sad part is that you have to update the constant every time you rename the field (or class), but luckily you won't be changing that too much and in case you do, you will have to change just one place.

The nice part is that now, you can see where all your reflections references are with most IDEs and if you happen to delete or rename the field carelessly, having that constant near is a reminder of the existing dependencies.

Of course this is not an ideal solution, but is good enough to keep you in control of your code.

Extra note

I prefer to use "_FIELD" as a suffix when the constant refers a field so that the IDE auto-completion capabilities offers me the constant and I know that is not a getter or setter method. I use "_GETTER" suffix when the property is not available as a field on the class.

Sunday, August 7, 2011

The problem

So you need to query objects near some location?
You say ‘hey, this is Android, it should be easy, everybody makes a “search-near-me app”’, right?

Well, the truth is that is not that easy. Making a query for items near a geographic point is not as easy as one might expect. One of the reasons is that SQLite (SQL engine that comes with Android) doesn’t support trigonometric operations, or custom functions, or spatial extensions.

The short answer

In a nutshell, the solution is to use an alternative distance function to order objects based on this distance to the reference point and filter a few to show them on a map. Specifically, use the Manhattan distance that is based on simple operations easily written in Sqlite. You will get a small ordering error but will be good enough for many cases.

Where

d1 is the distance function

are vectors and

pi and qi are scalar values.

The long answer

Let’s see step by step what should you do.

Limit the result set

The first thing we need to accept and to do is to reduce the number of elements we are handling. There are two good reasons for this. The first one is usability; users don’t feel comfortable if you throw a hundreds of items over the map. The second is performance, there’s no point on processing elements that won’t be visible.

So the first thing you should do is to decide the number of elements you will be showing the user. That might depend on the type of application, but you should be careful not to show less or more than expected. Make some tests on real phones to discover hardware limitations to this number.

Let’s say 40 elements is ok for us. We know that we don’t care about all the results; we just want the first forty elements near to the reference location (i.e.: the phone current location). Because we are using an approximate distance for our calculation, we will make the limit a little higher. So let’s make it 65 elements. I will explain why later.

At the end of the query you should have a limit clause, something like ‘LIMIT 65’ as we want the first 65 elements.

Order results by proximity using Manhattan distance

This is a proximity based result display, so we want to show the user the first nearest elements. For that purpose we need to order the results, otherwise they will be in any random order and you will get any 65 elements.

This kind of distance is very easy to calculate for Sqlite and has a small error compared to using a real geometric distance for ordering.

Add an order by clause: ‘ORDER BY abs(latitude - (?)) + abs( longitude - (?))’
Here I’m assuming the query arguments are the user current coordinates (latitude and longitude) and ‘latitude’ and ‘longitude’ are the columns with the values for each element.

The query should look something like this:
"SELECT _id, latitude, longitude FROM EXAMPLE_DOTS ORDER BY abs(latitude - (?)) + abs( longitude - (?)) LIMIT 65"
And we will add as arguments the latitude and longitude of the reference point.

Error in Manhattan distance

Ordered in this way, our query will be enough to make the trick. If you only show the elements in the map, and they are scattered enough, nobody will notice the difference.
But, if you must show a list of elements, or they are not so sparsed you will still have to make a few corrections.

The Manhattan distance has a little problem (compared to real distance); it favors elements close to the axes. That means that elements near the vertical or horizontal axis will get lower values for their distance. That means better positions on the order, and will likely to be first that other elements. In other words elements near 45º, 135º, 225º and 315º will receive a worst order than they should.

If elements are regularly scattered you will get a
diamond-like shape with manhattan distance.

When using this alternative distance to order elements and limiting the results you could include an element farther that is close to an axis and exclude an object that is nearer because it’s not close to the axis.

This is why we add additional elements on the limit clause; we don’t want axis elements to be favored and diagonal elements to be discarded erroneously. To avoid this situation we will use a real geo-distance (at least a far better approximation to real distance).

Use Android Location-distance calculator to reorder and filter

The Android platform comes with an excellent implementation of an approximate geo distance calculation formula based on WGS84 ellipsoid (http://en.wikipedia.org/wiki/World_Geodetic_System#Longitudes_on_WGS_84).
This formula cannot be used inside the query, but as we limited the number of results (only 65) we can use it to re-order them in memory very fast so we have a real distance ordering.

You should call Location.distanceBetween(startlatitude, startLongitude, endLatitude, endLongitude, results)to calculate a better distance and re-order the elements obtained from the database. You can then, discard the exceeding elements.

I do this by using a few helper classes that I attached to this article as an Eclipse project :

/**

* Order the results based on real distance instead of manhattan. This method changes the order

As you can see, the above method is used to re-order the elements using a geo-distance and filtering out the exceeding elements. So, now we have the forty nearest elements we wanted.

With real geo-distance and regularly scattered elements you should get a circle-like shape (or approximate).

Conclusion

The method I showed above is not for every case but can be used for most of the applications that need to show results near some point.
In fact, just by using Manhattan distance on the query can be sufficient and you don’t need to make extra corrections.

You should be sensible as when to use it due to the error in the order that Manhattan distance makes. I recommend you to test with real data points so you can judge if the results are good enough for you.