DrupalCon Latin America took place in Bogotá, Columbia this past February. 270 people attended the conference. There was a training day, two days of sessions and the last day was the big sprint day. It is tradition at Drupal events (and many other Open Source events) for people to gather together and work to improve the software we use. We had lots of people from the conference attend the big sprint day, 101 people, 37% of conference attendees. It was the highest percentage wise ever at a DrupalCon.

As an event attendee: should you attend a sprint?

Yes. But.. Sometimes participating at Sprints can be intimidating especially if you have never been to one. Where do you start? How do you participate? Who can you ask for help? Sprint mentors! And your next question might be, "Great… where can I find one of those?"

As an event organizer: What can you do to help attendees have a good sprint experience?

Having a way for attendees to identify people there to assist them, improves the experience for everyone.

Identifying Sprint Mentors

One way to identify mentors is with sticker name tags. (Note for future sprints: Name tags that hang from lanyards often end up below the table edge, and then other people around the table cannot see the name tag.)

Another way is with bright colored mentor shirts. However, at DrupalCon Latin America, we unfortunately did not have pre-printed special color mentor shirts.

Making Mentor T-shirts that Standout
T-shirts

Jared Smith (jsmith) had some black t-shirts with subtle bluehost branding on them, and positive messages people could identify with like: GEEK, and Helping OpenSource Projects Make an Impact. Some people were already wearing dark grey or black t-shirts.

The Internet of Things (or IoT for short) is probably even more of a buzzword than "Headless Drupal", but maybe not so much in Drupal land. As I am a man of buzzwords, let’s try to combine these things in one article (Also, there will be video demos)

Or maybe a couple of articles. I feeI have so much to say about this, partly because many articles one can read on the subject IoT deals with having your "thing" on your local network and playing with it over the wifi. We are not going to do that here, as your local network is not the internet. Of course you could forward your router settings to actually put that "thing" on the internet. And then that thing would be available to all hackers that would want to access it. So we are not going to that either in this article. For a first article, I want to explain how I see Drupal and IoT connecting together, and explore the patterns for this.

When we refer to “The Internet of Things” we often refer to devices capable of networking. This could be a car, cell phone or something like a Raspberry Pi, Arduino or Tessel. In this article I will simply refer to a "thing". This means a device we want to extract data from, or interact with via Drupal.

A common scenario

First, let’s look at a common way of testing out Internet of Things at home. You have a Raspberry pi and you have a temperature sensor. Now, the raspberry pi probably runs a flavour of linux, so you can actually install apache, mysql, php and finally Drupal on it. Then maybe you find a python script that reads the temperature, and you create a node in Drupal with the php filter that will exec this python script and print it inside Drupal. That will work. Except it is not a good idea. For several reasons. Let’s go through them:

Your "thing" will both be a sensor and a webserver. Make it do one thing, and focus on that

Your Drupal site will run on your local network, and to access it from (for example) your office you would have to make it publicly available in some way.

Your Drupal site will have the php module installed. You don’t want that.

Your php code will be doing system calls. Please don’t do that

Now that I have got that rant out of the way, let me also just say that if you want to go down that road, it is of course a low entry barrier, and if you are restricting access to your local network, then the security concerns are looking a little better. And of course, the php in a node part is not strictly necessary, it just fits with my arguments. So, if you are looking for that kind of tutorial, there are plenty others on the internet.

Patterns for communication

As I now have been ranting a bit, let me just point out that this is not a canonical article about IoT best practices or some absolute truths. These are just my opinions on how one could approach this exciting buzzword. Continuing on, let’s look at some ways of interacting between a Drupal site and your "thing.

If we look at it as simply one "thing" and one Drupal site, you have two actors in this communication model. So to establish a two-way communcation, we would both want the "thing" talking to Drupal, and Drupal talking to the "thing". This "thing" may represent something physical in the world, like a temperature sensor or a relay switch. So basically it is an interaction between the physical world and your site, so let’s use that metaphor. This article will deal with the first and most simple concept of this interaction:

The physical world talking to Drupal

Isolated, the wording of that heading looks kind of poetic, doesn’t it?

When the physical world is talking to Drupal, I mean it as somewhat of a “push” mode for your "thing". Let’s say you are monitoring the temperature in your apartment (physical world) and want to communicate this to your Drupal site. A simple thing to do here is to define an endpoint in your Drupal site where your "thing" would just post updates, and the Drupal site would store the information (of course with some sort of authentication). A one-way communcation to push updates from the physical world to Drupal.

Another theoretical and more intricate form could be something like this:

Say you have a physical store that also is an online store (a typical use case for Drupal). And you are about to have a sale in both places. But you want visitors in the store to have the same oppurtunity as the online visitors to get the good deals. In this scenario you could make it so that the moment the lock on the door was opened, a request is sent to the online store enabling the "sale mode". And when you close in the afternoon, the online store "sale mode" automatically gets disabled. This way, the physical store (or more precisely, the lock on the physical store) actually dictates the state of the online store.

Granted, this is a theoretical example, so let’s look at practical and implemented examples instead. I have put together a few quirky demos with varying degrees of usefulness. All examples are actual Drupal 8 sites running on Pantheon, so there is no localhost Drupal instance to talk about. This is the physical world talking to the internet.

Remote controlled Drupal 8

The first one kind of reminds of the above example, although maybe not so useful. It is a remote control to "shut down" the Drupal site (put it in maintenence mode). Or more precise: I am turning off the site with my TV remote. If you are wondering why the site refreshes a couple of times, it is because since I used one hand to film and one hand to press the remote, I had the site just update itself every 2 seconds.

Temperature monitoring

The second one is a more common one. Presenting the current temperature at a path in Drupal. Here we are just polling for updates to make the video actually show that it works, but a more practical example is probably to post updates every 10 or 30 minutes. Also note that now we can view the temperature from anywhere in the world, while still having our device unreachable over the internet. If you are wondering why am using water, it is because this triggers temperature changes much faster. The glass contains cold water, the cup holds warm water.

And here is a third one. Since I felt like being silly. This one displays the temperature, draws a nice historical graph of the temperature, and changes the color of the header based on the temperature. I must admit that the last part is purely client side in the video, but could theoretically be expanded to actually do this through the color module. I also must admit that the actual hot/warm color calculation could use some tweaking (more than the 6 minutes used on it), but you probably get the picture

Drupal 8 as a "surveillance backend"

For the last example there is something a little more elaborate, and maybe even practical. It uses a sound sensor to listen for sound changes. When the sound trigger is triggered, it takes a picture with the webcam on my mac (you can see the light next to the camera after I snap my finger), posting it to my Drupal site, creating a node. A simple surveillance camera with Drupal as a backend. Also, a very concrete example of the physical world interacting with Drupal, as it is the snapping of my finger (very physical) that creates a node in Drupal.

This article is already getting pretty lengthy, so I'm going to end it here. And before you ask: No, the code for the examples are not yet available. And yes, it will be made available. As I said, all these examples were put together quickly on a sunday morning, and they are all very hardcoded and hackily put together. I will post an update here, and probably a code-dedicated blog post about just that.

Also, I will be following up with the next scenario: Interacting with the physical world from Drupal. If you have any questions, please feel free to ask them in the comments. And I would very much be delighted to hear about alternative ways of doing this, people doing similar things or other thoughts on the subject (rants or ideas).

The ending of this post will be a lo-fi gif describing what sceptics usually call the Internet of Things - The Internet of Lightbulbs. Have a nice week!

In the last weeks I did some work on optimizing queries for the "Manage Supporter"-Interface in campaignion (online campaigning / online fundraising distribution). The goal is to filter Redhen contacts and then apply bulk-operations on them. In our larger databases we have up to 500,000 contacts and millions of activities - and the queries are not that simple either. How do you get “All supporters that signed at least two petitions last year but never made a donation”?

Cas Russell finds things for people, often involving some strategic
violence. She belongs to that genre of action novel protagonists who have
a rough code of ethics, but who don't exactly follow the law. Her friend
Rio is much worse: a functioning sociopath with his own code of ethics,
the derivation of which is a key plot point. When the book opens, Cas is
on a mission to find a person and rescue them from a drug cartel — not her
normal mission, but her contact said she was referred by Rio. Oh, and Rio
is currently hitting her in the face.

This setup matches any number of present-day thrillers. The SFnal twist
is that Cas is very good at numbers, in a completely unrealistic action
hero way. (I found it unsurprising that Huang was inspired by the
superhero genre; that's the right model to have in mind when reading about
Cas.) She can calculate where bullets are going to go, knows just the
right angle and velocity with which to throw things (and, even more
notably, can get her body to do that), and, in one particularly memorable
scene, sets up an eavesdropping sound concentrator by changing the angles
of available random surfaces in the neighborhood, like trash cans. This
ability is not without drawbacks. When she's not in a middle of a job,
with something to focus on, she usually ends up drinking herself into a
stupor to get her brain to stop working. But it's an extremely useful
ability that requires the villains of this book go to great efforts to try
to kill her.

The plot starts out as fairly typical thriller material, involving threats
and dire consequences to those Cas loves (or at least likes a lot) and an
unfolding sense that this retrieval of a kidnapped woman is the tip of a
very deep iceberg. The expected counterpart, a private investigator with
a less casual attitude towards killing than Cas, shows up early on. (Rio
does not play that role in the story. His role is much more complex.)
But the superhero inspirations show up in the villains as well, in a twist
that many on-line summaries spoil, but which I will leave unmentioned.

Mostly, Zero Sum Game is a fast-moving story with lots of violence,
lots of guns, shadowy conspiracies, and a hypercompetent protagonist.
(Female, refreshingly, particularly since she doesn't fall in love with
any of the other characters in the book.) It's a recipe for enjoyable
brain candy, and I think that's the best attitude to bring to it.
However, a couple of things set it apart for me.

First, Cas spends quite a bit of time really thinking about her life and
questioning her decisions, rather than just blithely enjoying her world of
stress and violence. There's more introspection here than in the typical
thriller plot, but she stops short of wallowing in angst and stays
decisive. I liked that balance: a bit of inner discomfort, and a few hard
ethical decisions, but not to the point of paralyzing her.

Second, her relationship with Rio is something special. Rio himself is a
character type that I've seen before in books like this, but I don't think
I've seen the dynamic with a character like that handled this well before.
I particularly liked that the focus of the book stayed on Cas, not on Rio,
and the reader was encouraged to see that relationship as a reflection on
Cas and her sense of internal ethics. Seeing Rio through Cas's eyes, and
then seeing other characters react to him and react to their relationship,
touched some chords that I really enjoyed reading.

Unfortunately, the villains weren't as successful, at least for me.
Partly this is a personal quirk: the nature of the threat posed (not
revealed for about half the book) is a kind that I dislike reading about.
It makes my skin crawl in a way that I don't enjoy. But, even putting
that aside, the story ends on a very odd and disturbing anti-climax. It's
clearly the first book of an ongoing series, and I hope later books will
salvage this. (I certainly liked it well enough to read on.) But the
ending left me unsettled and rather irritated at the author. Huang plays
fair, and the ending is consistent with what we know by the end of the
book, but I read this sort of action-thriller story for catharsis and the
glory of competent people doing what they do well.

I got deeply engrossed in this book and had a hard time putting it down.
Both Cas and Rio are great characters, as are most of the supporting cast.
I wish the ending wasn't quite as much of a letdown so that I could
recommend it more strongly. But it's still a fun superhero thriller. If
you're looking for something with unrealistic superpowers, a large helping
of competence, and a high body count, this may be worth picking up.

(And no, I don't know what the series title means. I know what the series
title refers to, but I haven't yet figured out what connection it or the
Axiom of Choice has to the plot.)

In part 2 of the 3 part series, we are looking at how get a Drupal website and database setup and running. If you followed part one, you will remember we are doing this all on our local environment using MAMP. As the video will show, we begin by going to Drupal.org/project/drupal and downloading the newest version of Drupal. At the time of this DDoD, Drupal is at version 7.36.

PHP 5.3 reached end of life in August 2014, this means that if you are running this version, you are running an insecure version of PHP that potentially has security holes in it. This is bad for obvious reasons.

Bundled opcode cache

PHP 5.5 is the first version that bundles an opcode cache with PHP, this means there is also no need to also run APC (unless you need userland caching in APCu).

It would be worth considering a dist upgrade though, but this at least can buy you some time.

Acquia Cloud UI

If you use Acquia Cloud for hosting there is a convenient PHP version selector in the UI.

More information can be found in the documentation. Be aware, once you upgrade beyond PHP 5.3, you cannot downgrade, so ensure you test your code on a development server first ;)

Common coding issues

Although Drupal 7 core, and most popular contributed modules will already support PHP 5.5, it would pay to do a code audit on any custom code written to ensure you are not using things you should not be. Here are some links you should read:

The reason being that drupal_render() expects a variable to be passed in (as it is passed by reference).

How do you find coding issues

Enable the syslog module, and tail that in your development environment, hunt down and fix as many notices and warnings as possible. The more noisy your logs are, the harder it is to find actual issues in them. While you are at it, turn off the dblog module, this is only helpful if you do not have access to your syslog (as it is a performance issue to be continually writing to the database).

Real world performance comparison

This was taken from a recent site that underwent a PHP 5.3 to 5.5 upgrade. Here are 2 New Relic overviews, taken with identical performance tests run against the same codebase. The first image is taken with PHP 5.3 running:

You can see PHP time is around 260ms of the request.

With an upgrade to PHP 5.5, the time spent in PHP drops to around 130ms. So this is around a a 50% reduction in PHP time. This not only makes your application faster, but also it means you can serve more traffic from the same hardware.

Comments

If you have gone through a recent PHP upgrade, I would be interested to hear how you found it, and what performance gains you managed to achieve.

DrupalGap continues to rock. Today I wanted to test the geofield module and get some maps in my apps... duh... it came about as a result of meeting with Tom Cormons from Appalachian Voices. He mentioned a problem - you often need two trees to pollinate eachother, and only have space for one... an app might help neighborhoods coordinate healthy tree communities.

Anyhow - my dev skills were amok. I followed the instructions on Tyler Frankenstein's site however I wasn't getting the maps to show up on my phone. They showed up fine on the web-app side of things (eg https://www.cvillecouncil.us/mobile-application/index.html#node_23 ) however after building and piping to my android phone I couldn't get the maps to show up on my kyocera hydro (c5170...) ... I needed a debug environment... enter gapdebug! oh yeah... gapdebug only works with android 4.4... my ghetto phone is running android ice cream 4.04... enter genymotion!

A couple of times recently the issue of managing variables in Drupal 7 has come up in conversation with other developers. This post outlines the various ways of managing variables in Drupal sites. The three things this guide ensures:

Sensitive data is kept secure

Variables are correct in each environment

You are able to track your variables (and when they changed)

The Variables Table

The most common place you'll find configuration variables is in Drupal's variable table (aka {variable}). The values in this table are often managed via admin forms that use system_settings_form(). Users enter the values click "Save configuration" and the data is stored in the database.

If you prefer to manage your configuration via the command line and know the variable you wish to set you can use drush vset. This does exactly the same thing as admin form, without needing to click on a mouse.

$conf Array

While the variables table is great at storing our variables, there are times when you want to enforce a setting. This might be because you want to prevent users from changing it (accidentally or otherwise) or because you need it to be different in each environment. The $conf array in settings.php always overrides any values in the variable table.

Acquia, Pantheon and platform.sh all provide environment variables so you can use different values in your $conf array depending on the environment.

Exporting Variables

In Drupal 7, the common way to export your variables is by using Strongarm with Features. I'm not going to cover how to do this as there is loads of documentation already available on this topic.

If your variable changes on a per environment basis or if it calculated on the fly, then you won't want to use strongarm+features as the exported values are static. You will need to put them in settings.php.

This is a common problem, especially on more complex sites. To avoid this I recommend creating sites/default/settings/settings.[env].php files. Your settings.php file should check for the environment in an environment variable and then include the appropriate settings.[env].php file.

What About Sensitive Data?

You can encrypt variables on a case by case basis using the encrypt module and some custom code similar to what I recently implemented in the Acquia SDK module (see on store and on read examples). Warning: This doesn't encrypt the data if you're using drush vset.

If you are storing sensitive data in your variables table I would recommend you implement hook_sql_sync_sanitize() which will delete the sensitive data from your db when drush sql-sanitize or drush sql-sync --sanitize are run.