Posts from the Swrve engineering team

Menu

Sometimes in frontend development there are really simple UI elements that we take for granted. I’d argue that tooltips fall squarely in that category. They get taken for granted because they just work, and generally if something works you shouldn’t mess with it, right?

Well at Swrve we felt that tooltips weren’t quite as good as they could be , at least in React/Redux apps, and here’s why:

1. Orphaned tooltips

Ever remove an element from the DOM and have a tooltip linger around? React apps may redraw your DOM, replacing a button that has a tooltip with a different button instance on click. I found that under the hood projects like React-Tooltip and others often referred to DOM refs or, heaven forbid, jQuery.

The fact is, a React app should wherever possible use the virtual DOM and synthetic events rather than relying on actually rendered DOM nodes or a library that reads the DOM.

It turns out that handling all the important events for a tooltip implementation are available to us for free with synthetic events and lifecycle hooks so you can ensure that a tooltip is removed before its parent unmounts.

2. React is composed, tooltips should be too

I mentioned above that a tooltip should unmount when its parent does, but that’s not quite how tooltips work in HotTip. With React we build UIs by composition, so that’s the interface that HotTip uses. In JSX this would mean that the tooltip is the parent and the element that the tooltip belongs to is actually the tooltip’s child.

You may be familiar with the decorator pattern, es6 or otherwise, and that’s all this is. You decorate your element with tooltip behaviour, your element doesn’t change but your tooltips are managed, easy!

3. Positioning

HotTip doesn’t have endlessly customisable tooltips, they’re pretty simple. It also doesn’t do realtime repositioning the way jQuery solutions might. What it does do is calculate whether there is enough room to display your tooltip centred or if you need to offset to make use of the available space, then these are applied as CSS classes.

This is a lightweight and powerful solution that is possible because of the constraints HotTip ships with (max-width etc.). You save on performance by not needing to calculate based on mouse position and you get a consistent result between development and production.

4. Redux

Redux manages your app state. Is a tooltip state? Of course it is. Should you ever need more than one tooltip at a time? I don’t think so.

Given these two factors a singleton hotTip redux store seems like a sensible solution to use and it is very efficient.

TL;DR

No jQuery or browser node references, do it the React way

Add tooltips by writing normal JSX tags wrapping your elements.

Positioning is relative to an element, not the cursor. There are no expensive recalculations on mouse move events this way and the result is consistent.

CSS classes determine most of the layout

All your state is in Redux, there is no component state.

Events kick off from the virtual DOM and in all likelihood complete before the render cycle completes so you’ll get efficient rendering.

Finally

HotTip is open sourced, and the source is reasonably straightforward, (read it here). If there’s anything you’d like to see added, changed, or if you want to fork of this as a starting point, we’d love to hear from you and see what direction you went with it. HotTip is maintained by Swrve engineers but we’d love to collaborate on it.

RateLimitedLogger is an Apache-licensed, SLF4J-compatible, simple, fluent API for rate-limited logging in Java. A RateLimitedLog object tracks the rate of log message emission, imposes an internal rate limit, and will efficiently suppress logging if this is exceeded. When a log is suppressed, at the end of the limit period, another log message is output indicating how many log lines were suppressed.

We’ve been using this logger for all Swrve’s high-performance Java services for nearly 3 years now, and it’s become an invaluable part of our toolkit.

Thanks to Fabien Comte, this release removes all dependencies on Joda-Time and Guava, and is focussed on Java 8. If you need Java 6/7 support, please use version 1.1.0 of the RateLimitedLogger.

Temple Street Children’s Hospital is Ireland’s leading paediatric hospital, caring for more than 145,000 children every year, and so is a charity which is important to many of us here at Swrve.

Techies4TempleStreet is an annual treasure hunt aimed at raising funds for the hospital. In 2015, over 800 attendees (including two teams from Swrve) managed to raise over €150,000. This year the event was set to be even bigger!

Each team pledges to raise €500 for Temple Street, so we hosted a bake sale, which contained all manner of delicious homemade and bakery-bought treats.

Thanks to support from our fellow Swrvers as well as the Financial Services Union and Analytic Partners who we shared a building with, the bake sale was a huge success!

Everyone was very excited to set off on a day of adventure! Our volunteers were split into two teams, Team Aces and Team Supremos.

When our teams arrived at the starting venue, they were given a map with sixty-four points of interest all over Dublin city centre. Each number marked on the map had a corresponding question, which you could only figure out by visiting these locations;

Without a photographic memory and an extremely varied knowledge of every single square meter of Dublin itself, our teams had quite a bit of ground to cover in very little time!

But thankfully Adam H had an ingenious plan….

Register, run around, solve clues, win. Easy!

Perhaps we should have broken these tasks down into smaller story points… but the registration deadline was upon us!

Team Aces registered and ready to go!

Along with the map and corresponding questions, there were a number of mini challenges for our teams to complete. For example there was the hunt for Periscope Pete, who gave mysterious clues about his whereabouts over twitter during the event.

Periscope Pete, caught by Team Supremo!

There were also an assortment of challenges posted to twitter during the day which needed picture proof, like hugging trees.

Or getting selfies with the Gardai!

After three long hours of adventuring, during which they covered over 16km of ground, our teams raced back to the RDS to hand in their completed answers, and take a much needed rest.

A very tired Bob 😦

With good reason to be exhausted, because our teams came third and seventh!

And even better;

We raised €1,135.40 this year, with the overall total raised by the whole event coming to an enormous €200,000!

A celebration of beer and pizza was much deserved, and perked our hard working teams back up after a long day.

If you want to see more of how the day went, Ciaran has a fantastic video montage of the whole event;

At Swrve we are given two Volunteer Days every year, with which we can take part in charity or community activity as additional paid time off.

We live in a broader community where things are frequently not so rosy. Swrve would like to make a tiny dent in that by supporting any team member who wants to contribute work time to volunteer community activity.

Over 800 people die by suicide in Ireland every year, and globally the rates of suicide have increased 60% in the past 45 years. Cycle Against Suicide aims to get people talking about mental health, and break the stigma associated with asking for help.

Mental health is an issue that affects everybody at some point during their lives; either directly or by supporting a friend or family member through a hard time.

For 2016, a few kind Swrvers donated their own Volunteer Days to allow Ciaran to do the full 2 weeks, joined by Tim and Dom.

Ciaran, Dom and Tim taking a much deserved break during Cycle Against Suicide 2016

In Ciaran’s own words;

It’s an unbelievable experience, something I can’t put words to. It’s a force of change and an amazing community of people.

He also filmed his journey of around 1400 kilometers!

Congratulations to all involved in the cycle this year, and hopefully the 2017 event will be even bigger! As the Cycle Against Suicide mission statement says;

Here at Swrve we’ve got a world class engineering team. We’re an active lot. It turns out quite a few of us get restless when we sit at our desks for too long. Whether it’s cycling 1400km around Ireland, hiking in the hills at the weekend, hitting the crossfit gym in the morning on the way to the office, swimming in the sea, taking part in running races, or just cycling to work a few days a week, it’s safe to say sitting stationary at a desk wasn’t what drew many members of the Swrve engineering team to our profession. Mix this up with the usual lower back pain most of us have suffered at some point in our working lives and it was inevitable that we’d end up trying out standing desks in our office at some point.

Back in May one person used their Friday Skunkworks time to build a standing desk from an Ikea coffee table. Quite a few members of the engineering team took the opportunity to use the new desk for a few days to evaluate whether it was a setup that they liked. We also picked up a commercial desk from Varidesk at around the same time which has been used by team members following back problems.

Logging is vital for production-ready, operable code; however, in certain situations, it can be dangerous. It is easy to wipe out throughput of a performance-critical backend component by several orders of magnitude with disk I/O in a performance hotspot, caused by a misplaced log call, or input data changing to something slightly unexpected.

RateLimitedLogger is an Apache-licensed, SLF4J-compatible, simple, fluent API for rate-limited logging in Java. A RateLimitedLog object tracks the rate of log message emission, imposes an internal rate limit, and will efficiently suppress logging if this is exceeded. When a log is suppressed, at the end of the limit period, another log message is output indicating how many log lines were suppressed.

This style of rate limiting is the same as the one used by UNIX syslog; this means it should be comprehensible, easy to predict, and familiar to many users, unlike more complex adaptive rate limits.

Once the rate limit is hit, logging becomes just a couple of java.util.concurrent.AtomicLong operations, so it’s very inexpensive, performance-wise.

(TL;DR: using an SSD cache in front of EBS can give a massive I/O throughput boost)

Background

Internally, Swrve is built around an event-processing pipeline, processing data sent from 100 million devices around the world each month, in real time, with an average events-per-second throughput in the tens of thousands.

Each event processor (or EP) stores its aggregated, per-user state as BDB-JE files on disk, and we use EBS volumes to store these files. We have a relatively large number of beefy machines in EC2 which perform this task, so we’re quite price-sensitive where these are concerned.

We gain several advantages from using EBS:

easy snapshotting: EBS supports this nicely.

easy resizing of volumes: it’s pretty much impossible to run out of disk space with a well-monitored EBS setup; just snapshot and resize.

reliability in the face of host failure: just detach the volumes and reattach elsewhere.

EBS hasn’t had a blemish-free reliability record, but this doesn’t pose a problem for us — we store our primary backups off-EBS, as compressed files on S3, so we are reasonably resilient to any data corruption risks with EBS, and in another major EBS outage could fire up replacement ephemeral-storage-based hosts using these. In the worst case scenario, we can even wipe out the BDB and regenerate it from scratch, if required, since we keep the raw input files as an ultimate “source of truth”.

One potential downside of EBS is variable performance; this can be addressed by using Provisioned IOPS on the EBS volumes, but this is relatively expensive.

However, our architecture allows EPs to slow down without major impact elsewhere, and in extreme circumstances will safely fall back to a slower, non-realtime transmission system. If things get that slow, they generally recover quickly enough, but in the worst-case scenario our ops team are alerted and can choose to reprovision on another host, or split up the EP’s load, as appropriate. This allows us to safely use the (cheaper) variable-performance EBS volumes instead of PIOPS; it turns out these can actually perform very well, albeit in a “spiky” manner, with occasional less-performant spikes of slowness.

As we were looking at new EC2 instances several months back, we noticed that decent spec, high-RAM, high-CPU instances were starting to appear with SSDs attached in the form of the “c3” instance class. These SSDs, in the form of two 40GB devices per instance, were far too small to fit our BDB files with enough headroom for safety, unfortunately. But could they be used as a cache?

Our data model tends to see lots of accesses to recent data, with a very long tail of accesses to anything older than a few days, so there was a good chance caching would have a good effect on performance. Let’s find out!