message passing

Wednesday, 14 October 2015

It's Oracle OpenWorld time, and MySQL Cluster will be in San Francisco again along with the rest of the MySQL team.

The session agenda is online, but can be tricky to navigate given the breadth of the conference, but it's possible to narrow down to those sessions in the MySQL track.

From a MySQL Cluster perspective, there two conference sessions from the Cluster development team, a Hands-On Lab from our Cluster Support team and most exciting, a conference session from some real-world users at NEC.

200 Million QPS on Commodity Hardware—Getting Started with MySQL Cluster 7.4

[CON2178]

Frazer Clement, MySQL Cluster Technical lead, Oracle

Bernhard Ocklin, Director MySQL Cluster Engineering, Oracle

Do you have performance demands that a database can’t scale to meet — especially one as simple as MySQL? And if it were even possible, would it require top-of-the-range servers and storage? We previously demonstrated that MySQL Cluster could scale to 1 billion writes per minute, and MySQL Cluster 7.4 exceeded 200 million queries per second (QPS), all with open source software running on commodity servers. Discover how MySQL Cluster achieves this scalability while also delivering in-memory performance, 99.999 percent availability, active-active update-anywhere geographic redundancy, ACID transactions, and both SQL and NoSQL access. Finally, this session offers some tips on getting MySQL Cluster up and running so that you can try it out for yourself.

Conference Session

Monday, Oct 26, 2:45 p.m. | Moscone South—262

Fully Elastic Real-Time Services with MySQL Cluster

[CON4772]

Bernhard Ocklin, Director MySQL Cluster Engineering, Oracle

In this session, learn how the MySQL Cluster in-memory real-time engine brings together the best of both worlds—SQL and NoSQL. It scales from a single Raspberry Pi to systems on hundreds of servers. MySQL Cluster powers networks for more than a billion mobile phone users worldwide and serves massive multiplayer online gaming back-ends. The MySQL Cluster architecture allows the addition or removal of nodes in seconds without interrupting service. It adopts to capacity demands making resources available instantaneously and when needed. Its native Node.js platform and connectors for Java make it easy to write real-time web applications in the cloud. This sessions guides you through common architectures and a use case for elastic MySQL Cluster cloud deployments.

NEC
is developing India’s financial inclusion gateway service utilizing
MySQL Server and MySQL Cluster. To handle the massive, business-critical
financial transactions involved in this project, NEC has conducted
deep, detailed research on internal architectures and behaviors of MySQL
products. This session explores use cases of MySQL Server and MySQL
Cluster in India’s financial inclusion gateway service, and explains how
to select appropriate software and optimize systems architecture with
methods of software architecture analysis at NEC.

Conference Session

Tuesday, Oct 27, 5:15 p.m. | Moscone South—250

Get Started with MySQL Cluster

[HOL3348]

Benedita Vasconcelos, Principal Technical Support Engineer, Oracle

Attend this hands-on lab to learn the basics of MySQL Cluster—when to use it, when not to use it, and how to install, configure, administer, and access it. MySQL Cluster is a write-scalable, real-time, ACID (atomicity, consistency, isolation, and durability)-compliant transactional database combining 99.999 percent availability with the low TCO of open source. Developers and DBAs attending this session have the chance to familiarize themselves with MySQL Cluster and get a better understanding of how to use it to meet the database challenges of next-generation web, cloud, and communications services with uncompromising scalability, uptime, and agility.

HOL (Hands-on Lab) Session

Thursday, Oct 29, 9:30 a.m. | Hotel Nikko—Peninsula (25th Floor)

There are many other relevant talks from the MySQL team covering Server improvements, Replication improvements, OpenStack integration, Customer success stories etc.

Outside the MySQL track, there are sessions covering the breadth of Oracle products and services. Despite developing the world's most popular Open Source Database, Oracle still puts some effort into developing older DBMS products :). I will be interested to hear of the latest advances there.

However the main attraction of a conference like this is meeting people and getting to talk face to face. If you don't have an OOW pass, but want to mix with the MySQL crowd then please come to the MySQL Community Reception on October 27th. You can pre-register to avoid any queueing.

Thursday, 6 March 2014

I've been letting the blog rest recently, and not so recently as well. The problem is not a lack of subjects, but a lack of time to do them any justice. However it is quite sad to see that my last entry was in September 2012, so it is time to post again.

Of late I have been pondering what I have to say about :

Distributed MVCC and write-scaling

Different approaches to eventual consistency with replicated RDBMS

Various MySQL Cluster related topics

Various general rambling and unstructured topics

However, these will take some time to percolate and calcify.

In the meantime here are some things I have found interesting recently :

Learn You Some Erlang for Great GoodI actually rediscovered this online book after watching some Joe Armstrong + Erlangvideos, after watching some spoof video about bringing Erlang up to date. All recommended Erlang and Ndb Cluster share some Plex heritage, which can still be seen in their architectures today. Since Plex, Erlang has mated with Prolog, and Ndb Cluster was involved in a car crash with C++.

HyperDex + Hyperdex WarpSomething I discovered last year from Emin Gün Sirer's blog and have returned to since. There are a number of nice ideas combined here (chain replication, value dependent chaining, hyperspace hashing, subspaces). My favourites are the concept of 'spurious coordination' and their solution w.r.t. transaction consistency : ordering the route of the optimistic 'distributed commit' based on the affected keys. I guess we need more independent analysis and evaluation to understand the strengths and weaknesses of these techniques.

KronosThis is a distributed HA 'event ordering system' from the same Cornell HyperDex team. Thinking about distributed MVCC led me to thinking about efficiently maintaining a distributed partial ordering of events while avoiding 'spurious coordination'. Kronos is an attempt to solve part of that problem in a kind of abstracted SOA way. There is some nice detail in the paper about their dependency graph traversal optimisations, and how dependencies are immutable once discovered, so can be cached, replicated for read scale-out etc. This could be a great systems building block.

Systems Performance bookI am slowly reading this doorstop book from Brendan Gregg, an ex Solaris kernel engineer at Sun, now at Joyent. It contains a great amount of recent practical information about Linux + Solaris performance analysis and optimisation. Unix performance tools have always been a little opaque to me, with very little of how-to-approach a performance problem ever being documented. This book covers many old and new tools, but also includes rare information on how to analyse problems with these tools, rather than just syntax and units of values returned. Perhaps even better is his supreme confidence about tackling and solving any performance problem that foolishly catches his eye. I guess that comes from experience, but maybe a little can be conveyed to his readers by this book.

One of my favourite discoveries was this quote attributed to Charles Babbage :

"On two occasions I have been asked 'Pray, Mr. Babbage, if you put into
the machine wrong figures, will the right answers come out?' I am not
able rightly to comprehend the kind of confusion of ideas that could
provoke such a question."

Sunday, 30 September 2012

The first day of the first MySQL Connect conference is done. It's been a busy day! Many attendees are interested in the new MySQL Server 5.6 release, but of course MySQL Cluster is the main draw here. After a session from Oracle on the new features in 7.2, and early access features in 7.3, I attended Santo Leto's MySQL Cluster 'hands on lab'. Despite having started more clusters than most, it felt like a new and exciting experience installing and running my cluster alongside everyone else. The lab machines had some networking issues, but with Santo's help we seamlessly failed-over to some downloads he'd prepared earlier - very professional!

Afterwards it was my turn to talk on the subject of MySQL Cluster performance. The quality of the questions was impressive - there seems to be a very capable crowd in San Francisco this weekend. I was flattered that some audience members were taking pictures of my slides, but there's no need, you can download them here, and probably from Oracle somewhere too.

Finally we had a 'Birds of a Feather' session where a number of users asked questions about replication, backups, failover + arbitration, system recovery etc... Great to get the chance to explain some things, hear about user experiences, and get some feedback about user experience.

There's been a lot of good content so far, and interesting discussions, but the highlight for me has been
meeting with colleagues old and new, from MySQL, Sun and Oracle times. More of the same tomorrow, I'd better get some sleep!

The content is not finalised yet, so if there's something you would like to hear about which fits with the abstract, then comment below. If it doesn't fit in with the abstract then we will just have to talk about it afterwards over some drinks.

This is the first year of MySQL Connect, but there's some great content. I'm looking forward to meeting with users, customers, other MySQL engineers, and hopefully get the chance to learn something!