2010-12-21

Within Monty Program Ab we have during this year had a lot of discussions about how to go forward with the MariaDB trademark. It's been clear that everyone wants to have something that is substantially freer than the MySQL trademark to ensure the survival of MariaDB whatever happens to Monty Program Ab.

We wanted to make something that should work well, both for open source and commercial usage (and yes, I know that in some cases these are one and the same), which is not very common with many other trademark policies. My belief is that by having a very liberal trademark policy we will create a bigger ecosystem around MariaDB which will help all of us.

2010-12-16

Congratulations to Oracle and the MySQL team for getting 5.5 out the door!

I first have to acknowledge that I have not been able to follow the MySQL 5.5 development as closely as I would have wanted during the last 2 years as most of the planning of MySQL 6.0, 5.4 and 5.5 has happened behind closed doors, without insight for the community. The commits have been open, until recently, but it's not easy to follow what is happening just based on the commits. I am sure that I am missing below some of the important features in 5.5 and forgetting to acknowledge some of the people that have done great work on 5.5.

That said, the 5.5 release contains some things that a lot of heavy users of MySQL have waited a long time for. Here follows some of the main things that I know about.

InnoDB improvements:

The new version 1.1 of the already improved InnoDB plugin is now the default (and only) InnoDB (in 5.1 the InnoDB plugin version 1.0.x was an optional engine).

Multiple buffer pools and rollback segments, splitting of log_sys and flush_list mutex, and other improvements that together give a major contribution to the improved scalability and performance of 5.5.

Thanks to the InnoDB team for continuing to do impressive work!

Adding DDL locking.

In earlier MySQL versions LOCK_open, which protected table opening (among other things), was one of the main things that stopped MySQL from scaling on many CPU's, especially when many tables were used. By introducing proper DDL locking this issue is now mostly fixed.

A big thanks to Konstantin Osipov and Dimitry Lenev for the hard work they have done in getting this to work.

Reducing scalability bottlenecks in the core server, such as LOCK_open, LOCK_alarm, LOCK_thread_count, Query cache and binlog mutexes and so on.

It's now almost exactly 2 years since the MySQL 5.1 GA release, so what's the verdict of 5.5?

The two things that impress me are the work on the InnoDB storage engine and the DLL locking. Most of the other new features are minor (technically) adjustments even if the results, like with the windows speed improvement, can be impressive. As a technical person I have a hard time getting impressed by something that could be done in a few hours and could easily have been done in 5.1.

What is worrying me the most is the things that are not in 5.5. Some if the things that were supposed to be in the next MySQL release are:

Pool of threads (instead of one thread per connection like it is in MySQL 5.1).

As far as I know, this feature has been moved to appear in 5.6 enterprise and will not be part of the community server.

An open source, free backup utility and backup API that would work for all storage engines. This has probably been the most awaited feature of MySQL ever!

This project was discontinued by Oracle the same day the feature was deemed ready to be pushed into 5.5.

In 5.5 some internal subsystems, like safemalloc, a portable memory checker, were removed.

Another worry I have is how long Oracle will be able to continue development of MySQL while important core developers continue to leave. Of the above, Konstantin Osipov and Vladislav Vaintroub have already left Oracle and there are not many old-timers left.

When it comes to MariaDB we are continuing to work making MariaDB 5.3 stable. MariaDB 5.1, 5.2 and 5.3 use the InnoDB-plugin compatible XtraDB storage engine, which already has many of the speed improvements of the new InnoDB plugin, but not yet the windows speed improvement.

We have already merged MariaDB 5.3 with MySQL 5.5 and we will release the MariaDB 5.5 tree shortly. It has taken some time as there are a lot of things done in MariaDB 5.3 that conflict with new code in MySQL 5.5 and we also have to add back some of the features removed in MySQL 5.5.

The next big decision is to decide which version we should release next as a stable release; MariaDB 5.3 or MariaDB 5.5.

The problem is that it's very hard to evaluate how stable 5.5 really is as that 5.5 has not been tested by the public enough and some of the bugs in the public bug system, like the critical Bug #33082, can't be accessed. The bug database shows 294 open bugs for MySQL 5.1 and 150 for MySQL 5.5. I haven't however had time to evaluate all public bugs, so I don't know how severe they really are. A lot depends also how many bugs will be filed during the next few weeks after the GA release.

Note: I corrected the above sentence after getting confirmation that the MySQL team has not yet switched to Oracle's internal bug system. This is scheduled to happen first in the spring. Thanks for helping me get the facts right!

So our plan is to continue to fix the last issues in MariaDB 5.3 and at the same time keep MariaDB 5.5 up to date. Early next year when it's time to decide which one to release as stable, we will have more information about the stability of MySQL 5.5. Let's all hope for the best!

2010-12-14

We have received many requests for a client library for MariaDB (and MySQL) under a license other than the GPL, and decided that it's now time to do something about it.

Some time ago we released the LGPL client library used in MySQL 3.23. This can be used by many applications to connect to MariaDB, MySQL, and Drizzle, but doesn't satisfy all the requirements one would like to see for a true replacement for the current GPL client library.

Minimum requirements are:

It should be binary link level compatible with the current MySQL 5.1 client library. To use it, one would just have to re-link the application with the new library. No re-compilation of the source should be necessary.

It should have all functionality of the MySQL 5.1 client library (including prepared statements).

It should be able to connect to all current versions of MariaDB, MySQL and Drizzle.

It should support all the new functionality in MariaDB's client library like new client functions and pluggable authentication.

The license should be one of BSD/LGPL/Apache.

We have now started to actively plan for getting this done. The plans can be found in our Worklog.

We have already found some companies that are willing to sponsor this project, but not yet enough to get it all done.

If you are willing to donate some time and effort on this, please contact us at'maria-developers at lists.launchpad.com' or 'community at askmonty.org' (we strongly prefer the public maria developers list).

If you are willing to sponsor this effort, please contact us at 'sales at askmonty.org' or use our contact form.

We will put all monetary sponsorship into the Worklog. As with all worklog items, if someone from the community delivers a working solution that is accepted into the MariaDB source they will get 60% of the so far committed money.

2010-10-15

I am now back in Finland after the Monty Program Ab / MariaDB meeting in Istanbul.

As all our meetings, this meeting was also open to the public (except a few closed session about Monty Program Ab internals). As before, we had guests attending, and this time we had people from Facebook, Intel, Galera, Percona, SkySQL and from another database company. PBXT sent their greetings but could not attend because of customer meetings.

During the meetings, we presented our development plans for replication and for the new Knowledgebase system (KB). One of the objectives with the KB is to eventually create a free MariaDB / MySQL manual, but it's also a place to ask questions and find/get answers. We also had some hacking sessions, which gave me time to review the phone home feature that we will add to MariaDB 5.2. We also discussed the dynamic columns feature that is planned for MariaDB 5.3. We also had great presentations from Facebook and Intel that will help us improve MariaDB.

It was also very fun to be part of the SkySQL kickoff meeting that also happened to be in Istanbul (We didn't know that SkySQL would also be there when we planned our meeting. The two teams were mostly meeting at separate locations, but sharing one day together.). It was great to see a lot of old friends from the MySQL time and it felt like we had used a time machine to go 5 year back into the past (especially when "Helan går" was sung many days in a row by people that doesn't really speak Swedish). The old MySQL spirit is very much alive!

We had of course also a lot of good food (mostly kebab in a lot of different forms, including raw), visited a hamam, and did some sport and dancing. Henrik Ingo has documented some (un)memorable pieces in his blog.

What is becoming more and more clear is that MariaDB is not only a Monty Program Ab project. MariaDB is not only getting a lot of contributions from individual developers but there is also an increasing collaboration between companies working on the code. MariaDB 5.2 includes for example code (in no particular order) from Percona, OpenQuery, Primebase, Google, Sphinxsearch and Stardoll, and naturally from Oracle. In 5.3 and/or 5.5 we will in addition work with companies like Galera, Facebook, eBay, Sphaera and of course SkySQL to add many new features into MariaDB.

Next Monty Program Ab / MariaDB meeting will probably be in 6-9 months in Lissabon. Hope to see many of you there!

2010-10-01

MariaDB 5.2 is finally released as gamma (RC). I had hoped to release this in July at Oscon but our new QA person, Philip Stoev, find at the last moment some problems with Aria recovery and virtual columns that we wanted to fix before doing the release.

The new features in 5.2 are quite isolated and as most have been in use by members in the MySQL community for a long time, we don't expect any big problems with 5.2 and we should be able to declare it stable within a few months.

5.2.2 gamma is based on MariaDB/MySQL version 5.1.50. It has the following new features:

A built-in Sphinx client which allows MariaDB to talk to searchd, run search queries, and obtain search results.

We have also done several smaller speed improvements and code cleanups.

We will now continue with working on MariaDB 5.3. As soon as MySQL 5.5 is stable enough, we will create a MariaDB 5.5 tree (based on MariaDB 5.3 + MySQL 5.5) and start working on that.

Please download MariaDB 5.2, test and give us feedback on maria-discuss@lists.launchpad.net or file a bug. We believe it's already quite stable, but we will only know after we have enough users testing it!

2010-07-18

One of the hot topics here at the Community Leadership Summit in Portland is "what is an open source company ?". Simon Phipps has a got a lot of good points on this in his blog about Open Source Business.

We have companies like SugarCRM and Eucalyptus marketing themselves as "open source companies", even while not all of their code is available under an open source license.

To me it's clear that just because some of your product(s) is available under an open source license, you can't claim to be an open source company, as that would make the term meaningless. Under such a definition even Microsoft would be an open source company, as some of their products are now available as open source.

SugarCRM and Eucalyptus are clearly 'open core' companies, not open source companies. While open core is somewhat better than closed source, open core products have all the same disadvantages as closed source if you depend on a single feature of the closed parts for your business. In this case:

- You can't change, modify, port or redistribute the code.- You can't fix bugs or extend the code.- You are locked to the platforms that the vendor provides- You are locked to one vendor.

In other words, the product as a whole should be regarded as a closed source product.

A little background why I feel so strongly about the term "open source company".

When MySQL AB was founded, David's and my intention was to create an open source company. Our definition was back then very simple "all software we produce should be under an open source license". When we took in investors we ensured that MySQL AB would stay as an open source company by putting a clause about this in our shareholder contracts.

David and I did however make a small mistake in that the shareholder agreement only said that "MySQL software" should be kept under an open source license. This allowed the MySQL management in 2006 to release Merlin, the MySQL monitor, as a closed source product, by claiming "this was not based on the MySQL server code". So even if we, the founders, managed to keep the MySQL server free, MySQL AB was only an "open source company" until 2006.

Learning from my mistake and to ensure that Monty Program Ab would always be an open source company, Zak Greant and I created the Hacking business model. Monty Program Ab follows this model and has additionally made a public promise that everything we create and release to our users will be under an open source license.

So what would then be a good definition for calling onces company "an open source company"?

I would like to suggest the following one:

1) You have to be a company that produces software.2) All software the company delivers to its users must be available to everyone under an open source license. This includes all server code that is required to run and use the software.

In addition it would be good if the company could publicly state that all code they produce and release in the future will be under an open source license, but personally I would not require the company's to have to do this as some companies would have a hard time to do this.

At least here at the Leadership summit, the above definition seems to be acceptable to those that I have talked to. Please comment what you think about this!

2010-07-01

I am happy to see that SkySQL, a new home for MySQL talents, is being formed.

SkySQL is being lead by Ulf Sandberg, who was fundamental in building up the MySQL support, training and consulting organization at MySQL AB and will employ many of the original and best MySQL talents!

During the last 2 years, I have seen a lot of the people that originally worked at MySQL AB and who joined Sun together with me, go away in different directions.More than 50 % of them have already left Sun/Oracle. Even if many of them are still working with something that is related to MySQL, others have started new careers doing something completely different.

I have done my best to keep the MySQL core development talent together by employing them at Monty Program Ab, working on MariaDB, but I have not been able to provide a good home for the very talented support, training, consulting and sales people from MySQL AB, who I also care a lot for.

I am happy to see that there is now another home for MySQL talent being formed where they can continue to do what they do best; Providing support and other services around MySQL and also MariaDB.

This is also very good news for all MySQL users as this will ensure, in spite of whatever happens with MySQL at Oracle, that there will continue to be high quality support and services around MySQL and MariaDB.

This also solves the problem for MySQL customers that were using Oracle and chose MySQL partly because they didn't want to depend on only one vendor for all their database systems. Now they can continue to get all services they need from multiple vendors!

The forming of SkySQL is also in line with Monty Programs Ab's strategy; We see ourselves as the technical experts when it comes to MySQL and MariaDB and want to focus on doing development of MariaDB and L3 (critical) bug fixes for MariaDB and MySQL for our customers and partners.

Neither Monty Program Ab nor I have any active role in SkySQL. SkySQL will become a partner for Monty Program Ab, among others, and we will continue with our partnerships with our existing partners as before. By having a lot of independent companies working tightly together we will be stronger than ever before. We will of course have new interesting challenges, but I believe these will be easy to solve as all the companies have the same goal: Give the best possible services to our customers and to keep MySQL and it's future incarnations, like MariaDB, alive.

2010-06-30

I was spending my Midsummer holiday again in the nice Finnish archipelago of Biskopsö, Nagu together with 30+ friends. As always there was a lot of good food (Fish first, meat later) and lot of drinking (Snaps (in various forms, including Salmiakki), beer, wine and Cognac).

What was new was that we had Felix Schupp, from blackray as a guest with us.

As a proof that we also know some other drinking song than the (in)famous "Helan går", we did sing for Felix a drinking song targeting database users:

Yesterday,All those backups seemed a waste of pay.Now my database has gone away.Oh I believe in yesterday.

Suddenly,There's not half the files there used to be,And there's a milestone hanging over meThe system crashed so suddenly.

I pushed something wrongWhat it was I could not say.

Now all my data's goneand I long for yesterday-ay-ay-ay.

Yesterday,The need for back-ups seemed so far away.I knew my data was all here to stay,Now I believe in yesterday.

I hope to see many of you in Oscon again, which will take place 19-23'th of July in Portland. I will be there talking about MariaDB, enjoying some bottles of black vodka with like minded people and maybe, if we can find someone that can lead the singing, participate in singing about lost backups...

2010-05-25

Support for databases has generally been available only per server/year support contract where one has to pay for all master and slaves one has to get support contract for the few needed machines. In some cases one has even to pay per CPU (or core), even if you are not using all CPUs/cores for the database server. There have been a few exceptions, but there have usable been limits depending on company size or offerings that are only available for the first year.

What is wrong with server/year, for larger database installations is that you are also paying for database servers for which you don't need or want support. The value of the offering is thus worth much less than what you are paying for it.

For a introductory price of only US$36,000, we offer 24/7 support for all your MariaDB and MySQL® database servers. We call it "Unlimited" because the price is not dependent on how much you use MariaDB/MySQL®, how many servers on which you deploy MariaDB/MySQL®, the number of CPUs or cores you have in your systems, your company size, the number of beers you happen to have in your refrigerator, or any other arbitrary number you might find in other support contracts.

The support offering also includes things like monitoring tools, knowledgebase articles and some non-support development engineering.

You can also attach add-on services like Backup, Proactive support and support for community storage engines. For a full description of the Unlimited offering, please refer to our FAQ page.

You can, of course, also buy normal server/year support contracts from us. Information about this also exists on the FAQ page.

What is unique with Unlimited compared to all other MariaDB/MySQL® support offerings is that it's backed by the creator, original core developers and architects of MySQL® and MariaDB. Monty Program Ab is also the company that has most competence in all MySQL/MariaDB code. If you want to be sure that you can get a patch/binary to fix your bugs and that your wishes are taken into account for future MariaDB versions, the Unlimited support offering is what you need.

A part of the subscription money is also given back to the community developed parts of MariaDB, like XtraDB or PBXT, to ensure that they can continue their development of integral parts of the MariaDB ecosystem.

Sorry if this post sounds like a commercial. However, everyone here at Monty Program Ab is very excited about this offering. We think it's an excellent value for the money for most bigger MySQL®/MariaDB users, and it will help us to continue develop MariaDB in the ways our users and customers want. We are also giving back to the community upon which we depend.

By subscribing to the MariaDB Unlimited Support subscription, you are ensuring that your current investment in MariaDB/MySQL® is secured, as the money we earn is put into future MariaDB development.

This February we released the first stable version of MariaDB, version 5.1.42, which is our enhanced and backwards compatible MySQL branch/fork. You can download it from the askmonty.org website. Please test it out and comment upon it here or on Launchpad, the code host for the MariaDB project.

The vision for MariaDB is for it to be user and customer driven, as well as more inclusive of community patches and plugins. The 5.1 release already contains, in addition to patches from the community, some features that were sponsored by customers, and wouldn't have happened without that. For instance, Sergey Petrunia recently blogged about a feature we call table elimination that optimizes certain joins by not reading some tables if the information can be inferred from the other table(s). (We've since learned that PostgreSQL 9.0 adds a similar optimization but calls it "join removal"; Oracle 11g also offers table elimination.)

Because of the vision, the roadmap is never set in stone, but will be influenced by user demand, what patches are produced by other developers or what projects customers want to pay for.

I'm planning to spend some time between April 6-21 (right before and after the O'Reilly MySQL Conference) meeting with important MariaDB/MySQL users to define features for the future MariaDB roadmap so that for the next release we can deliver what the MySQL/MariaDB users really want and need.

If you are on the east or west coast of the USA and you think there is something we could do together on MariaDB or MySQL, let me know about your needs at sales@askmonty.org so that we can arrange to meet. I'm especially interested in meeting if you are interested in purchasing some MariaDB/MySQL development to cover your own needs, if you are planning to develop some code of your own that could be contributed to MariaDB, if you want to secure your MySQL investment, or if you want to have advanced end user support backed by the original developers and creators of MySQL.

In MariaDB 5.1 we were able to include a lot of existing work that's been available in the community, such as the PBXT storage engine which is well known and has been RC quality for months already, FederatedX to replace the old MySQL Federated engine, and several patches from Percona and others that add statistics here and there. We also use Percona's XtraDB as the default InnoDB version, which means the InnoDB performance is equal to or better than the InnoDB plugin and the beta versions of MySQL 5.4/5.5.

Since all of this code has been "out there" and even in production for a long time, we believe MariaDB 5.1 is a good and stable product, and brings to you many interesting features now finally in a stable (GA) release. As MariaDB 5.1 is a drop in replacement of MySQL 5.1 (no recompiling of clients or conversion of data is needed) it's trivial to upgrade to MariaDB.

In early March the core developers met in Iceland to plan for the next few MariaDB releases. All MariaDB plans are public, so you can follow and even contribute to them on the askmonty.org wiki: 5.2 plan, 5.3 plan.

For the 5.2 release the focus is still on including remaining existing patches and possibly a few new storage engines.

For 5.3 the optimizer team is focusing on backporting and completing features and optimizations that were left hanging in the cancelled MySQL 6.0 source trees. The MariaDB 5.1 release already contains the "table elimination" I discussed above. When the work for 5.3 is done, MariaDB will finally support the subqueries and JOINs that today are supported on a syntax level, but in practice yield poor performance in some common cases.

Sergei Golubchik is continuing his mission to make the server more pluggable and recently finished a pluggable authentication api (planned for MariaDB 5.2 and also contributed back to MySQL). Kristian Nielsen (the original creator of the Pushbuild system that is used internally for MySQL automated testing) and Hakan Küçükyılmaz are working on further automating MariaDB QA. Much to Kristian's satisfaction, this system is now fully open source and based on Buildbot. Kristian is also working together with Robert Hodges (Tungsten), Seppo Jaakola and Alex Yurchenko (Galera) on a new pluggable replication API. We hope that through this work MySQL replication can take another leap forward, something I've seen many bloggers hoping for.

I hope that you now have a good high level picture of what is going on in MariaDB. If you have comments you are welcome to join the Maria discuss mailing list or #maria on Freenode IRC.

We at Monty Program Ab hope to meet many of you at the upcoming O'Reilly MySQL conference. Almost everyone at Monty Program Ab will be there, holding keynotes, talks, sessions, BoFs (look out for the black vodka) etc. You will, of course, also find many of us at our booth in the Expo hall.

2010-02-21

The mobile world is changing. It's changing faster than the database world did back when MySQL was started and grew to be one of the most widely used database in the world.

Change brings turbulence and it's difficult trying to see the big picture to find the major trends. It also means different philosophies of doing things clash and fight for survival.

There are two large debates at the moment around mobile. One is about open versus closed platforms and the other is around native applications versus web based. One of them is an important philosophical issue, the other one a more technical question of the best way to bring a good user experience to mobile.

The success of the iPhone and the App Store has meant a huge leap for both mobile applications and mobile web. But the iPhone platform is closed. The entire ecosystem is controlled by one company.

On the Internet it's (somewhat) safe to say that the philosophy of open is winning. In fact, open is at the very heart of what the Internet is. As we are moving over to a world where Internet access predominantly is from a mobile device, do we want this to be an open world or a closed one controlled by one company?

This is a very important question for the future of mobile - and the Internet at large!

There's certainly room for both native applications and web based ones in the mobile world. But is it of outmost importance that the platform that grows to be the dominant one for native applications is an open one.

This is why I am now using an Android phone (Hero from HTC).

That is also why I chose to invest in the Swedish cross platform and open source tool for mobile development: MoSync. In this company I see the same potential as the early days of MySQL.

If you are developing a mobile application that you want to work on practically all mobile phones, you should definitely check MoSync out!

2010-01-18

We have now released MariaDB 5.1.41 RC for download and testing. You can download it here. It should be a drop in replacement for any recent MySQL 5.1 release.

We currently have tar archives available and we should have many package formats available shortly.

MariaDB 5.1 is based on MySQL 5.1 codebase, which has been GA for a year now. Most of the things we have included in addition to the vanilla 5.1 code upstream from Sun/MySQL (more on this below) have also been available in the community for quite a long time, so we are quite confident that this is a very stable release; however, being conservative we have had a four month Beta cycle before now going to RC.

I would now like to invite everyone in the community to download and test this release, and if the feedback is good will we then rapidly release a Final release.

On the Askmonty website you can find more information about what's included in MariaDB 5.1. But here are some highlights:

Storage engines

XtraDB replaces InnoDB so you get the Google and Percona enhancements built-in, no need to load it as a plugin. Basically this gives you most of the performance of the upcoming MySQL 5.4 today.

PBXT is included, I'm really happy that this interesting engine finally can get wider adoption thanks to being included and fully supported in MariaDB.

FederatedX by Patrick Galbraith and Antony Curtis is included. This replaces the old Federated from Sun which is not actively maintained by anymore.

And then of course we have Maria, which finally brings you MyISAM users crash safe tables.

Other highlights

The pool of threads feature is based on code from the canceled MySQL 6.0 release. This is good if you have lots of connections to your database and/or you have many short lived queries.

Table elimination is a feature done for a Monty Program customer, that optimizes certain kinds of joins.

In our effort to actively include all the patches floating around in the MySQL community, we have in 5.1 included Microslow and Microsec process patches from Percona. In 5.2 we will continue to include similar patches, there are still many of them floating around the MySQL community that have not yet found their way into a GA release.