Friday Dec 19, 2008

Kudos to Colin Charles, who has the not easy task of being the program chair of the conference, for pulling the schedule together.

The schedule is not complete yet. If you haven't got either an approval or a rejection, your proposal is still under review. In the previous years, some of my talks were approved as late as February, but we are trying to finish off the schedule much earlier this time.

This is my last blog post for this year. I am going on vacation this evening. No computers, no internet. Therefore, no comments are enabled for this post (there would be nobody to moderate and approve them).

Sunday Dec 14, 2008

Organization for FOSDEM 2009 is in full swing. MySQL has a devroom, for which we want talk proposals. The call for papers ends on January 15, 2009.
There is no registration for the event, but if you want to have a 1 hour speaker slot, please send us a proposal using this form. As usual, we are looking for quality presentations, regardless of the level. We like original talks and we will privilege innovative, creative, and well documented abstracts. You will find some guidelines on how to submit a successful proposal and more info on the event in MySQL Forge wiki.

FOSDEM 2009 is not only a geeky reunion, but also a family opportunity. There will be guided tours for spouses/partners!. If your significant one is not interested in technology or doesn't get a kick out of the intricacies of open source development practice, there are other pleasures to discover in Brussels.

According to MySQL Lifecycle policy, MySQL 4.0 is now in extended support, meaning that only security and extremely severe bug fixes were applied to this release. The present condition ends on December 31, 2008. Then, MySQL 4.0 will join 3.23 in the list of unsupported versions.

The next version to meet the same fate will be 4.1, exactly one year later.

Monday Dec 08, 2008

Enough of one-sided stories. Let's see a different angle of MySQL 5.1.

First, let me thank my colleague Chris Powers for taking a stand in defense of the management. But saying "everyone does so" is not a good explanation. The truth is much more complex and requires some narrative.

MySQL 5.1 didn't start on the right foot. The effort to produce its
features was underestimated, mostly because, at the time when it was
designed, the company was still unearthing the architectural bugs that were
haunting MySQL 5.0.

MySQL 5.0 was GA in October 2005. One month later, MySQL 5.1 started its
alpha stage, while a rain of bugs fell on the freshly released server. When
the version was hastily declared beta 6 months later, the implication of
the architectural problems weren't even found yet.
That's why the beta stage of this version had a long and troubled course.

In September 2007, when 5.1 was declared RC, there were really too many
open bugs and instability problems. The decision of going RC had little
engineering justification. A fierce internal debate followed, with a
prompt decision to review the release criteria, which kept many people busy
for several weeks. The desire of shipping 5.1 GA before the Users
Conference 2008 was rightfully dwarfed by the discovery of new, more
disturbing bugs. Two more RCs were released, while the developers fought to
fix a staggering number of bugs.

More than 3500 bugs affected MySQL 5.1, and by June we had fixed 2300 of
them. There were still some outstanding critical bugs, and Marketing and
Sales were pressing for a release. It was understandable. The economic
situation of Sun was not good, the company had just cut 2500 jobs, and we
needed the new release to boost sales. However, the outstanding bugs were
so bad that the people who were in direct contact with users (Support and
Community Team) strongly objected to a GA declaration at that point in
time. In a joint effort, we identified 40 critical bugs that needed fixing
before going GA. The position of the Support team was clear. They were not
going to support row-based replication and partitioning, unless these bugs
were fixed. The offending bugs were examined by a committee chaired by Sun
Software CTO, and 35 bugs were recommended for immediate fixing.

You can see the result in the changelog. MySQL 5.1.27, which was the intended GA, was not released. Version 5.1.28 fixed 79 bugs, and version 5.1.29 fixed 58 more, and 8 fixes went into 5.1.30. The critical bugs
identified by Support as the basis for supporting the new release were all
fixed. Of the usability bugs identified by the Community team, only two
were deferred to MySQL 5.1.31, and will be available in the next binary
release.

The above explanation makes little sense if we don't explain the criteria
for triaging bugs. Until MySQL 4.1, the release criteria for GA was "zero
known bugs". That rule was artificially ignored in 5.0, with the results
that we all know. But in 5.1, there was no way of shipping with such rule.
MySQL has become a much more complex product than it was in 4.1, and a flaw
doesn't necessarily affect all the users. It is likely the contrary, in
fact, that a defect identified in our QA laboratory only affects a small
portion of our user base, or none at all, like it happened to a bug in
prepared statements, found by a Support engineer, 4 years the feature was
released, and so far claimed by no one (The bug was fixed, BTW).

Back to the triage criteria. There were 1000 open bugs when we went RC. The
only way of getting rid of them was to prioritize the bugs. Classifying the
bugs based on one single parameter, as we did before was not an option. We
needed to be more specific. So all the open bugs were reviewed, and
classified according to defect class, impact, workaround, risk to fix. Not
a perfect system, I grant you, but nonetheless it allowed us to start
fixing the most serious bugs with the greatest impact first. If a bug is a
crash, but it only happens after a convoluted series of actions that our QA
engineers have devised, and has little chance of affecting real users, that
bug gets fixed after a similar one that occurs in a common sequence of
events. If a bug affects two customers, then its impact is automatically
raised, and thus fixed with priority.

The other important point is the risk factor. If a bug fix involves a
substantial change of existing feature, it is deferred to the next version.
When we went GA, we still had some open bugs, but none that prevented the
normal usage of the server. The Community and Support teams are in contact
with customers and other users who have been using MySQL 5.1 in production
for years, and we made sure that the bugs reported by these brave souls
were addressed before the GA release.

You see, there was a lot of work involved in our GA, and the engineering
department, together with Support and Community teams, did an outstanding
job of bringing this release to the public.
Are the processes perfect? No. But they have improved significantly over
the ones that led to MySQL 5.0 GA.

And speaking of that, we know that our engineering process leaves a lot of
room for improvement.

Despite our claim to be the most popular open source database in the
world, our development practices are very much closed source, and our
release cycle is definitely in need of a revision. The obstacles towards
opening the development model are quite a few, and we have been working
hard to meet this goal. The first change was to get rid of the proprietary
revision control system that was a real obstacle towards participation.
Next, we published the Worklogs. Then we removed the MySQL Contributor
License Agreement, which was another serious impediment. Next comes the
release cycle and development model. There are many people at work on this
issue, but this is work in progress and we don't want to raise your
expectations on a quick resolution for these items. The important point is
that everyone agrees on the need for change, and we are working toward this
goal. It's a team effort, which will eventually bear fruits.

In the meantime, please honor the effort of the developers who created new
features like Partitioning, Row-based replication and the Event Scheduler
and have squashed thousands of bugs, and give it a try.

Sunday Dec 07, 2008

I have just started a blog in Italian. Although Italian is my mother tongue, most of my technical writing is in English. Since my tasks are more widespread than a single country, it was easier and simpler to write in English straight away.

After Sun acquisition, the MySQL presence in Italy is growing, and then it makes sense to employ my birth speaking abilities to spread the word in my country.

Wednesday Nov 26, 2008

Notes of one week with MySQL community in US and France

Background

I wanted to attend the first edition of the Open SQL Camp in Charlottesville, from November 14 to 16. For some mysterious reason, a four days plane ticket to any place in the US costs EUR 2,500, but if I stay 8 days, it costs EUR 800. Considering that I was in Frankfurt until November 8, the most sensible thing to do was flying to the US as early as possible and meet as many communities as I could. Among the ones I asked, Boston and New York answered enthusiastically, and then, serendipitously, I found an unexpected group in Baltimore, just before the last leg to Charlottesville.
I sent all groups a list of ten topics to choose from:
\* MySQL 5.1 features
\* Using MySQL partitions in practice
\* testing with MySQL Sandbox
\* MySQL Community How To
\* Recent community additions to MySQL code
\* Creative cross-language programming with MySQL
\* MySQL Proxy wizardry
\* advanced replications techniques
\* advanced Lua programming for MySQL Proxy
\* How to fake a list of ten with only nine topics

Boston MySQL meetup

The Boston user group has a clockwork organization. They were able to accommodate my request for a meeting quickly and efficiently. The meeting takes place in one of the MIT buildings, and a large load of pizza and soft drinks was delivered on the spot, so the meeting started with everybody happily fed.
The topic chosen by this group was MySQL Sandbox. The presentation lasted 50 minutes, followed by 30 minutes of discussion.

New York

MySQL meetup

The MySQL user group in New York is quite large. The venue for the meetings is a pub in South Manhattan that can accommodate 40 people comfortably seated, or 60 if there is standing audience.
The room was filled to capacity for the talk about MySQL Proxy. The participation was intense and widespread. We spent about 2.5 hours discussing the subject and related topics. There is much interest in the Proxy as a component for HA solutions.
Host of the evening was Philip Antoniades, who is widely known in the community.

PHP meetup

The second day in New York should have started with a presentation at Columbia University, which was postponed, and thus the ad hoc presentation made by Dups for this occasion will have to wait for another event.
Dups joined me in late morning. We had lunch with Ronald Bradford, and then spent the afternoon discussing the status of the community.
Dups presentation at the PHP group was a success. The room (the same as the previous day) was packed. The contents were very much appreciated and discussed afterwards for a few hours.
Philip did not attend, because (lame excuse!) his wife was delivering his second child, just while Dups was delivering his first presentation as Community Manager.

Baltimore MySQL meetup

Transfer to Baltimore using the Greyhound. We found out that the Amtrak train cost $400 for the ride, while the Greyhound was $70 for the two of us. The ride was very useful. It allowed us to discuss the community needs in more detail and to make plans for a MySQL Proxy extension that will deal with memcached transparently (or at least Dups says he will do it. We'll see how good his Lua skill will be in a while).
Some excitement came in the cab ride from the bus station to the hotel, when the cab driver got lost twice and had to ask for directions.
Fortunately, we paid a flat fee for the ride, and no damage was done. The host of the MySQL meetup is Greg Haase, who is known in the community as the author of a convincing use case for MySQL 5.1. He is also a supporter of MySQL Sandbox, for which he is creating some additional features.
The topic in Baltimore was MySQL Proxy, with a very passionate debate following, and I am happy to say that I was able to address all the concerns.
We ended the evening with a dinner at a popular restaurant, packed with very noisy football enthusiasts.
The locals were genuinely surprised - no, better, amazed! - that we came by bus, and we actually survived!

Charlottesville Open SQL Camp

Public transportation from Baltimore to Charlottesville was not an option, so we rented a car. While I drove, Dups interviewed me efficiently, and took notes about the wishes and needs of the community team. It resulted in a tentative plan for engaging the community in a better web environment. Stay tuned for news on this subject in the next weeks.

Friday

The initial day was about registration, introductions, and finalizing the schedule for the next day. The efficiency of the organization was impressive. Considering that Baron has done most of the work on his own, with little help from others is really commendable, and an example for who wants to organize something similar.
The schedule was hand made, quite useful, and fit for this kind of event.
http://www.flickr.com/photos/datacharmer/3031964690
There were about 60 attendees, out of 100 registered people. The level of the audience went from intermediate to advanced, with very few beginners.

Saturday

The meat of the conference was on Saturday. The efficiency of Baron's organization (with some sponsors help) was evident with the huge breakfast provided to the attendees.
Th event was dominated by MySQL/Drizzle presence. Brian Aker's keynote on the ubiquity of open source databases caught everyone's attention.
Some of the sessions were very innovative (libdrizzle, with its evolutionary features and backward support for MySQL, is one of the breakthrough topics), others, like the PostgreSQL ones, were more traditional and down to earth.
The lightning talks were much appreciated.
After the normal sessions, there was planning for the hackfest on Sunday and an unplanned prize giving, when we awarded Baron with a T-shirt signed by all the participants.
The inner force behind the organization was Sheeri, who has organized the wiki, the donation page, a legal entity for donations, and a sergeant major attitude which, together with her personal example, involved many people in the organizational tasks.
Networking was the major activity of the day. The lunch (not enough food for the latecomers, I was told) and the dinner (too much food) catalyzed the attendees together in temporary bonds that often led to serendipitous findings.

Met a lot of people, and expanded my Facebook and LinkedIn portfolios.

Sunday

The hackfest was a collection of coding efforts in different directions. Several bugs were found and fixed during the day, in both MySQL and Drizzle, while an ambitious group of web developers tried to create a startup in 6 hours (they failed, but just barely, and it was a splendid effort!)

Washington

The travel to Washington brought more brainstorming, facilitated by some traffic jams. Dups drove, while I collected the ideas, and we arrived to Washington with a fully detailed plan about enhancing the MySQL Users Conference with some surprising things that you will hear about, probably next year.

That was the end of the good news. The meeting with the local Sun rep was canceled, and we were on our own. Following ill advice from Jay, we left the car at the airport and went looking for a locker for our luggage. There was none, and we were sent to a museum downtown that had lockers. After a quick train ride, we were at the museum, where our luggage was minutely and messily inspected, only to find out that the lockers did not have locks.
So we had lunch at the museum cafeteria and went back to the airport, where we scanned the ether for free wifi, and spent some productive hours online.
We parted company in the afternoon and that was the end of my US campaign.

Paris

MySQL meetup

Paris meetup is held very far away from the center. It's a different town actually, which gave me time to catch some sleep in the cab ride.
The attendance was not high, but everyone was interested, and all resisted
until the end. How many? We were 10, which is decimal ten, not binary two. The funny thing is that the next day, at least ten more people told me that they would have come to the meetup, if it hadn't been so far away from the center!

The topic was, once more, MySQL Proxy, and the questions asked were a bit different from the usual ones. Someone wants to use MySQL Proxy as a front end to hide the complexity of using different versions of MySQL in a sharded environment.
After Proxy, Nat Makarevitch, a local celebrity presented a long topic on server optimization, which lasted until 10:30pm, when finally we had food and drinks.

Customers conference

More talks, met more people, was asked to explain Proxy features again, discussed the Infobright roadmap, collected more business cards, and fell asleep in an armchair, only to be woken up suddenly for an incoming meeting.
Oh, well! it was an interesting exercise, but don't ask me to do it again soon!

Tuesday Nov 25, 2008

If you hear a distant rumble in the air, it could be either Dups testing his new mountaineering gear, or, more likely, MySQL 5.1-GA seeding to the mirrors.

We promised that it would come soon, and it's coming. All internal QA automated tests were passed, a batch of additional manual tests with the packaged binaries (called "smoke tests" internally) were performed to make sure that no embarrassing errors were left, and now the train of bits is traveling to the mirrors. Once the mirrors are loaded, expect the announcement, within a few days.

Monday Nov 03, 2008

Jay Pipes has joined the Drizzle project, and we have been busy looking for a replacement.
Cloning Jay Pipes brought serendipitous results. You know that cloning is not an exact science, and we could not get a perfect replica. Perhaps it was a MySQL replication issue. We should try with row-based logging, next time. Anyway, we were looking for a replica of Jay's skillset, and instead we found a match for the kind of enthusiasm that brought Jay to start working for MySQL in 2006 and then to embrace Drizzle this year.

Exit Jay, whom we will miss dearly. He was the first person that I met in the MySQL community who went to work for the company. That's when I started thinking that maybe I could become an employee as well. Best of luck to Jay, and I am sure we will hear from him in the MySQL/Drizzle ecosystem

Enough with the past, fast forward to the present. Who is this Dups anyway?

Dups is a fantastic mixture of technical skill, enthusiasm, vision, and reliability. He's a doer with an attitude. He has a solid experience as a web programmer, an infectious sense of humor, a gift for simplifying complex problems and for describing them in plain English. Ah, and to solve them! Dups has a well deserved reputation of Getting Things Done .

I met him in cyberspace a few weeks before he came to work for MySQL, and we exchanged some very inspiring email messages. Since then, I have always had a high respect for his skills and his opinion, and when I finally met him in person I was confirmed that I was in front of a remarkable person.

That's why, when he sent me a message saying "how tempting ... ", minutes after I wrote the blog post in search of a new team member, I rushed to contact him in IRC and ask if he was serious.

He was! And the process of interviewing him and checking his requirements, his attitude towards community work, his set of values, was truly enjoyable. Dups has a vision, and he will show it in public very soon.
He was not the only candidate. Within minutes, I got not one, but three candidates, and three more applied the next day. So it was a tough choice. Thanks to the ones who applied, and sorry I could not hire all of you!

Saturday Nov 01, 2008

We are almost there. Four days more, and then it's over. The CfP for the MySQL Users Conference 2009 ends on November 5th, 2008.

We have already received more proposals than we got last year, and (yes!) many of them are innovative.

Thanks to all the submitters.

I have still a few wishes, though.

There are still no proposals or very little on

The event scheduler

OpenOffice integration

Java

The plugin architecture

MySQL Proxy

If you have a technical story to tell about one of the subjects above, don't be shy, and come forward!
And of course, if you have some innovative ideas on any topic related to MySQL, feel free to submit a proposal.

Monday Oct 27, 2008

I have always been respectful of rules and regulations. I respect the artists, and I am glad to provide them with a compensation for their work.

I buy movie DVDs, and music CDs.

And then I get angry.

More and more times it happens that, inserting a brand new DVD in the player, I am given a nasty advertising against illegal copies, warning me that piracy is a crime.

Why me? I am the one who bought this bloody DVD, am I not? So why are you inflicting this punishment on me, with this stupid advertising that my player can't skip?

I know why. Because you can't inflict that on the ones who are actually doing the illegal copying. And what's more annoying is the thought that the illegal copies don't have this advertising.

I, the one who have paid good money for this movie, can't watch the movie the way I want it. And you know what I need to do to watch the movie again without the additional pain? Why, rip it, of course! Once I convert the legal copy into an illegal one, I can do whatever I want with the movie.

Some other things that drive me mad? Go to a movie theater, pay 8 EUR, and you have to endure 25 solid minutes of advertising. Hey! The movie theater was supposed to be for the movies. Ads are for TV! No way. Nobody is listening. And the illegal copies are ads free ...

And then there is the prices. I bought DVDs ten years ago. They cost 15,000 lire, or about 7.5 EUR. That was in the period when few people had DVD players, so I suppose that they would sell much less than what they sell today. And what's the price today? Regular price is 16 EUR, for a movie just out of the theaters schedule. The top performers at the box office cost even 25 EUR.
WTF?

It's really hard to stay honest. Piracy seems very appealing, and it's the entertaining industry that is pushing me in that direction.

Are you around these locations?
If yes, please attend the meetups, or drop me a note.

These events are happening because several people have worked on the issue behind the scenes. Thanks to Sheeri K. Cabral, Angelo Rajadurai, Josie Kaufman, Mark Rubin, Tom Coveney, Philip Antoniades, Ronald Bradford, Greg Haase for taking care of the organization and apologies if I forgot someone.