The MariaDB Blog The Official Blog of the MariaDB Project

Tag Archives: MariaDB

The MariaDB project is pleased to announce the immediate availability of MariaDB 5.5.32. This is a Stable (GA) release. See the Release Notes and Changelog for detailed information on this release and the What is MariaDB 5.5? page in the AskMonty Knowledgebase for general information about the MariaDB 5.5 series.

New functionality

On Windows, it is now possible to use named pipes if server enables them. To connect to the server using named pipe, pass pipe=<pipe_name> URL parameter or property. The pipe filename name should be passed without the \\pipe\<machine> prefix. (CONJ-36)

It is now possible to do SSL validation for self signed certificates, using serverSslCert, (provide CA certificate for the server’s certificate). It allows to avoid SSL handshake errors (untrusted CA) when working with self-signed certificates, and is more secure than the existing trustServerCertificate option. (CONJ-48)

Bugs fixed in this release

Some of the bugs fixed include the following:

Change assertion in PacketOutputStream.startPacket() to be IOException, so it can be caught. “Last packet not finished” condition can happen , if underlying stream is broken (e.g write operation failed) (CONJ-34)

If custom SocketFactory creates an already connected socket, do not execute Socket.connect() (CONJ-37)

Ensure proper locking in MySQLCallableStatement.execute(), in scenarios that use single connection simultaneously from multiple threads. Hold the connection lock for the duration of 3 operations – setting parameters , executing statement, and reading off output parameters) (CONJ-38)

(Performance enhancement) Changed DatabaseMetadata.getImportedKeys() not to use information schema, but to parse the output of “show create table” instead. This considerably speeds up several scenarios involving Hibernate (CONJ-35)

Fixed ResultSet.getDisplaySize() for character columns – due to UTF-8 usage it was 3x more than expected (size in bytes, instead of size in chars)

Fix bug in prepared statement parsing code (skipping next character after backslash ‘\’)(CONJ-44)

My fellow testers and others who run RQG tests on MySQL flavors might be interested in some additions that are being used for MariaDB testing. While none of them is a major breakthrough, maybe they will make somebody’s life a little easier.

RQG Introduction

A quick introduction for those who have never heard of RQG, but are still curious what this blog post is about.

RQG stands for Random Query Generator, also known as randgen — an open-source product, available under the GPL v2 license. Quoting its home page on Launchpad, it is a “pseudo-random data and query generator that can be used to test any Perl DBI, JDBC or ODBC-compatible SQL server, in particular MySQL, but also JavaDB and PostgreSQL”.

The framework was created by my former colleague Philip Stoev, who not only developed a great tool, but also managed to make it a very important part of the MySQL testing routine. I think by now all MySQL-based products use it to some extent; and to those that don’t, I recommend to try it out.

Along with data and query generation, the framework features “pluggable” modules of different types, which perform various checks during or after test execution. This article is mainly about adding more of them to the existing set. When you see ‘validator’ or ‘reporter’ below, it means the type of the corresponding module.

About these MariaDB additions

All listed additions have been pushed into the main randgen tree on Launchpad, along with some other stuff that is more specific for MariaDB and thus less interesting for the general public. However by the time you are reading it, those versions might already be a little outdated. Since the randgen tree is now self-regulated, I feel like it can easily fall into the “too many cooks” trap, so I have chosen the most conservative approach for merging into the main tree. Whenever a change affects anything but a MariaDB-specific component, I will send an email to the randgen mailing list with the description, and if there is at least one voice against the change, it will not be merged. The downside is that it makes merges a bit fussy, so will not be happening often. Moreover, rejected changes will never make it to the main tree.

Most recent versions, as well as all new and rejected patches can be found in our work tree.

Galera mode

RQG is very useful for running tests on a Galera cluster, but doing it is a pain: start all the nodes, run an instance of RQG for each node, check results, don’t forget to shut down the nodes at the end… Probably most of us who have had to do it by now have scripted some parts of it, but still, it felt all wrong, so I finally decided to make it “native”. Now Galera cluster can be started by RQG itself, with a topology configurable via a command-line option.

This is just the first drop, which is likely to be fixed and improved later.

CheckFieldValue validator

If you need a simple one-time check that does not deserve implementing a separate validator, you might try this one.

Adding a specifically formatted comment to a query triggers a validation. The validation is defined in the comment itself: it says which field in which row in the result set should be checked, and provides the condition (currently simple numeric comparison: ‘=‘, ‘<‘, ‘>‘, ‘<=‘, ‘>=‘).

Here is a real example of how it can be used: MDEV-4578. A user reports sporadic bizarre values in I_S.processlist.time field. You can of course write a validator for it, and I initially did, but I soon realized that it was inconvenient since I wanted to keep modifying the checking query. So instead I added the SELECT from the I_S to the grammar, with the condition that the time field must be less than some ridiculously big value.

The set of possible validations will be expanded on need basis.

SlaveCrashRecovery reporter

The reporter is for testing crash-safety of master=>slave replication. It crashes the slave periodically, restarts it and makes sure it starts all right.

BinlogConsistency reporter

The reporter checks that the contents of the binary log(s) correctly reflects the data at the end of the test flow. After the main test flow is finished, the reporter creates a data dump of the server, shuts down the server, starts a new one with the same options, but on a clean data directory, replays binary logs of the first server into the new one, creates a data dump of the new server, and compares it with the previous dump.

CrashRecovery reporter

A new implementation of the old Recovery reporter, more suitable for current server versions. Instead of hard-coded set of options for the restarted server, it uses the same configuration as the initial server had.

Comparison of VIEW Algorithms

RQG allowed to configure a view type to be used during the test, but you could only create views of the same type on servers that you were comparing. I found it to be a shortage while testing the extension of MERGE views implemented in MariaDB 10.0 (see MDEV-3862): I needed to compare MERGE views on one server with TEMPTABLE views on another server. So, --views1 and --views2 options have now been added. It might be not the most elegant solution, but it is in line with other similar options in RQG, and it does the trick.

Multiple redefining grammars

I found that my set of redefining grammars started growing beyond reason because I only could provide one at a time, and often I would need to combine several, so I would have to create a new grammar for each combination.
So, now you can add as many --redefine= options as you want, all grammars will be applied on top of the main grammar.

New grammar keywords _basetable and _view

The RQG keyword _table means any table or view from the test dataset. Often it’s good enough, but sometimes it badly increases the number of erroneous statements, e.g. with the “not a base table” error. Now tables and views can be differentiated in a grammar depending on what the goal is: _table can still be used for both tables and views, while _basetable will only pick tables, and _view will only pick views.