SA's MySQL guru on databases, Web2.0

For those of you unaware, we have an unassuming MySQL master in Cape Town. Ian Gilfillan wrote Mastering MySQL 4 and is a regular contributor to databasejournal.com. During the past month I have been increasingly concerned with MySQL performance tuning for high-volume sites and Ian’s articles have saved me on many occasions. I first met him at the Digital Citizens Indaba last year where we co-presented a panel with Mike Stopforth and realised that Ian is not only an authority on database management but he also has a spiritual/philosophical side that seems to help rather than hinder him.

Last week I fired off some questions to him that I wanted to know his thoughts on, and for publication here.

You seem to be the only South African who has published a book on MySQL – is this true and can you explain how this came to happen?

I’m fairly sure it’s true – there are not that many MySQL authors worldwide to keep track of. Writing a book was always an itch I wanted to scratch.

I had something like Alan Ginsberg’s epic poem Howl more in mind as a first work, but MySQL seemed to be easier to write about than naked confessions.

I got to write the book, when, as lead developer at IOL, I was working quite closely with the database. Someone had decided that MySQL was to blame for the performance issues, and wanted to buy Informix to save the day. I disagreed, as no-one had ever really configured MySQL properly, and the experience provided the basis for the book. IOL still runs MySQL.

Given your experience with the platform, what do you consider to be its strengths and weaknesses?

Strengths are its performance, ease of use and configuration, and its widely available free support and skill base. As for weaknesses, integration with enterprise-level tools is probably the main one, but this is something that is changing quite fast. It’s still claimed that MySQL is lacking in features, but this is mainly a legacy of the early situation, when it was, as well as some mistaken statements by MySQL developers at the time, saying that transactions and so on weren’t necessary.

How has the open-source movement impacted on the way you understand intellectual capital and the value software occupies in the publishing environment?

It’s been more of an experiential, pragmatic understanding. I studied IT in the pre-Windows days, and gave it up for a while as I travelled and studied English and Philosophy. When I got back into it, having been out of touch for a while, I naturally gravitated towards open source, as it allowed me access to a far-greater variety of tools, and the potential to understand how they work. The term ‘open-source movement’, as popularly used, covers quite a broad range of schools of thought, from free software to simply the availability of the source code, and even Creative Commons.

A South African like Mark Shuttleworth could never have done what he did, from South Africa, without open source software. I could never have written the MySQL book without access the latest versions of the database to tweak at home. There are so many angles to this I can’t do it justice in a short piece.

MySQL effectively built their business by giving their software away freely, and working to make it extremely popular. Their mixed licence has been hailed as a good business model. Now that it’s reached a certain critical mass, they seem to be tightening things up slightly, no longer releasing precompiled binaries for example.

In a publishing environment, the discussion also overlaps with the build/buy debate. Open source makes building much easier. IOL, which has less resources than most of its competitors, gained an early advantage from its initial decision to build, based upon open source. Building based on open source gives a step-up compared to building using proprietary solutions. At that time, the software was ahead of anything else out there. Things have changed since then – commercial companies have identified the gap and built solutions that would work in the inetgrated modern media environment. Whether to use one of these, or continue to build/customise, is an ongoing debate at many media companies.

You were at IOL in the early days and now consult to them – how has this role changed over the years and what kind of things occupy your mind now in terms of the future direction technology is taking?

I was initially lead developer at MyWealth, a financial services company partly owned by IOL. It dot bombed, and was merged with IOL. I became lead developer at IOL, later IT manager. I left in 2005. I much prefer my current role, as I’m not caught in the day-to-day grind, and can spend more time on things that interest me, or playing with new technology and environments.

My frustrations now are about not being able to implement the changes I’d like, rather than having too many changes to implement. I also would never have installed quite as many Facebook applications if I was still working fulltime!

I’m particularly passionate about Wikipedia, and the multilingual aspect, having done some work on the local South African language wikipedias, and translation. As a collaboration tool in general, wikis are immensely powerful.

What role do you think relational databases will play in the future and are there newer ways of storing and accessing data that might change the way developers and publishers structure their information?

I still see relational databases playing a key role in data access and storage. I haven’t been convinced of most of the ‘newer’ alternatives, many of which seem more like marketing gimmicks. Relational databases still haven’t been properly implemented (from both the software and implementation angles) – the survival of the ‘denormalise for performance’ school is testament to that.

Do you think the Semantic Web is the next big thing and why?

The ‘next big thing’ is too often a vapourware terms for something short-lived. The semantic web is happening though, and it will change everything. But it won’t be as clean and clear cut as most expect or hope, with the legacy mess web we have.

Using Wikipedia as an example, even two years ago people were still questioning Wikipedia. Now it’s unquestionably the premier resource on the web, it’s relevance only questioned by those with vested interests, trolls, or publicity-seekers.

What are your thoughts on social media and Web 2.0 in general?

A wonderful flowering of human potential.

Are you concerned about the environmental impact of large-scale web applications like Google or Facebook, in terms of power usage and other factors?

Yes, but there are many more important issues. PC’s have short lifespans, and big companies are hammering the chip and PC manufacturers about this, so I see the power usage aspect improving quite quickly. The short lifespan has more serious issues as huge quantities of e-waste are dumped.

The focus is there though – I was quite pleased to see that Apple, fingered by Greenpeace as the worst hardware vendor, have responded and announced some changes as a result of the negative attention, and probably won’t be bottom next time the list is released.

Is there a philosophical angle to working with high volumes of data and, if so, how would you describe that?

Perhaps the main difference is that small volumes of data can still be handled manually. Someone can ‘go into the database’ to fix or check if the volumes are small. Large volumes require the system to be correctly implemented from the start, otherwise the problems can grow insurmountable.

What do you think has been the most significant change in the way technology is used over the past five years?

The openness. The understanding that others, outside the core group, can contribute something constructive. Wikipedia and Facebook are prominent examples, as is all the software that allows users to contribute plugins, skins, and so on.