System Techniques to Remove I/O Bottlenecks in Large Query Intensive Applications

Traditionally running complex queries on large database workloads requires expensive high-performance hardware. To solve performance bottlenecks and scaling issues, big SMP machines or large MPP clusters are usually deployed, coupled with significant investment in disk and network I/O subsystems. This trend is running against the grain of MySQL which promotes cost saving solutions. Kickfire is reversing this trend to align with MySQL with its unique system solution, combining hardware and software techniques.

In this talk, Kickfire founder Joseph Chamdani will present the key system techniques that remove the I/O bottlenecks to deliver order of magnitude(s) performance improvements on large query intensive applications. He will discuss specific optimizations they did in their Kickfire storage engine, data structure, memory, hardware accelerator module, and disk I/O subsystems. He will also share his experience in running TPC-H, the industry standard benchmark for decision support systems (DSS), which has some representative workloads of BI applications such as reporting, data warehousing, and analytics.

Joseph Chamdani

Kickfire

Dr. Joseph Chamdani is an accomplished technologist. Prior to founding Kickfire, he was co-founder, Chief Systems Architect, and VP Systems Integration at Sanera Systems, which was later acquired by McDATA. At Sanera and McDATA, Joe led the architecture design for the high-end switching systems and fabric-based storage virtualization and services using hardware acceleration. Before Sanera, he was the lead architect for Sun Microsystems’ UltraSPARC IIIplus and UltraSPARC IV microprocessors for their high-end SMP servers, with a focus on improving database performance. He received his Ph.D from Georgia Tech and currently holds 28 issued patents in domains relating to computer architecture, microprocessors, compilers, networking, storage, and circuit design.