Introduction

The point of this test was to find out whether Linux kernel AI/O (libaio1) can solve the problem that we had with sync I/O, namely, that each write required 8.3 ms, i.e. time to rotate the disk to get the head to the correct position.

Scenario

Write cache on the disk is turned off. NCQ is turned on. Speed of the disk is 7500 rpm.

Data are writen to a raw device (using O_DIRECT) in a linear manner. Each data chunk is 512 bytes long. Size of AIO queue is set so that it can hold all the write requests. All the write requests are posted as fast as possible. The times when individual posts happened are stored. Separate thread waits for notifications of write completion and stores the times when they are received. Check the code below for the details.

Results

We've run the test with all the available I/O schedulers. For each scheduler the first graph shows when a write request was enqueued (red point) and when the notification was received (black point). The second graph shows latencies for individual writed, i.e. notification time minus enqueue time.

Noop scheduler

CFQ scheduler

Anticipatory scheduler

Deadline scheduler

Conclusion

Linux kernel AI/O helps with the 8.3ms issue. It does so by batching ~30 write request into a single unit, so that each individual write doesn't require getting the head to the right position.

The impact of I/O scheduling algorithm on this scenario is more or less zero. The only difference we've seen is that deadline scheduler performs a bit better for few initial write requests.

There are two problems we've encountered:

The enqueueing of writes seems to block every now and then although the size of AIO queue was large enough to hold all the requests in the test.

Although 8.3ms peak for each write was removed, the peaks for first message in a batch are much higher than 8.3 us. (~30-60 ms).