ignite-user mailing list archives

write behind performance impacting main thread. Write behind buffer is never full

Date

Thu, 02 Nov 2017 20:45:50 GMT

I have looked at
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Write-Behind-performance-td5385.html
which seems similar, but does not answer my question.
I am using write behind caching with a single server it has 16 cores and 32
G of memory.
My write behind config is
setWriteBehindEnabled (true)
setWriteBehindFlushSize(1024 * 100)
setWriteBehindFlushFrequency(5000L)
setWriteBehindBatchSize(512)
setWriteBehindFlushThreadCount(1)
My intent with the config is to have plenty of room in the buffer, do large
batches to the database, but never fall more then 5 seconds behind if things
are not busy.
I have a main thread that reads from kafka and writes the cache. Depending
on the message, different caches may be written, but they all use the same
settings in the cache config documented above.
What I am seeing is a degradation in performance on the main thread as the
database fills and the writes on the background threads take more time. The
write behind buffer is not filling, I am not seeing cache critical size
errors.
Other than contention for the system resources, is there any reason that I
should see a slow down on my main thread which does the cache put? Is there
any locking on writing of the grid that happens when the background threads
pull from the write behind buffer?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/