Search This Blog

Mysql deadlock during delete query and index

I had written a hierachical lock manager using mysql and once a thread finishes it was unlocking by deleting the locks. The 20 thread perf test was fine but daily some 100-200 of deadlock exceptions would come randomly on some nodes. Now each thread is working in isolation so it doesnt makes sense to get deadlock, the query was like

delete from hie_path_locks where customer_id=? and thread_id= ?

Finally after some hit and try and troubleshooting I found that we had an index on customer_id only and when 10 thread on same workgroup would try to read locks to detect conflicts they would do

select * from hie_path_locks where customer_id=? and thread_id= ?

apparently mysql default isolation of REPEATABLE_READ would take locks on even rows that are read, check http://www.mysqlperformanceblog.com/2012/08/28/differences-between-read-committed-and-repeatable-read-transaction-isolation-levels/

Adding an index on customer_id,thread_id instead of just customer_id solved the random deadlocks.

I have a thumbnail generator that launches multiple processes and the correct way to shut it down is to send kill -HUP to the parent process. To automate I had to write a pid file from python, it was a piece of cake
def writePidFile():
pid = str(os.getpid())
f = open('thumbnail_rabbit_consumer.pid', 'w')
f.write(pid)
f.close()

About Me

I like writing good code and improve myself every day by learning new things. I like working for startups as they give you tons of challenges than working for a big company, not sure if you guys feel it but simple programming quirks give me tons of joy. This blog is about the things that I worked on and found worth sharing with fellow programmers who are googling for solutions like me.
This weblog does not represent the thoughts, intentions, plans or strategies of my employer. It is solely my opinion.