tag:blogger.com,1999:blog-23496029.post6570367643577255104..comments2016-10-14T08:19:04.011+02:00Comments on MySQL Musings: Binary Log Group Commit in MySQL 5.6Mats Kindahlhttp://www.blogger.com/profile/07528917029894926261noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-23496029.post-85026170266531341312016-06-23T13:58:50.850+02:002016-06-23T13:58:50.850+02:00It was delivered in MySQL Server 5.6.6. See https:...It was delivered in MySQL Server 5.6.6. See https://dev.mysql.com/doc/relnotes/mysql/5.6/en/news-5-6-6.html.Mats Kindahlhttp://www.blogger.com/profile/07528917029894926261noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-56847309147189565812016-06-21T20:04:10.061+02:002016-06-21T20:04:10.061+02:00Thanks Mats for this valuable information.
When yo...Thanks Mats for this valuable information.<br />When you refer to Mysql 5.6 June, which version is it?Franck Leveneurhttp://www.blogger.com/profile/15272615761769092257noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-155419694255930262014-12-18T09:46:34.281+01:002014-12-18T09:46:34.281+01:00hi
i had one doubt...
is the &#39;binlog_cache&...hi <br /><br />i had one doubt...<br /><br />is the &#39;binlog_cache&#39; parameter related to the &#39;transaction cache&#39; you mention ?<br /><br />are both of them the same ?<br /><br />http://dev.mysql.com/doc/refman/5.1/en/replication-options-binary-log.html#sysvar_binlog_cache_size<br /><br /><br />vishnuraohttp://www.blogger.com/profile/13865191899113945498noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-30751099792344414092012-06-06T14:03:37.956+02:002012-06-06T14:03:37.956+02:00Kristian,
I added references to your blogs on bin...Kristian,<br /><br />I added references to your blogs on binary log group commit (and to your blog) as well as to Mark&#39;s original post that started it all.Mats Kindahlhttp://www.blogger.com/profile/07528917029894926261noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-33615617515724322812012-06-06T13:05:11.065+02:002012-06-06T13:05:11.065+02:00I have no idea if I &quot;beat&quot; MariaDB, but ...I have no idea if I &quot;beat&quot; MariaDB, but one of the goals with the<br />implementation was to avoid changes to the handlerton interface so<br />that existing engines should not have to be changed to work.<br /><br />Regarding the changes to the handlerton interface that you point to:<br />the thd_requested_durability is not part of the handlerton interface,<br />so storage engines do not have to be changed to take advantage of<br /><i>binary log group commit</i>. According to the Bazaar logs,<br />flush_logs handlerton member was added by &quot;Brian&quot; (not sure if it is<br />Brian A or Brian M) in 2006, so that is not really anything new in<br />5.6. Whether the storage engines themselves support group commit or<br />not is a completely different issue.<br /><br />The design I discussed in earlier posts were partially implemented,<br />but shelved since there were implementation issues making it more<br />difficult to work well. Basically, the IO_CACHE does not support<br />pwrite very well, affecting performance, so decided on a traditional implementation to ensure that it got into 5.6.<br /><br />MariaDB also have a binary log group implementation and I think<br />that is good since it offers users some alternatives on what to<br />choose.<br /><br />I haven&#39;t kept up to date with the latest changes in MariaDB, but<br />AFAICT, the only similarities between the MariaDB implementation and<br />this implementation is that both use queues (a single queue in MariaDB), mutexes, and condition<br />variables. Apart from that, the two implementations are very<br />different. For example, the MariaDB implementation does not have any<br />stages, and commits are handled by the threads themselves instead of<br />batched.Mats Kindahlhttp://www.blogger.com/profile/07528917029894926261noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-37785786151421801042012-06-06T11:13:42.969+02:002012-06-06T11:13:42.969+02:00&gt; Existing storage engines benefit from binary ...&gt; Existing storage engines benefit from binary log group commit since there<br />&gt; are no changes to the handlerton interface.<br /><br />Mats, you are not telling the full story ;-) Did you intend to keep it a<br />secret, hoping to beat the MariaDB group commit? :)<br /><br />You extend the handler interface in two ways. You add<br />thd_requested_durability() so that InnoDB will not fsync() at commit();<br />without this, storage engines will have no group commit in their commit()<br />method. And 5.6 also adds handlerton-&gt;flush_logs() which is necessary for<br />XA recovery to work. This is basically equivalent to the commit_ordered() and<br />commit_checkpoint_request() methods that MariaDB adds, it is good.<br /><br />I am really disappointed that you completely fail to mention my group commit<br />work in MariaDB. You basically ditched your original design and instead<br />re-implemented the MariaDB group commit algorithm. I think you even did a<br />Ph.D. at University, so you should know proper decency in regards to proper<br />attribution to prior work. Shame on you!knielsenhttps://launchpad.net/~knielsennoreply@blogger.comtag:blogger.com,1999:blog-23496029.post-59049946410464636862012-06-06T07:53:45.565+02:002012-06-06T07:53:45.565+02:00Dathan, Robert, Thanks!
Robert,
No,binary log gr...Dathan, Robert, Thanks!<br /><br />Robert,<br /><br />No,binary log group commit does not introduce any new serialization issues, there are just the old ones.<br /><br />I&#39;m not really talking about the binlog format in the post, but there are no changes to the binary log format done <i>for implementing binary log group commit</i>, but there are changes to the binary log format to implement other features, such as the multi-threaded slave.<br /><br />I assume that you were actually asking about the handlerton interface. There are some new informational functions in the server that InnoDB uses to get information about the transactions, but no changes to the handlerton interface.Mats Kindahlhttp://www.blogger.com/profile/07528917029894926261noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-22505631969657998922012-06-06T03:21:34.503+02:002012-06-06T03:21:34.503+02:00Excellent work and long awaited by many of us. I&...Excellent work and long awaited by many of us. I&#39;m looking forward to using sync_binlog=1 for all masters all the time. <br /><br />I was a little confused in the paragraph under &quot;Transactions Galore&quot; where you discussed isolation. Does group commit introduce any new serialization issues that were not present in 5.5 and previous releases? <br /><br />Also, your write-up implies there are no changes to binlog format. Is that the case?Robert Hodgeshttp://www.blogger.com/profile/05379726998057344092noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-17758063155511589142012-06-06T02:21:31.852+02:002012-06-06T02:21:31.852+02:00Fantastic! I love learning new things. Great job!Fantastic! I love learning new things. Great job!Dathan Pattishallhttp://www.blogger.com/profile/00356367514107959723noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-24309033209214422992012-06-05T22:53:39.314+02:002012-06-05T22:53:39.314+02:00Hi Mark,
I assume that you&#39;re referring to th...Hi Mark,<br /><br />I assume that you&#39;re referring to the max queue time in the flush stage. The flush stage leader doesn&#39;t wait at any point, but if new transactions keep queuing up in the queue it is necessary to <em>stop</em> skimming of transactions from the flush queue to prevent a high latency of those transactions that have been flushed and are waiting for the leader to move over to the sync stage. This is what the max flush queue time is used for: to prevent the leader from continue to skim of transactions.Mats Kindahlhttp://www.blogger.com/profile/07528917029894926261noreply@blogger.comtag:blogger.com,1999:blog-23496029.post-30646224997050087472012-06-05T22:25:27.799+02:002012-06-05T22:25:27.799+02:00This sound excellent but I don&#39;t understand ho...This sound excellent but I don&#39;t understand how the leader decides to wait (or not wait) during the flush stage. Can you provide more detail on that? I know it is more complex than always waiting for the timeout to expire as that would make singled-threaded workloads slow.<br /><br />Do you use the number of threads currently doing InnoDB prepare to help decide whether the flush stage leader should wait?Mark Callaghanhttp://www.blogger.com/profile/09590445221922043181noreply@blogger.com