tag:blogger.com,1999:blog-37614418780964624092019-01-31T01:58:39.550-08:00moradan blogIvan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.comBlogger42125MoradanEnhttps://feedburner.google.comtag:blogger.com,1999:blog-3761441878096462409.post-39814600787317377282018-09-15T05:37:00.000-07:002018-09-15T05:37:03.771-07:00CTE (with) in Oracle and PostgresMajor part of my career to the moment passed working with databases and mostly with Oracle and Postgres. Several years ago I was really surprised by the difference in handling CTE (common table expressions) in these databases. From that time I spread this knowledge and recreated a sample for it several times. The last one was this week and I decided to store it somewhere - this mostly dead blog seems like a good place ;-) <br />So here is a really simple case of using CTE on Oracle. <br /><pre><code><br />create table test_table(<br /> id int primary key,<br /> foo varchar2(10)<br />);<br /><br />insert into test_table(id, foo)<br />select rownum, 'val'||rownum from dual<br /> connect by rownum&lt;=100000;<br /> <br />explain plan for select * from (select * from test_table) sel where id=42;<br />select * from table(dbms_xplan.display);<br />--| 0 | SELECT STATEMENT | | 1 | 20 | 1 (0)| 00:00:01 |<br />--| 1 | TABLE ACCESS BY INDEX ROWID| TEST_TABLE | 1 | 20 | 1 (0)| 00:00:01 |<br />--|* 2 | INDEX UNIQUE SCAN | SYS_C00328137 | 1 | | 1 (0)| 00:00:01 |<br /><br />explain plan for with sel as (select * from test_table) select * from sel where id=42;<br />select * from table(dbms_xplan.display);<br />--| 0 | SELECT STATEMENT | | 1 | 20 | 1 (0)| 00:00:01 |<br />--| 1 | TABLE ACCESS BY INDEX ROWID| TEST_TABLE | 1 | 20 | 1 (0)| 00:00:01 |<br />--|* 2 | INDEX UNIQUE SCAN | SYS_C00328137 | 1 | | 1 (0)| 00:00:01 |<br /></code></pre><br />You can see that from the resulting execution plan there is no difference between CTE and sub-select. <br />But here is similar sample on Postgres: <pre><code><br />create table test_table(<br /> id serial primary key,<br /> foo text<br />);<br /><br />insert into test_table(foo)<br />select 'val'||generate_series from generate_series(1, 100000);<br /><br />explain analyze select * from (select * from test_table) as sel where id=42;<br />-- Index Scan using test_table_pkey on test_table (cost=0.29..8.31 rows=1 width=12) (actual time=0.037..0.038 rows=1 loops=1)<br />-- Index Cond: (id = 42)<br />-- Planning time: 0.053 ms<br />-- Execution time: 0.048 ms<br /><br /><br />explain analyze with sel as (select * from test_table) select * from sel where id=42;<br />-- CTE Scan on sel (cost=1541.00..3791.00 rows=500 width=36) (actual time=0.025..32.644 rows=1 loops=1)<br />-- Filter: (id = 42)<br />-- Rows Removed by Filter: 99999<br />-- CTE sel<br />-- -> Seq Scan on test_table (cost=0.00..1541.00 rows=100000 width=12) (actual time=0.016..5.859 rows=100000 loops=1)<br />-- Planning time: 0.069 ms<br />-- Execution time: 35.305 ms<br /></code></pre> Difference is dramatic. <a href="https://www.postgresql.org/message-id/24672.1319561025%40sss.pgh.pa.us">Quoting Tom Lane</a> (real Postgres expert) "CTEs act as optimization fences. This is a feature, not a bug. Use them when you want to isolate the evaluation of a subquery." But a case when CTE should be used in Postgres is a theme for another post (maybe, so you here in another 4 or 5 years :-))Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com1tag:blogger.com,1999:blog-3761441878096462409.post-52264702112168106852015-08-14T14:53:00.000-07:002015-08-14T14:53:05.783-07:00Benchmarking an interview question<i>Author benchmarks a super-simple method with a dumb implementation from his recent job interview. Results of this benchmark contradict with assumptions. The process of benchmarking produces some public benefit.</i><br /><br /><br /><a name='more'></a>So, on a recent job interview it was suggested to write implementation for a really simple method<br /><br /><span style="font-family: Courier New, Courier, monospace;">public int sum(int x, int y);</span><br /><span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: inherit;">The main question for implementation is surely dealing with integer overflow (we were speaking about Java programming language). As it was left for me to decide what to do in case of overflow I suggested a standard way - throwing exception. In Java8 there is a standard method that does exactly this - Math.addExact. However I needed to&nbsp;</span>re-implement<span style="font-family: inherit;">&nbsp;it myself in order to use on older versions. To ease possible migration I used the same&nbsp;</span>ArithmeticException that is used in the 8th version of standard library.<br /><br />Another difficulty that is natural for a job interview is that I could look some nice bit-shifting trick on the Internet, but needed to write that implementation myself. Since we are dealing with ints it is really easy to write correct implementation just casting arguments to longs inside the method:<br /><br /><span style="font-family: Courier New, Courier, monospace;">static int sum(int x, int y) {</span><br /><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; long xl = x;</span><br /><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; long yl = y;</span><br /><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; long result = xl + yl;</span><br /><span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; if (result &lt; Integer.MIN_VALUE || result &gt; Integer.MAX_VALUE) {</span><br /><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; throw new ArithmeticException("integer overflow");</span><br /><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; }</span><br /><span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; &nbsp; &nbsp; return (int) result;</span><br /><span style="font-family: Courier New, Courier, monospace;"><br /></span><span style="font-family: Courier New, Courier, monospace;">&nbsp; &nbsp; }</span><br /><span style="font-family: Courier New, Courier, monospace;"><br /></span>This method is correct - i wrote some tests to prove it. But my assumption was that it will be much slower than the standard one, or some using some bit-shifting operations. <br /><br />After the interview I have benchmarked it together with the standard one and was very surprised to see that this dumb implementation is even marginally better than the standard one on my laptop (Java8 on the x64 CPU and Linux OS). I decided to dive a bit deeper with the perf profiler and support for it in the JMH, but discovered that this support has been broken for my version of linux kernel and perf tool. I <a href="http://mail.openjdk.java.net/pipermail/jmh-dev/2015-August/001996.html">reported this regression</a>&nbsp;and it <a href="http://mail.openjdk.java.net/pipermail/jmh-dev/2015-August/002006.html">was fixed the same day</a> and <a href="http://mail.openjdk.java.net/pipermail/jmh-dev/2015-August/002016.html">released</a> on the next day. So even such a very simple interview question may produce some public profit.<br /><br />Back to this method, here are the result of benchmarking with the perfnorm profiler (some results omitted for brevity):<br /><br /><span style="font-family: Courier New, Courier, monospace;">Benchmark &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Mode &nbsp;Cnt &nbsp; Score &nbsp; &nbsp;Error &nbsp;Units</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.addWithLongs &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; 15 &nbsp; 4.288 ± &nbsp;0.043 &nbsp;ns/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.addWithLongs:·CPI &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;avgt &nbsp; &nbsp;3 &nbsp; 0.313 ± &nbsp;0.014 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.addWithLongs:·branches &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; &nbsp;3 &nbsp; 7.921 ± &nbsp;2.076 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.addWithLongs:·cycles &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; &nbsp;3 &nbsp;12.057 ± &nbsp;2.867 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.addWithLongs:·instructions &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; &nbsp;3 &nbsp;38.571 ± &nbsp;8.450 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.mathAddExact &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; 15 &nbsp; 4.306 ± &nbsp;0.100 &nbsp;ns/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.mathAddExact:·CPI &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;avgt &nbsp; &nbsp;3 &nbsp; 0.356 ± &nbsp;0.072 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.mathAddExact:·branches &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; &nbsp;3 &nbsp; 6.782 ± &nbsp;0.589 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.mathAddExact:·cycles &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; &nbsp;3 &nbsp;12.102 ± &nbsp;3.079 &nbsp; #/op</span><br /><span style="font-family: Courier New, Courier, monospace;">AddExactBenchmark.mathAddExact:·instructions &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; avgt &nbsp; &nbsp;3 &nbsp;33.972 ± &nbsp;2.560 &nbsp; #/op</span><br /><div><br /></div><div>Let me comment these results a bit. The difference between methods is really small - it varies from run to run (maybe I should try more forks), but my dumb implementation is always better than the standard one. My next assumption (the very first one - that standard method is faster in all cases - was not true) is that this will not be the case on x86 and other 32-bit hardware and JVMs. This dumb implementation has more instructions than the standard one (instructions result), but exact instructions seems cheaper since CPU executes more of them per cycle (CPI - cycles per instruction - result that is the inverse of what I'm talking about) and as a sequence - a bit less cycles are used for execution. <br /><br />Another interesting result (probably it would be better to observe it with something like perfasm, but I have no such experience) is that this dumb implementation has more branches, than the standard one.<br /><br />The source for this benchmark may be <a href="https://github.com/isopov/isopov-jmh/blob/master/src/main/java/com/sopovs/moradanen/jmh/AddExactBenchmark.java">found on Github</a>.</div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-24165017869115315732014-03-05T15:04:00.000-08:002014-03-06T02:03:37.769-08:00Specifying the size for StringBuilder (jmh)Several years ago I started doing some microbenchmarks. I don't think that I really progressed since than, but at least some errors became obvious. So I decided to take one of my old <a href="http://moradanen.sopovs.com/2010/11/specifying-size-for-stringbuilder.html">benchmarks</a>&nbsp;and reimplement it using the JVM library. Here are the results that I got:<br /><br /><pre>Benchmark Mode Samples Mean Mean error Units<br />c.s.m.j.StringBuilderSize.expandingSize avgt 15 7566.477 374.611 ns/op<br />c.s.m.j.StringBuilderSize.predefinedSize avgt 15 5640.386 133.672 ns/op<br /></pre><br /><br />And <a href="https://github.com/isopov/isopov-jmh/blob/master/src/main/java/com/sopovs/moradanen/jmh/StringBuilderSize.java">here</a> is the code for it. (I have also posted some other jmh-based benchmarks in that repo).Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-37430154483460967762014-02-23T12:22:00.002-08:002014-02-23T12:22:58.531-08:00Freemarker loading taglibs from classpathIt is pretty common problem to try using various jsp taglibs from <a href="http://freemarker.org/">freemarker</a> templates<a href="http://stackoverflow.com/questions/20100276/adding-spring-security-jsptaglib-to-a-freemarker-template-issues-with-controll">[1]</a><a href="http://sourceforge.net/p/freemarker/patches/37/">[2]</a><a href="http://stackoverflow.com/questions/12069602/adding-spring-library-for-usage-of-jsp-taglibs-for-security-in-freemarker">[3]</a><a href="http://jira.magnolia-cms.com/browse/MAGNOLIA-3092">[4]</a>. There is <a href="http://freemarker.org/docs/pgui_misc_servlet.html">some support</a> for this in freemarker, but this support is a bit ugly - it requires dancing with jar files like placing them in WEB-INF/lib folder and there is no easy way to use taglibs just from the classpath. At least - there was no such an easy way. After a day of debugging and investigation I figured the solution that works pretty good for me. The key is to override just the two methods in the ServletContext that is used by freemarker TagLibFactory. To do this I used standard dynamic proxy, but other solutions are possible two. After this taglibs can be referenced by their paths in the classpath - like:<br /><pre><code class="xml"><br />&lt;#assign security=JspTaglibs["/META-INF/security.tld"] /&gt;<br /></code></pre><br />for spring-security taglib instead of the usual <br /><pre><code class="xml"><br />&lt;#assign security=JspTaglibs["http://www.springframework.org/security/tags"] /&gt;<br /></code></pre><br />And the InvocationHandler that can be used to create the proxy for the ServletContext is here: <br /><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.lang.reflect.InvocationHandler;<br />import java.lang.reflect.Method;<br /><br />import javax.servlet.ServletContext;<br /><br /><br />public class ServletContextResourceHandler implements InvocationHandler {<br /> private final ServletContext target;<br /><br /> private ServletContextResourceHandler(ServletContext target) {<br /> this.target = target;<br /> }<br /><br /> @Override<br /> public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {<br /> if ("getResourceAsStream".equals(method.getName())) {<br /> Object result = method.invoke(target, args);<br /> if (result == null) {<br /> result = ServletContextResourceHandler.class.getResourceAsStream((String) args[0]);<br /> }<br /> return result;<br /> } else if ("getResource".equals(method.getName())) {<br /> Object result = method.invoke(target, args);<br /> if (result == null) {<br /> result = ServletContextResourceHandler.class.getResource((String) args[0]);<br /> }<br /> return result;<br /> }<br /><br /> return method.invoke(target, args);<br /> }<br />}<br /></code></pre><br />And the complete solution can be found in my pet-project <a href="https://github.com/isopov/fan/blob/6e2042ddce9d67fea37768aab266b33dc4c68ff9/fan-web/src/main/java/com/sopovs/moradanen/fan/WebApplicationConfiguration.java">here</a>.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-9368324882867489712014-02-16T05:21:00.001-08:002014-02-16T05:21:51.986-08:00System.arraycopy versus simple copy in a loopProgramming in Java daily I rarely need to use arrays. But sometimes this happens and even more often I trap into reading libraries code that heavily use arrays. So proper working with them is the real question for me. And recently I stumbled upon a code that used plain loops for copying values from one array to the other. So I decided to check whether this is viable option when we have System.arraycopy method. I wrote a simple <a href="https://github.com/isopov/isopov-jmh/blob/master/src/main/java/com/sopovs/moradanen/jmh/ArrayCopy.java">benchmark</a> using excellent JMH library.<br /><br />And so what I have got as a result:<br /><br /><pre><code><br />Benchmark Mode Samples Mean Mean error Units<br />c.s.m.j.ArrayCopy.loopcopy avgt 15 1.463 0.160 ms/op<br />c.s.m.j.ArrayCopy.systemcopy avgt 15 1.457 0.112 ms/op<br />c.s.m.j.ArrayCopy.loopcopy sample 10322 1.450 0.018 ms/op<br />c.s.m.j.ArrayCopy.systemcopy sample 11071 1.351 0.013 ms/op<br />c.s.m.j.ArrayCopy.loopcopy ss 15 1.653 0.925 ms<br />c.s.m.j.ArrayCopy.systemcopy ss 15 1.657 0.788 ms<br /></code></pre><br /><br />So there is no real difference in hand-written loop and using native method. But I really doubt that even with these results we should ever prefer hand-written loop instead of System.arraycopy native method. Here is a short list of ad-hock parameters that may come into play:<br /><br /><ul><li>Different CPU. I used the most powerful current desktop CPU for running by benchmark. Your code may be run on very different CPU with very different result - maybe even on differnt architecture. Who knows - maybe in just several years the major part of java application servers will be run on ARM-servers.</li><li>Different hardware cache usage. It is no doubt that System.arraycopy is optimized for the proper usage of CPU caches and not interfering with the other code running on the same CPU concurrently. For the loop - you cannot be absolutely sure.</li><li>Different JVM implementation. With only three major JVM-implementations currently - OpenJDK (but again - I doubt that ARM-version can be called absolutely the same JVM in this specific context), IBM J9 and Dalvik (not a JVM actually, but your library that uses arrays can easily get to running on it) - you cannot be sure in the JIT already. But also there are other JVMs, like Azul Zing, Azul Zuzu,&nbsp;Excelsior JET and many others.</li></ul><div>So, nonetheless that I was not able to observe any difference in running hand-written loop and native built-in arraycopy method - I prefer the latter.</div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-2025142256385319602013-12-27T14:15:00.000-08:002013-12-27T14:15:19.953-08:00Learning Vaadin 7 reviewSome time ago I was proposed to review the second edition of the "<a href="http://www.packtpub.com/learning-vaadin-7-second-edition/book">Learning Vaadin 7</a>" (<a href="http://www.amazon.com/Learning-Vaadin-7-Second-Edition/dp/1782169776">On Amazon</a>) book. Shame on me for holding it for so long. Previously I have used Vaadin 6 and was really impressed by its features and simplicity of doing real desktop-like web-applications. But at the same time every now and then I was frustrated by some of the methods and mostly by the methods returning plain Objects. Fortunately now these type-safety problems are solved with the Vaadin 7.<br />But what about the book? It seems as really good introduction to Vaadin for junior developers:<br /><br /><ul><li>It has a really extensive and balanced introduction that defines Vaadin place in the Java and Web ecosystems. Actually, I think that it is worth reading on its own.</li><li>It proceeds with detailed instructions on the development environment setup with a dive into production setup.</li><li>It is really detailed with all the basic and not so basic concepts of building Vaadin applications clearly explained.</li></ul><div>The drawbacks of the book, that I can name are:</div><div><ul><li>Examples use ant+ivy as a build system that seems less widespread to me, but it may have its benefits since it leaves much less area for "build magic" and gives more control and understanding of the build. Also information about Maven is given in a separate chapter.</li><li>Book seems like a one time reading for me. It can not be used as reference - but again, it is clearly stated that this is not a goal. Also it will motivate readers to search for information as it should be done after the book is read.</li></ul><div>As an overall conclusion, I would not buy such book for myself, but I will probably recommend it to newbie developers.</div></div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-73390586726990892522013-11-25T07:50:00.002-08:002014-01-05T06:12:52.292-08:00Java8 HashMap is not compatible with Java7... in a way...One of the things that shine in Java (not the only one!) is its performance. And it constantly improving. One of the performance improving features of Java8 is <a href="http://openjdk.java.net/jeps/180">an improvement to HashMap</a> and many related standard hashing collections. Actually from the title of this JEP (JDK Enhancement Proposal) "Handle Frequent HashMap Collisions with Balanced Trees" it is clear that this enhancement uses Comparable and Comparator functionality. If class do not implement Comparable interface or not Comparable with each other classes are stored in the map - Comparator based on the hashcode is used. But if class implements Comparable - this implementation is used.<br /><br />So based on all this here is the very simple question. What does this code output on the JVM version 7 and on the JVM version 8? <br /><a name='more'></a><br /><br /><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.util.HashSet;<br />import java.util.Set;<br /><br />public class HashMapTest {<br /> public static void main(String[] args) {<br /> Set&lt;Foo&gt; hashSet = new HashSet&lt;&gt;();<br /> for (int i = 0; i &lt; 1000; i++) {<br /> hashSet.add(new Foo(i));<br /> }<br /> for (Foo foo : hashSet) {<br /> if (!hashSet.contains(foo)) {<br /> System.out.println("Bingo!");<br /> break;<br /> }<br /> }<br /> }<br /><br /> public static class Foo implements Comparable&lt;Foo&gt; {<br /> private final int id;<br /><br /> public Foo(int id) {<br /> this.id = id;<br /> }<br /><br /> @Override<br /> public int compareTo(Foo o) {<br /> // This is badly broken compareTo method<br /> return 1;<br /> }<br /><br /> public int hashCode() {<br /> // This is valid hashcode but that will leave only 4 buskets<br /> return id % 4;<br /> }<br /><br /> public boolean equals(Object obj) {<br /> // This is default equals() method generated by Eclipse<br /> if (this == obj)<br /> return true;<br /> if (obj == null)<br /> return false;<br /> if (getClass() != obj.getClass())<br /> return false;<br /> Foo other = (Foo) obj;<br /> if (id != other.id)<br /> return false;<br /> return true;<br /> }<br /><br /> }<br /><br />}<br /></code></pre><br /><br />On JVM version 7 this code outputs nothing. But on the JVM version 8 it outputs "Bingo!". Actually the very first object tested (with id=0) can not be found in the Set based on the broken Comparable implementation.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com3tag:blogger.com,1999:blog-3761441878096462409.post-79562999860773137652013-11-21T08:31:00.000-08:002013-11-21T08:33:16.885-08:00Benchmarking Guava's ImmutableMapSo I decided to bench (with the help of the excellent <a href="http://openjdk.java.net/projects/code-tools/jmh/">JMH suite</a>) <a href="https://code.google.com/p/guava-libraries/">Guava</a>'s <a href="http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/ImmutableMap.html">ImmutableMap</a>. The first my try revealed some not expected results.<br /><br />Source of the benchmark: <style>#gist7583378 div.gist-data {height: 400px; overflow:auto} </style><script src="https://gist.github.com/isopov/7583378.js?file=MapsBenchmark1.java"></script>Results: <script src="https://gist.github.com/isopov/7583378.js?file=MapsBenchmark1.results"></script>What I was not expected is that Guava's ImmutableMap with single entry is much faster to create than standard Collections.singletonMap(). Another surprise was in no performance benefits from using ImmutableMap with more than one entry comparing to HashMap and HashMap inside standard Collections.unmodifiableMap().<br /><br />But this was test for creation of the map only. So here is the second version with simple work of the "contains" method: <script src="https://gist.github.com/isopov/7583378.js?file=MapsBenchmark2.java"></script>Results: <script src="https://gist.github.com/isopov/7583378.js?file=MapsBenchmark2.results"></script>The result is almost the same.<br /><br />What is different in Collection.singletonMap comparing to the Guava's ImmutableMap is cached keySet, entrySet and values fields. I decided to make a little test to know whether not-initialized (and thus pointing to null) and not-used fields impact performance. <script src="https://gist.github.com/isopov/7583378.js?file=ObjectWithEmptyFieldsBenchmark.java"></script>Results: <script src="https://gist.github.com/isopov/7583378.js?file=ObjectWithEmptyFieldsBenchmark.results"></script>As it is seen the difference is significant. So probably the difference in Collections.singletonMap() and Guava's ImmutableMap is caused by these three fields even if they are not used.<br /><br />In order to compare performance using these caching fields (or at least one of them) I made the third version: <script src="https://gist.github.com/isopov/7583378.js?file=MapsBenchmark3.java"></script>Results: <script src="https://gist.github.com/isopov/7583378.js?file=MapsBenchmark3.results"></script>Guava's ImmutableMap is still a bit better than Collections.singletonMap() but the difference is much less than in the first version. I assume that if using all the fields cached in standard version - it will be better.<br /><br />Another surprise it no performance benefits from using Guava's Maps.newHashMapWithExpectedSize(). 128 entries is definitely enough to produce at least one rehash of the HashMap but probably they do not damage performance in a significant way, or there was really no rehashes since the expected size of the HashMap was inferred form the subsequent code but the JVM.<br /><br />And the last surprise was that HashMap inside the Collections.unmodifiableMap() may have a better performance not only comparing to the Guava's ImmutableMap, but also comparing with exactly the same HashMap without any covers.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com1tag:blogger.com,1999:blog-3761441878096462409.post-82014768051038582662013-06-29T04:56:00.002-07:002014-02-16T05:32:00.851-08:00Synchronized vs threadlocal SimpleDateFormat based on JMHMicro-benchmarks are really hard. If you have seen <a href="http://moradanen.sopovs.com/search/label/Silly%20benchmarks">some</a> in this blog and you think that at least some of them are correct - you are wrong! ;-) Actually I don't think that I would ever really need to do some micro-benchmarks. To the moment all performance optimizations that I have done were about SQL (mostly), or not loading all the data into memory, but loading and processing it in chunks instead, or introducing some caching, or introducing multi-threading, or just rewriting the algorithm from scratch (introduction of multi-threading is usually the complete rewrite too). <br /><br />But to have some fun (and to have at least anything to write in blog about) I sometimes do such stupid benchmarks. Of course it is better for this fun activity to have at least some similarity to the real world. Recently (maybe not really recently) Oracle Java&nbsp;Performance&nbsp;Team has open-sourced their <a href="http://openjdk.java.net/projects/code-tools/jmh/">framework for micro-benchmarks</a>. I decided to try it and rewrote <a href="http://moradanen.sopovs.com/2012/01/synchronized-vs-threadlocal.html">my benchmark</a> for patterns of multi-threaded usage of the standard SimpleDateFormat (why actually use it since we have joda-time for Java &lt;= 7 and new shiny standard datetime api for Java &gt;= 8?). So here is the result of this benchmark on the Core i7-3632QM (laptop) processor.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-9SNNS8-IjLQ/UbXMS8_U65I/AAAAAAAAAis/PsulIeOmHRM/s1600/simpledateformat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-9SNNS8-IjLQ/UbXMS8_U65I/AAAAAAAAAis/PsulIeOmHRM/s640/simpledateformat.png" height="408" width="640" /></a></div>The code for this can be <a href="https://github.com/isopov/isopov-jmh/blob/master/src/main/java/com/sopovs/moradanen/jmh/SimpleDateFormatBenchmark.java">found here</a>.<br />I have tried the same benchmark on the Core i7-3770K (desktop) processor also. And here is the graph:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-HynypqHCN-s/UbYlISh7Z2I/AAAAAAAAAi8/BrhXI92jCHE/s1600/simpledateformat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-HynypqHCN-s/UbYlISh7Z2I/AAAAAAAAAi8/BrhXI92jCHE/s640/simpledateformat.png" height="384" width="640" /></a></div>What we can see is the expected result, except that for the synchronized access to the single SimpleDateFormat the results are not very smooth and stable. So here is the same graph with the synchronized access to the SimpleDateFormat measured in 30 iterations (5s each) instead of 10.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-53ESKgzFLp4/UboQDgLYrhI/AAAAAAAAAjQ/MTmMPvCk_AQ/s1600/simpledateformat2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-53ESKgzFLp4/UboQDgLYrhI/AAAAAAAAAjQ/MTmMPvCk_AQ/s640/simpledateformat2.png" height="400" width="640" /></a></div>This measurement has not cleared anything actually... What I think is that JIT-compilation of of the version with single synchronized SimpleDateFormat is not stable, so the result of the measurement is stable across different iterations of the same run, but is not stable across different runs and maybe contrintuitive for different number of threads in the same run (different methods =&gt; different acts of compilation =&gt; different code).Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com1tag:blogger.com,1999:blog-3761441878096462409.post-68657282793289727872013-06-10T12:28:00.000-07:002013-06-10T12:33:29.886-07:00First attempt with scalaI decided to try <a href="http://scala-lang.org/">scala</a>&nbsp;once again (all previous attempts did not last long). I have not read any book about scala so far, and this is just dummy attempt, so I tried to use everything that I already know from Java. To my luck I have stumbled on the <a href="https://github.com/GrahamLea/scala-spring-hibernate-maven-webapp">project</a>&nbsp;by Graham Lea that uses maven for building, spring-mvc as the web-framework and hibernate as a database access layer. This project actually works, so it is possible to try it moving from one working state of the project to the other.<br />What I have done in <a href="https://github.com/isopov/scala-spring-hibernate-maven-webapp">my fork</a>&nbsp;so far:<br /><br /><ul><li>maven-eclipse-plugin is a deprecated way of generating Eclipse project files for the maven project, since Eclipse itself has a pretty good support for maven-basd projects. However inside the stable version of <a href="http://scala-ide.org/">Scala-ide</a>&nbsp;a rather old version of Eclipse is used internally, so maven support is not so great, but still it is good enough to not use maven-eclipse-plugin. So I removed its configuration from the project.</li><li>Selenium tests was the part that didn't work for me in this project (probably because I'm on Linux), so I removed them and started tests using spring-test library.</li><li>The latter required spring framework upgrade. Also I have upgraded Hibernate and scala and scala-maven-plugin to the latest versions. This required a bit of editing the scala sources and changing the scala-ide installation since different versions are required for scala 2.9.x and 2.10.x.</li><li>I migrated the project from using Hibernate SessionFactory to EntityManager and spring-data-jpa (scala traits can be used as spring-data repository interfaces). It seems that this way of using Hibernate is a bit more up-to-date and is free from issues like <a href="https://hibernate.atlassian.net/browse/HHH-7486">this one</a>, that I trapped into some time ago.</li></ul><div>Glitches that I have stumbled over are about Scala-ide and both can be seen in&nbsp;<a href="https://github.com/isopov/scala-spring-hibernate-maven-webapp/blob/f911d99ddc23a65726ced769c78faaf431dce15f/src/test/scala/com/example/scalawebapp/controller/AbstractControllerTest.scala">this file</a>:</div><div><br /></div><div><ul><li>Organizing imports action removes imports in use making compilation to fail (in the file mentioned&nbsp;SpringJUnit4ClassRunner import is removed, but I have seen this in other situations too)</li><li>Formatting the source remove comments (in the file mentioned the "//TODO..." comment is removed)</li></ul><div>So now is the time to read something about scala and all the surrounding magic. Any thoughts on what book or resource should I read?</div></div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-86604547892348269982013-05-09T12:13:00.002-07:002013-06-10T12:34:08.611-07:00Github vs Bitbucket<a href="https://github.com/">Github</a> totally owns the open-source movement today. And this situation impacts enterprise development. But as far as I can tell <a href="https://bitbucket.org/">Bitbucket</a> is much better for proprietary development in private repositories. Here are the arguments: <br /><br /><ul><li>Option to <a href="https://bitbucket.org/site/master/issue/3338/git-allow-option-to-enable-disable-force">prevent non-fast-forward pushes</a>. It even allows to prevent non-fast-forward and still allowing removal of feature-branches! As far as I know, this may require a bit of configuration in standalone git repositories. <strike>No such option on Github</strike>. <b>UPDATE:</b>&nbsp;<a href="https://twitter.com/cobyism/status/332159018019733505">Requires email</a> to support@at@github.com but the response is fast &nbsp;- even in my not straightforward case.</li><li>Pricing model. <a href="https://github.com/plans">Github offers</a> unlimited&nbsp;collaborators&nbsp;form the start and&nbsp;additional&nbsp;private repos for additional money. <a href="https://bitbucket.org/plans">Bitbucket offers</a> unlimited private repos from the start and additional collaborators for additional money. I think the latter option is better - division of project in modules residing in separate repositories is the technical task. And technical decision person should have freedom to do the best. Technical staff usually does not make financial decisions. And extending the team is really that kind of decision that could not be made without considering financial side. So I think if choosing what side of the plan to make&nbsp;dependent&nbsp;on money - number of collaborators is better because without any Github or Bitbucket this side does depend on money. And number of repositories, if not made&nbsp;artificially&nbsp;dependent on money, does not depend on money at all.</li></ul><div>So there are only two of them - <strike>but the first I would consider as a show-stopper, and</strike>&nbsp;the second really&nbsp;annoys leading to several unrelated projects in the single repository.&nbsp;</div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-40865129932588169832013-04-04T14:08:00.003-07:002013-06-30T06:32:23.503-07:00Use newer Java FTW!<br />One of my recent tasks was trying to convince client, that we should target the 7th version of the JVM, and not the 5th. I failed. Actually I don't think that such task is possible unless there are no any real reasons in using older Java. Nevertheless here is the text, that I wrote:<br /><br /><a name='more'></a>What Java implementation are we going to use? The most used one is Oracle.<br />Oracle Java5 reached its official end of life in Oct 2009<br />Oracle Java6 reached its official end of life in Feb 2013 (there will be no public updates anymore, nor any security updates)<br /><br />Reasons to use the newest java possible:<br /><ul><li>This year Spring 4 will be released with minimum Java version 6. Most libraries and frameworks are in process of dropping the Java5 support. If Application is going to be supported more than for one year it is reasonable to ease that future support as much as possible.</li><li>Recent Eclipse versions are targeted for Java6 and Java7 and are tested only on these versions. Developing new project with the old versions of tools and libraries should be really justified and reasoning is well-understood by everyone in the team.</li><li><strike>Most recent Eclipse WTP (tooling for developing web-applications - we are going to build the web-application, right) supports only Java6+ out-of-the-box.</strike> <b>UPDATE:</b> to get rid of warning that Java5 is not supported by WTP you just need to change version="3.0" in your web.xml to version="2.5" or below since servlet 3.0 requires Java6 to run. Not very intuitive, but after that support is good.</li><li>Many small aditions to the platform are made with every release and while it is definitely possible to live without them, but performance, readability and maintainability of the source will suffer. To name just a few that I really needed, but the target Java version of the project was too low are:</li><ul><li>&nbsp;(Added in Java 6) @Override annotation on interface method implementations. Hard minutes and sometimes even hours of debugging...</li><li>(Added in Java 6) NavigableSet and NavigableMap that are so easy to use for any non-trivial logic on ordered data.</li><li>(Added in Java 7) Try-with-resource. In any reasonably large application that I've seen there are hundreds of places without proper closing of resources - some of them are yet to be discovered bugs and some of them are just a junk that hinder the real problems. Any tooling that helps avoiding these problems is greatly appreciated and especially such a good tooling built right into the language.</li><li>(Added in Java7) ForkJoin framework - hardly ever used in dayly development, but if there is a proper place it is hardly possible to think of anything better. It could be used on Java6 with a lot of trouble involved, but its benefits are so high that in some cases it is reasonable. Of course if there is no option of moving to Java 7.</li><li>(Added in Java7) Diamonds instead of generics. Makes the code so much more readable and thus maintainable.</li></ul></ul><br />This is absolutely not a complete list of benefits, but only the list of those that I have felt myself.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com3tag:blogger.com,1999:blog-3761441878096462409.post-21601902627899728212012-11-21T04:39:00.000-08:002013-06-30T06:32:34.077-07:00To the hashcode() and equals() problem<div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-iZjA2YBVEho/UKuAmGEEHUI/AAAAAAAAAXA/h1cks1H_nqo/s1600/thift-hashcode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-iZjA2YBVEho/UKuAmGEEHUI/AAAAAAAAAXA/h1cks1H_nqo/s640/thift-hashcode.png" /></a></div><br />Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-39708280873876209012012-11-21T04:35:00.001-08:002013-06-30T06:32:12.346-07:00Oracle connect by prior with order siblingsTo not forget:<br /><pre><code class="sql"><br /><br />with hier as (<br /> select 'Foo' as name, 1 as id, null as pid from dual union all<br /> select 'FooBar', 2, 1 from dual union all<br /> select 'Bar', 3, null from dual union all<br /> select 'BarFoo', 4, 3 from dual union all<br /> select 'FooZo', 5, 1 from dual union all<br /> select 'FooBarFoo', 6, 2 from dual union all<br /> select 'FooZoo', 7, 1 from dual union all<br /> select 'FooZooo', 8, 1 from dual<br />)<br />select LPAD(' ', 2*(LEVEL-1))||name<br />from hier<br />start with pid is null<br />connect by prior id=pid<br />order siblings by id;<br /><br /></code></pre>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com1tag:blogger.com,1999:blog-3761441878096462409.post-75129326696144651602012-07-09T09:11:00.000-07:002012-11-13T05:04:31.937-08:00Guice a bit more convenient than Spring in injecting circular dependencies in session beansIt is not very easy to find, but <a href="https://jira.springsource.org/browse/SPR-5955">Spring does not support circular dependencies</a> for scopes other than Singleton (default scope for Spring). That may be very frustrating. There are some workarounds, but since I met this limitation right at the very&nbsp;beginning&nbsp;of the project, I just moved to Guice, that supports session scoped beans with circular references out-of-the-box.<br /><br />If you want to try it yourself, or try some workarounds for Spring - I have created a small simple<a href="https://github.com/isopov/guice-vs-spring-session"> github project</a> as an illustration.<br /><br /><b>EDIT</b>: Actually now there is an easy workaround to this intentional Spring&nbsp;behavior. You may use javax.inject.Provider of session-scoped beans inside other session-scoped beans and introduce circular dependencies. This will not work in Spring 3.1.0-3.1.2 due to a regression, but this bug is fixed in 3.1.3 Spring. I have updated the <a href="https://github.com/isopov/guice-vs-spring-session">example</a>. For session-scoped beans this workaround is really very easy since you may call provider.get() any number of times in one session and you will always recieve one and the same bean. This may be a bit trickier with prototype-scoped beans or any custom scope but this idea may in some form apply to them too.<br /><br />So Spring is not really owned and now allow you to do the same thing. But still you need to remember that Spring guys do not like circular dependencies in beans and they have reasons for that.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-73387235979616880062012-07-09T08:24:00.000-07:002012-07-09T08:24:06.968-07:00Thrift exposed via webapp servletThrift services can be exposed via servlets. Now the <a href="https://builds.apache.org//job/Thrift/javadoc/org/apache/thrift/server/TServlet.html">TServlet</a> class is part of the core thrift package in java. It is relatively easy to use, but there are some nuances.<br /><br />First - it requires some Processor and Protocol to be used. Example of such usage maybe found <a href="http://chamibuddhika.wordpress.com/2011/10/08/creating-a-osgi-based-thrift-service/">here</a>. Though it is about OSGI Servlets, not web applications - this example is easy and small. However it is stated there that due to&nbsp;violation&nbsp;of Servlet specification (no default constructor) it is not possible to use this servlet in web application. Actually it can be used.<br /><br /><br /><ul><li><span style="background-color: white;">We can either use it directly with some covering code, like in </span><a href="http://stackoverflow.com/questions/1276082/constructor-in-servlet/2659597#2659597" style="background-color: white;">this stackoverflow answe</a><span style="background-color: white;">r.</span></li><li><span style="background-color: white;">We can use any standard way of delegating servlets like <a href="http://www.springsource.org/spring-framework/">Spring</a>, or <a href="http://code.google.com/p/google-guice/">Guice</a> and maybe some other frameworks that most likely are already used in your application.</span></li></ul><div>I have created some small test project to illustrate these options (hand-written delegating code, Spring and Guice) and placed it on <a href="https://github.com/isopov/thrift-test">github</a>. Maybe it will be useful to someone except me ;-)</div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com1tag:blogger.com,1999:blog-3761441878096462409.post-23053200654245487562012-02-18T12:27:00.000-08:002012-02-21T12:00:59.490-08:00Fedora 16x64 and JavaFX 2.1.0-betaJavaFx is currently unavailable for Linux x86_64. So we will use 32bit one. The steps are:<br />1. Download&nbsp;jdk-7u3-linux-i586.tar.gz from <a href="http://www.oracle.com/technetwork/java/javase/downloads/jdk-7u3-download-1501626.html">here</a> (I have&nbsp;&nbsp;jdk-7u3-linux-x64.rpm already installed and I don't want to mix 32 bit jvm to my package system)<br />2. Unpack it to /opt<br />3. Add to your profile (e.g. /etc/profile)<br /><code></code><br /><pre><code>export JAVA_HOME="/opt/jdk1.7.0_03"1<br />PATH=$JAVA_HOME/bin:$PATH;export PATH<br /></code></pre>4. Download something like&nbsp;javafx_sdk-2_1_0-beta-b13-linux-i586-14_feb_2012.zip from <a href="http://www.oracle.com/technetwork/java/javafx/downloads/devpreview-1429449.html">here</a> and unzip it to /opt<br />5. Install packages to support 32 bit java (maybe not needed if you have installed rpm):<br /><pre><code>sudo yum install libgtk-java.i686 gtk2-engines.i686 libxslt.i686<br /></code></pre><br />6.You also need libavcodec.so.52 to fully support HTML5 in JavaFX. It is not shipped nor in the main Fedora repo, nor in Rpmfusion. I have found it in the <a href="http://atrpms.net/">http://atrpms.net/</a>. However this repo may come in conflict with Rpmfusion. Anyway, with this repo installed you may do <br /><pre><code>sudo yum install libavcodec52.i686</code></pre><br />That's it! Now you may install Netbeans 7.1 and try javaFX samples. Or try them in Eclipse with <a href="http://efxclipse.org/">this</a> plugin. I recommend to look through these <a href="http://www.apress.com/downloadable/download/sample/sample_id/1231/">examples</a>&nbsp;(code for <a href="http://www.apress.com/9781430242574">this</a> book - have not read it and I'm seriously doubt&nbsp;whether&nbsp;it is worth reading it). They are not very good, some of them do not compile (but are easily fixable), some of them do not work on Fedora x86_64, but I have not found anything better yet.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-13828111439934421522012-02-09T13:02:00.000-08:002012-02-09T13:03:33.889-08:00Order of enums in hashset and hashmapRecently I have found myself writing a complex sql-builder for two weeks already. Of cause I used <a href="http://en.wikipedia.org/wiki/Test-driven_development">TDD</a>&nbsp;technique and this was much less like a pain comparing to the times when I didn't wrote any tests. In some moment I decided that just operating with strings in some thousand or more lines of code is not very cool and started to write some <a href="http://en.wikipedia.org/wiki/Object-oriented_design">OOD</a> for the problem. In the resulting type hierarchy there were some enums. And I used these enums as keys in HashMap. Maybe some curious reader have already guessed what this post is about ;-) If not or if you just want some detailed illustration, read the rest:<br /><br /><a name='more'></a>What is the order of elements in the HashSet, or the order of entries in the HashMap? It is actually undefined. But should it be consistent across launches of the one and the same JVM on one and the same machine with no changes to the code? I think it is rather safe to assume this for testing purposes. So, OK, what impacts the order of elements in the HashSet? It is the hash code of these elements.<br /><br />And back to enums. What it the hash code of the enum instance? It has the default hashcode() implementation inherited from Object, so it is different every time the JVM is restarted. So this code:<br /><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.util.Arrays;<br />import java.util.HashSet;<br /><br />public class EnumHashTest {<br /> <br /> public static void main(String... args){<br /> System.out.println(new HashSet&lt;FooBar&gt;(Arrays.asList(FooBar.values())));<br /> }<br /> <br /> private enum FooBar {<br /> FIRST, SECOND, THIRD;<br /> }<br />}<br /></code></pre><br />will produce results similar to these on the&nbsp;subsequent&nbsp;invocations (with JVM restarting):<br />[FIRST, THIRD, SECOND]<br />[SECOND, THIRD, FIRST]<br /><br />Interestingly enough, while I'm currently trying - the order is very consistent, but different across usual running and debugging. This may, or may not, be affected by the fact that I'm using JRockit and Idea for this.<br />My first attempt to fix this problem was not successful. I used LinkedHashMap. The order of elements in this map is persistent and consistent across JVM. So, why not?<br />I said that I was writing not just some sql-builder, but "complex" sql-builder. And there were many methods in this code. And several methods to produce the resulting map with enums as keys. And some of this methods return map themselves with their content being added to the result map with the putall(). Here were come. In the&nbsp;intermediate&nbsp;map the order elements is not persistent across JVM launches and thus it is not persistent in the result map that is dependent on the order of insert of the elements in the map.<br />The following code:<br /><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.util.*;<br /><br />public class EnumHashTest {<br /> <br /> public static void main(String... args){<br /> Set&lt;FooBar&gt; orderedSet = new LinkedHashSet&lt;FooBar&gt;();<br /> orderedSet.addAll(new HashSet&lt;FooBar&gt;(Arrays.asList(FooBar.values())));<br /> System.out.println(orderedSet);<br /> }<br /> <br /> private enum FooBar {<br /> FIRST, SECOND, THIRD;<br /> }<br />}<br /></code></pre><br />produces the following output being run several times:<br />[FIRST, SECOND, THIRD]<br />[THIRD, SECOND, FIRST]<br /><br />That's it.<br />So what is the final solution? Enum's values are always limited in size. And enums always implement Comparable interface consistenly. So I just used TreeMap in the very end of this computation and got consistent ordering of elements across JVM launches.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-54760340699184545852012-01-19T11:00:00.000-08:002013-08-09T01:37:45.631-07:00Synchronized vs ThreadLocal SimpleDateFormat<b>EDIT:</b>&nbsp;This benchmark is not really reliable, as any other benchmark that is done just for fun - but here is <a href="http://moradanen.sopovs.com/2013/06/synchronized-vs-threadlocal.html">the new one</a> that pretends to have more sense.<br />It is widely known that SimpleDateFormat on the Java platform is not thread-safe. There are basically three approaches to deal with this unsafety. First - each time create new instance. Should be not very good, since it is not very simple and lightweight object. Synchronizing - I think should be better. And the third approach, that is used mostly in real world, I think - using ThreadLocal. I decided to test all these approaches. The result of my test is on the graph:<br /><div class="separator" style="clear: both; text-align: center;"><img border="0" src="http://3.bp.blogspot.com/-ILBWg1JxGO0/TrAsnYCrcoI/AAAAAAAAAVY/f24HU9Ni4ys/s1600/ThreadLocal-Synchronized-new-chart2.png" /></div>On the vertical axis here is the time, so the less - the better. On the horizontal axis is number of threads concurrently formatting dates. So what can be concluded from this graph?<br /><br /><ul><li>Either SimpleDateFormat is not so heavyweight object, either synchronization is so heavy operation (I incline to the second option), but there is very little difference in synchronizing on one and the same instance of creating new one each time it is needed. However synchronizing is a bit better.</li><li>Widely used approach among the&nbsp;naive&nbsp;is the best.</li></ul><div><b>EDIT:&nbsp;</b>Doing this test I used Java6 from Sun. I have retested it with Java7 from Oracle and get quite different picture. The hardware is different too, so these pictures can not be compared with each other in absolute numbers. But difference in the relations between "new instance each invocation" approach and one synchronized instance is very interesting:</div><div><div class="separator" style="clear: both; text-align: center;"><img src="https://docs.google.com/spreadsheet/oimg?key=0ApaN64TysovxdEVDUmJ3dlJSWGFjdk95VjU1dXRSWUE&amp;oid=2&amp;zx=d663x6h536ns" /></div><br /><div>Anyway, here is the code I used for testing:</div><br /><a name='more'></a><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.util.Date;<br />import java.util.Random;<br />import java.util.concurrent.ExecutorService;<br />import java.util.concurrent.Executors;<br />import java.util.concurrent.TimeUnit;<br /><br />import java.text.SimpleDateFormat;<br /><br />public class ThreadLocalVsSynchronizedTest {<br /> private static final int ITERATIONS = 10000;<br /> private static final int NUM_THREADS = 15;<br /> private static final int BOOTSTRAP = 30;<br /><br /> public static void main(String[] args) throws InterruptedException {<br /> //warming up<br /> test(new ThreadLocalFormatter(), 10);<br /> test(new SynchronizedFormatter(), 10);<br /> test(new NewInstanceFormatter(), 10);<br /><br /> for (int i = 1; i &lt;= NUM_THREADS; i++) {<br /> long threadLocalTime = 0, synchronizedTime = 0, newInstanceTime = 0;<br /> for (int j = 0; j &lt; BOOTSTRAP; j++) {<br /> synchronizedTime += test(new SynchronizedFormatter(), i);<br /> threadLocalTime += test(new ThreadLocalFormatter(), i);<br /> newInstanceTime += test(new NewInstanceFormatter(), i);<br /> }<br /> System.out.print(i + " "<br /> + TimeUnit.MILLISECONDS.convert(threadLocalTime, TimeUnit.NANOSECONDS));<br /> System.out.print(" "<br /> + TimeUnit.MILLISECONDS.convert(synchronizedTime, TimeUnit.NANOSECONDS));<br /> System.out.println(" "<br /> + TimeUnit.MILLISECONDS.convert(newInstanceTime, TimeUnit.NANOSECONDS));<br /> }<br /> }<br /><br /> private static long test(Formatter formatter, int numThreads) throws InterruptedException {<br /> long start = System.nanoTime();<br /> ExecutorService executor = Executors.newFixedThreadPool(numThreads);<br /> Executors.newFixedThreadPool(numThreads);<br /><br /> for (int i = 0; i &lt; numThreads; i++) {<br /> executor.execute(new FormatterWorker(formatter));<br /> }<br /> executor.shutdown();<br /> executor.awaitTermination(1, TimeUnit.DAYS);<br /><br /> return System.nanoTime() - start;<br /> }<br /><br /> private static class FormatterWorker implements Runnable {<br /> private final Formatter formatter;<br /><br /> public FormatterWorker(Formatter formatter) {<br /> this.formatter = formatter;<br /> }<br /><br /> @Override<br /> public void run() {<br /> Random r = new Random();<br /> for (int i = 0; i &lt; ITERATIONS; i++) {<br /> formatter.format(new Date(r.nextLong()));<br /> }<br /> }<br /><br /> }<br /><br /> private interface Formatter {<br /> String format(Date date);<br /> }<br /><br /> private static class SynchronizedFormatter implements Formatter {<br /> private final SimpleDateFormat format = new SimpleDateFormat();<br /><br /> @Override<br /> public synchronized String format(Date date) {<br /> return format.format(date);<br /> }<br /> }<br /><br /> private static class ThreadLocalFormatter implements Formatter {<br /> private final ThreadLocal&lt;SimpleDateFormat&gt; format = new ThreadLocal&lt;SimpleDateFormat&gt;();<br /><br /> @Override<br /> public String format(Date date) {<br /> SimpleDateFormat dateFormat = this.format.get();<br /> if (dateFormat == null) {<br /> dateFormat = new SimpleDateFormat();<br /> format.set(dateFormat);<br /> }<br /> return dateFormat.format(date);<br /> }<br /> }<br /><br /> private static class NewInstanceFormatter implements Formatter {<br /> @Override<br /> public String format(Date date) {<br /> return new SimpleDateFormat().format(date);<br /> }<br /> }<br />}<br /></code></pre></div>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com6tag:blogger.com,1999:blog-3761441878096462409.post-43209901391118634592012-01-16T10:54:00.000-08:002012-02-10T03:47:15.498-08:00Oracle 11g denormalizing hierarchy with recursive subquery factoring and LISTAGG functionIn Oracle 11g there are many new features. From the developer's point of view there are a bit less such features, but still there are some very impressive. Among these features there are <a href="http://docs.oracle.com/cd/E11882_01/server.112/e17118/statements_10002.htm#BCEJGIBG">recursive subquery factoring</a> and <a href="http://docs.oracle.com/cd/E14072_01/server.112/e10592/functions087.htm">LISTAGG</a> grouping function. Here is how they can be combined together: <br /><a name='more'></a><pre><code class="sql"><br />WITH plain_hierarchy AS ( <br /> SELECT 1 id, NULL pid, 'Level 0' value FROM dual UNION ALL<br /> SELECT 2 id, 1 pid, 'Level 1' value FROM dual UNION ALL<br /> SELECT 3 id, 2 pid, 'Level 2' value FROM dual UNION ALL<br /> SELECT 4 id, 3 pid, 'Level 3' value FROM dual UNION ALL<br /> SELECT 5 id, 4 pid, 'Level 4' value FROM dual UNION ALL<br /> SELECT 6 id, 5 pid, 'Level 5' value FROM dual UNION ALL<br /> SELECT 11 id, 1 pid, 'FooBar 1' value FROM dual UNION ALL<br /> SELECT 12 id, 11 pid, 'FooBar 2' value FROM dual UNION ALL<br /> SELECT 13 id, 12 pid, 'FooBar 3' value FROM dual UNION ALL<br /> SELECT 21 id, 1 pid, 'BarFoo 1' value FROM dual<br /> ),<br /> combined_hierarchy (initial_id, id, pid, value, reportLevel) AS (<br /> SELECT id AS initial_id, id, pid, value, 0 AS reportLevel<br /> FROM plain_hierarchy<br /> UNION ALL<br /> SELECT comb.initial_id, plain.id, plain.pid, plain.value, comb.reportLevel + 1<br /> FROM combined_hierarchy comb<br /> JOIN plain_hierarchy plain<br /> ON comb.pid = plain.id<br /> )<br />SELECT initial_id,<br /> LISTAGG(value,' -&gt; ') within GROUP(ORDER BY reportlevel DESC) combined_value<br />FROM combined_hierarchy<br />GROUP BY initial_id<br /></code></pre>Here the first subquery in the "with" is just test data. The second is the recursive one and where all the interesting things happen. As far as I undestand sql, this was not possible in Oracle 10g version of sql (<b>see UPDATE</b>), even if we ignore absence of LISTAGG function. However if we need not all full paths in the hierarchy, but only for some exact node in it, there is more productive solutuion that simple adding "where initial_id=" as the but-last string. (Same testing data) <br /><pre><code class="sql"><br />SELECT initial_id, LISTAGG(value,' -&gt; ') within GROUP(ORDER BY inner_level) combined_value<br />FROM<br /> (SELECT 6 AS initial_id, level inner_level, combined_hierarchy.*<br /> FROM<br /> ( SELECT id, pid, value FROM plain_hierarchy<br /> ) combined_hierarchy<br /> START WITH id =6<br /> CONNECT BY prior pid=id<br /> )<br />GROUP BY initial_id<br /></code></pre>The cost of the first query for only one denormalized path is 92, while the cost of the second is 78. On the real data and real tables the difference is much bigger.<br /><br /><b>UPDATE:</b><br />Ignoring the absence of LISTAGG grouping function in the Oracle 10g we may write sql that will produce the same result for all items in the hieararchy:<br /><br /><pre><code class="sql"><br />WITH plain_hierarchy AS ( <br /> SELECT 1 id, NULL pid, 'Level 0' value FROM dual UNION ALL<br /> SELECT 2 id, 1 pid, 'Level 1' value FROM dual UNION ALL<br /> SELECT 3 id, 2 pid, 'Level 2' value FROM dual UNION ALL<br /> SELECT 4 id, 3 pid, 'Level 3' value FROM dual UNION ALL<br /> SELECT 5 id, 4 pid, 'Level 4' value FROM dual UNION ALL<br /> SELECT 6 id, 5 pid, 'Level 5' value FROM dual UNION ALL<br /> SELECT 11 id, 1 pid, 'FooBar 1' value FROM dual UNION ALL<br /> SELECT 12 id, 11 pid, 'FooBar 2' value FROM dual UNION ALL<br /> SELECT 13 id, 12 pid, 'FooBar 3' value FROM dual UNION ALL<br /> SELECT 21 id, 1 pid, 'BarFoo 1' value FROM dual<br /> ),<br />combined_hierarchy AS (<br /> select CONNECT_BY_ROOT id initial_id, level inner_level, plain_hierarchy.* <br /> from plain_hierarchy<br /> connect by prior pid=id<br /> )<br />SELECT initial_id,<br /> LISTAGG(value,' -&gt; ') within GROUP(ORDER BY inner_level DESC) combined_value<br />FROM combined_hierarchy<br />GROUP BY initial_id;<br /></code></pre><br />The thing that does the right job is CONNECT_BY_ROOT function (or expression?) that just picks the value from the root of recursive query written with "connect by prior". It seems that this query in Oracle 11g (where it only can work due to LISTAGG) is much faster than the variant with recursive subquery factoring. <br /><b>Update 2:</b><br />Same (or almost same) results with different techniques. First is compatible with Oracle 10g. <br /><pre><code class="sql"><br />--Without both recursive subquery factoring and LISTAGG (one of variants - not very elegant)<br />WITH plain_hierarchy AS ( <br /> SELECT 1 id, NULL pid, 'Level 0' value FROM dual UNION ALL<br /> SELECT 2 id, 1 pid, 'Level 1' value FROM dual UNION ALL<br /> SELECT 3 id, 2 pid, 'Level 2' value FROM dual UNION ALL<br /> SELECT 4 id, 3 pid, 'Level 3' value FROM dual UNION ALL<br /> SELECT 5 id, 4 pid, 'Level 4' value FROM dual UNION ALL<br /> SELECT 6 id, 5 pid, 'Level 5' value FROM dual UNION ALL<br /> SELECT 11 id, 1 pid, 'FooBar 1' value FROM dual UNION ALL<br /> SELECT 12 id, 11 pid, 'FooBar 2' value FROM dual UNION ALL<br /> SELECT 13 id, 12 pid, 'FooBar 3' value FROM dual UNION ALL<br /> SELECT 21 id, 1 pid, 'BarFoo 1' value FROM dual<br /> ),<br />combined_hierarchy AS (<br /> select CONNECT_BY_ROOT id initial_id,SYS_CONNECT_BY_PATH(reverse(value),' &gt;- ') as combined_value2, level inner_level, plain_hierarchy.* <br /> from plain_hierarchy<br /> connect by prior pid=id<br /> )<br />SELECT ch1.initial_id,reverse(ch1.combined_value2),ch1.inner_level<br />FROM combined_hierarchy ch1<br />join (<br />select initial_id, max(inner_level) as inner_level<br />from combined_hierarchy<br />group by initial_id<br />) ch2<br />on ch1.initial_id=ch2.initial_id<br />and ch1.inner_level=ch2.inner_level;<br /><br />--Without LISTAGG function<br />WITH plain_hierarchy AS ( <br /> SELECT 1 id, NULL pid, 'Level 0' value FROM dual UNION ALL<br /> SELECT 2 id, 1 pid, 'Level 1' value FROM dual UNION ALL<br /> SELECT 3 id, 2 pid, 'Level 2' value FROM dual UNION ALL<br /> SELECT 4 id, 3 pid, 'Level 3' value FROM dual UNION ALL<br /> SELECT 5 id, 4 pid, 'Level 4' value FROM dual UNION ALL<br /> SELECT 6 id, 5 pid, 'Level 5' value FROM dual UNION ALL<br /> SELECT 11 id, 1 pid, 'FooBar 1' value FROM dual UNION ALL<br /> SELECT 12 id, 11 pid, 'FooBar 2' value FROM dual UNION ALL<br /> SELECT 13 id, 12 pid, 'FooBar 3' value FROM dual UNION ALL<br /> SELECT 21 id, 1 pid, 'BarFoo 1' value FROM dual<br /> ),<br /> combined_hierarchy (initial_id, id, pid, value, reportLevel,combined_value) AS (<br /> SELECT id AS initial_id, id, pid, value, 0 AS reportLevel,CAST(value AS VARCHAR2(2000)) as combined_value<br /> FROM plain_hierarchy<br /> UNION ALL<br /> SELECT comb.initial_id, plain.id, plain.pid, plain.value, comb.reportLevel + 1,<br /> CAST(plain.value|| ' -&gt; '||comb.combined_value AS VARCHAR2(2000))<br /> FROM combined_hierarchy comb<br /> JOIN plain_hierarchy plain<br /> ON comb.pid = plain.id<br /> )<br />SELECT ch0.initial_id, ch0.combined_value<br />FROM combined_hierarchy ch0<br />join <br />(select initial_id, max(reportLevel) as reportLevel<br />from combined_hierarchy GROUP BY initial_id<br />) ch1<br />on ch0.initial_id=ch1.initial_id <br />and ch0.reportLevel=ch1.reportLevel;<br /></code></pre>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-56273417108951894472011-10-30T03:48:00.000-07:002011-10-30T03:48:00.756-07:00Inject in toInstance bindingWith <a href="http://code.google.com/p/google-guice/">Guice</a> it is extremely easy to add to injector self-constructed beans. But what was not very intuitive for me is that various inject annotations in these beans will have their influence on the state of the bean. Here is very simple illustration to this fact: <br /><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import javax.inject.Inject;<br /><br />import com.google.inject.Binder;<br />import com.google.inject.Guice;<br />import com.google.inject.Injector;<br />import com.google.inject.Module;<br /><br />public class ToInstanceTest {<br /> public static void main(String[] args) {<br /> Injector injector = Guice.createInjector(new Module() {<br /> @Override<br /> public void configure(Binder binder) {<br /> binder.bind(String.class).toInstance("hello");<br /> binder.bind(Foo.class).toInstance(new Foo());<br /> }<br /> });<br /> Foo foo = injector.getInstance(Foo.class);<br /> System.out.println(foo.value);<br /> }<br /><br /> private static class Foo {<br /> @Inject<br /> String value;<br /> }<br />}<br /></code></pre>The output of this java program is "hello". At first I was really surprised.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-4206282875765071992011-10-29T03:44:00.000-07:002011-10-29T11:10:32.720-07:00Synchronized vs atomicIn a very good book <a href="http://jcip.net/">Java Concurrency in Practice</a> one of the first example is about generating sequence of unique numbers. To ways of making it thread-safe are mentioned - using synchronized method and using internally atomic object. Here is another little benchmark, this time - comparing these approaches. The result can be seen from this chart: <br /><div class="separator" style="clear: both; text-align: center;"><img border="0" src="http://2.bp.blogspot.com/-ZgsNY6fgoQ0/Tqp-UEZpPXI/AAAAAAAAAVM/rah86SEGwnw/s1600/synchronized-vs-atomic-chart.png" /></div><br />And here is the code, that I used to make data for it: <br /><a name='more'></a><br /><pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.util.concurrent.ExecutorService;<br />import java.util.concurrent.Executors;<br />import java.util.concurrent.TimeUnit;<br />import java.util.concurrent.atomic.AtomicInteger;<br /><br />public class SafeSequenceTest {<br /><br /> private static final int THREADS_NUM = 40;<br /> private static final int ITERATIONS = 30;<br /> private static final int SIZE = 1000000;<br /><br /> public static void main(String[] args) throws InterruptedException {<br /> //warming up<br /> test(new SynchronyzedSequence(), THREADS_NUM);<br /> test(new AtomicSequence(), THREADS_NUM);<br /><br /> for (int i = 1; i &lt;= THREADS_NUM; i++) {<br /> long syncTime = 0, atomicTime = 0;<br /><br /> for (int j = 0; j &lt; ITERATIONS; j++) {<br /> syncTime += test(new SynchronyzedSequence(), i);<br /> atomicTime += test(new AtomicSequence(), i);<br /> }<br /><br /> System.out.print(i + " " + TimeUnit.MILLISECONDS.convert(syncTime, TimeUnit.NANOSECONDS));<br /> System.out.println(" " + TimeUnit.MILLISECONDS.convert(atomicTime, TimeUnit.NANOSECONDS));<br /> }<br /> }<br /><br /> private interface Sequence {<br /> int getValue();<br /> }<br /><br /> private static class SynchronyzedSequence implements Sequence {<br /> private int value = 0;<br /><br /> @Override<br /> public synchronized int getValue() {<br /> return value++;<br /> }<br /> }<br /><br /> private static class AtomicSequence implements Sequence {<br /> private AtomicInteger value = new AtomicInteger(0);<br /><br /> @Override<br /> public int getValue() {<br /> return value.incrementAndGet();<br /> }<br /> }<br /><br /> private static final long test(Sequence seq, int numThreads) throws InterruptedException {<br /> long start = System.nanoTime();<br /> ExecutorService executor = Executors.newFixedThreadPool(numThreads);<br /><br /> for (int i = 0; i &lt; numThreads; i++) {<br /> executor.execute(new SequenceConsumer(seq));<br /> }<br /> executor.shutdown();<br /> executor.awaitTermination(1, TimeUnit.DAYS);<br /> return System.nanoTime() - start;<br /> }<br /><br /> private static class SequenceConsumer implements Runnable {<br /> private final Sequence sequence;<br /><br /> public SequenceConsumer(Sequence sequence) {<br /> this.sequence = sequence;<br /> }<br /><br /> @Override<br /> public void run() {<br /> for (int i = 0; i &lt; SIZE; i++) {<br /> //only get and do nothing with it<br /> sequence.getValue();<br /> }<br /> }<br /> }<br />}<br /></code></pre>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-54595899117915959642011-10-13T13:06:00.000-07:002011-10-28T03:47:01.172-07:00Synchronization on instance vs static methodSimple the difference is that synchronizing on instance method is like writing <b>synchronized (this)</b> and synchronizing on static method is like writing <b>synchronized (FooBar.class)</b>. <br />If you want not so simple explanation, here it is: <br /><a name='more'></a><pre><code class="java"><br /> public static void main(String[] args) {<br /> new FooBar().instanceFoo(true);<br /> //This is extremely bad and not the thing that should be used in reality. In fact it produces<br /> //compiler warning. Use Foobar.staticFoo(true) instead.<br /> new FooBar().staticFoo(true);<br /> }<br /><br /> private static class FooBar {<br /> public static synchronized void staticFoo(boolean recursive) {<br /> Thread thread = new Thread(new StaticRunner(recursive));<br /> thread.start();<br /> try {<br /> thread.join();<br /> } catch (InterruptedException e) {<br /> throw new RuntimeException(e);<br /> }<br /> }<br /><br /> public synchronized void instanceFoo(boolean recursive) {<br /> Thread thread = new Thread(new InstanceRunner(recursive));<br /> thread.start();<br /> try {<br /> thread.join();<br /> } catch (InterruptedException e) {<br /> throw new RuntimeException(e);<br /> }<br /> }<br /> }<br /><br /> private static class StaticRunner implements Runnable {<br /> private final boolean recursive;<br /><br /> public StaticRunner(boolean recursive) {<br /> this.recursive = recursive;<br /> }<br /><br /> @Override<br /> public void run() {<br /> if (recursive) {<br /> //This is extremely bad and not the thing that should be used in reality. In fact it produces<br /> //compiler warning. Use Foobar.staticFoo(false) instead.<br /> new FooBar().staticFoo(false);<br /> System.out.println("This will never be printed");<br /> } else {<br /> System.out.println("Internal static run - this will never be printed too");<br /> }<br /> }<br /><br /> }<br /><br /> private static class InstanceRunner implements Runnable {<br /> private final boolean recursive;<br /><br /> public InstanceRunner(boolean recursive) {<br /> this.recursive = recursive;<br /> }<br /><br /> @Override<br /> public void run() {<br /> if (recursive) {<br /> new FooBar().instanceFoo(false);<br /> } else {<br /> System.out.println("Internal instance run");<br /> }<br /> }<br /><br /> }<br /></code></pre>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-1852671687617289732011-10-13T12:58:00.000-07:002011-10-27T05:44:58.292-07:00ConcurrentModificationException in the nutshell<pre><code class="java"><br />package com.sopovs.moradanen;<br /><br />import java.util.ArrayList;<br />import java.util.Arrays;<br />import java.util.List;<br /><br />public class ConcurrentModificationExceptionTest {<br /><br /> public static void main(String[] args) {<br /> List&lt;String&gt; list = new ArrayList(Arrays.asList("first"));<br /> for (String el : list) {<br /> list.add("second");<br /> }<br /> }<br />}<br /></code></pre>Results in <br /><br /><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Exception in thread "main" java.util.ConcurrentModificationException</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>at java.util.ArrayList$Itr.next(ArrayList.java:791)</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>at com.sopovs.moradanen.ConcurrentModificationExceptionTest.main(ConcurrentModificationExceptionTest.java:11)</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><b>Update:</b> This may look strange, though... <br /><pre><code class="java"><br />ArrayList&lt;String&gt; list = new ArrayList&lt;String&gt;(Arrays.asList("foo", "bar"));<br />for (String el : list) {<br /> //Capacity will not be increased, but iterator will fail...<br /> list.ensureCapacity(1);<br />}<br /></code></pre>Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0tag:blogger.com,1999:blog-3761441878096462409.post-29240123286321067392011-10-08T14:09:00.000-07:002011-10-08T14:11:19.817-07:00ForkJoin factorial calculationSeveral months ago I wrote a <a href="http://moradanen.sopovs.com/2011/07/is-very-good-blog-post-about-how-to.html">multithreaded factorial</a>&nbsp;method. &nbsp;It was very simple from the point of view of the underlying technology, but not so trivial from the point of view of synchronizing threads. It used simple start() and join() methods that are available since the Java 1.0 And than I thought that with all the power of java I can improve it. So i used the ThreadPoolExecutor - a piece of technology from the Java 5 and really improved - here is my post about <a href="http://moradanen.sopovs.com/2011/07/multithreaded-factorial-with.html">multithreaded factorial using TreadPoolExecutor</a>.<br /><br />But Java 5 is a bit old now. And this year the Java 7 has been released! So here is the new version - using the new ForkJoin Framework available in it. Actually when I started writing this simple piece of code is didn'nt think that the result can be like that. I thought that all the power of my 4-core processor was already&nbsp;utilized&nbsp;by the variant with the ThreadPoolExecutor and considered this new method only as an exercise on the new API. I previewed that new version may be&nbsp;simpler&nbsp;as this is one of the stated goals of ForkJoin Framework and it is. But the actual performance increase was absolutely&nbsp;unforeseen&nbsp;by me.<br /><br />So no more jabber, here are the results:<br /><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">21 seconds for ForkJoin</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">27 seconds for ThreadPoolExecutor</span><br /><br />Here is the code:<br /><br /><pre><code class="java"><br /> private static BigInteger factFjPool(int input, int numThreads)<br /> throws InterruptedException, ExecutionException {<br /> ForkJoinPool forkJoinPool = new ForkJoinPool(numThreads);<br /> ForkJoinTask&lt;BigInteger&gt; future = forkJoinPool.submit(new FactorialRecursiveTask(1, input + 1));<br /> return future.get();<br /> }<br /><br /> private static class FactorialRecursiveTask extends RecursiveTask&lt;BigInteger&gt; {<br /> private static final long serialVersionUID = 1L;<br /><br /> private static final int THRESHOLD = 1000;<br /><br /> private final int lo, hi;<br /><br /> public FactorialRecursiveTask(int lo, int hi) {<br /> this.lo = lo;<br /> this.hi = hi;<br /> }<br /><br /> @Override<br /> protected BigInteger compute() {<br /> if (hi - lo &lt; THRESHOLD) {<br /> BigInteger result = BigInteger.valueOf(lo);<br /> for (int i = lo + 1; i &lt; hi; i++) {<br /> result = result.multiply(BigInteger.valueOf(i));<br /> }<br /> return result;<br /> } else {<br /> int mid = (lo + hi) &gt;&gt;&gt; 1;<br /><br /> FactorialRecursiveTask f1 = new FactorialRecursiveTask(lo, mid);<br /> f1.fork();<br /> FactorialRecursiveTask f2 = new FactorialRecursiveTask(mid, hi);<br /> return f2.compute().multiply(f1.join());<br /> }<br /> }<br /> }<br /></code></pre><br />I've tried to improve the previous version using other amount of threads, since this version obviously uses division into mush smaller sub-tasks, but to no result. Maybe I used wrong BlockingQueue&lt;Runnable&gt;, but this may be regarded as the simplicity of using this API. And definitely ForkJoinFramework is superior here, since not only decision about which Queue to use is not needed but also the division into sub-tasks is much simpler.<br /><br />As usual the full code for this example may be found on <a href="https://gist.github.com/1272876">github</a>.Ivan Sopovhttp://www.blogger.com/profile/06362174507359038965noreply@blogger.com0