Tuesday, October 2, 2007

WideFinder and Java

Update: I realized there is a stupid bug in the concurrent version that I posted. ConcurrentHashMap.putIfAbsent returns null if there was no previous value - not the new value - so I was missing one increment and the Executor seemed to be eating the NullPointerException. I've updated the code below.

Everyone seems to getting in on Tim Bray's WideFinder challenge; there is even a C++ implementation.I think it was originally supposed to be a competition between all those new and exciting dynamic languages, but they have not be overly impressive thus far. So I've decided to post up my various attempts at a solution in Java. Here is the basic WideFinder in Java (all import statements are omitted for brevity):

The basic version is very similar to the Ruby one, if a little more verbose. Like all of the others, it is primarily IO bound. One important thing I found, was that specifying the encoding as being ASCII made about 30% improvement in the time taken. This is because Java uses unicode strings internally, and if you read text without specifying an encoding it will use the platform default. ASCII is a good bit quicker to convert than ISO-8859-1.

For a file with 1 million entries I got the following times:

real 0m3.208suser 0m3.460ssys 0m0.215s

Taking the regexp out:

real 0m2.267suser 0m2.329ssys 0m0.210s

So we can potentially save up to 1 second, almost 1/3 of the time. My first solution was to try to keep things as close to the original as possible. For each line, instead of matching it directly, I submit a new task to an ExecutorService. To make the map safe to access from multiple threads I use a ConcurrentHashMap, with AtomicIntegers as values.

The performance for this version is pretty poor:real 0m5.407suser 0m11.921ssys 0m4.023s

Most of the locking is probably uncontended but there is still significant overhead. It also is probably not a great idea to use lots and lots of AtomicIntegers. With more work being done in the tasks it might be more reasonable. The next step was to get my hands dirty with low level threading. Many bloggers will have you believe this is impossible to get right, but somehow I've managed to write a fair bit of multithreaded code with manual synchronization without causing horrible errors. For this version I just used two threads: the main thread doing the reading, and another thread doing the regexp match and map update. I didn't bother trying to parallelize the sort. The main thread will pass lines on a queue to the regexp thread. Once the file is finished a special object is put on the queue to indicate that. We then wait for the regexp thread to finish and get the map back. Obviously locking overhead will be a problem so instead of putting lines on the queue one by one batches of lines are used.

Finally we get some speedup (for batches of 500 lines):real 0m2.597suser 0m4.137ssys 0m0.294s

You could generalize this for multiple threads, but it likely would not be worth the effort. Really, this whole exercise probably isn't worth the effort. In fact, I feel like we have all failed one of Steve Yegge's interview questions. A commenter on Tim Bray's blog shows that Unix command line utilities are the way to go, while we have been writing big programs, and trying to optimize file IO. Another - probably easy - way to do this would be to suck the data into your preferred DB, and run a single (simple) SQL query. But that wouldn't be fun, would it?

It is equivalent, but it probably isn't faster.AtomicIntegers aren't as cheap to create as regular Integers, so we probably don't want to create lots of them unnecessarily. You would expect most articles to have multiple entries in the log file, so you would be creating a lot of objects to throw away immediately. That probably isn't the best idea even for regular Integers. putIfAbsent probably holds onto a lock for ever so slightly longer than a get, so that might make a difference too.

Online Casino Games tyuueooruhttp://stonewalljacksoncarnival.org/ - No Download Casino Therefore, the more the online casino, the more the casino options and offers as well.[url=http://stonewalljacksoncarnival.org/]Virtual Casino[/url] Therefore, the more the online casino, the more the casino options and offers as well.Casino Money You can enjoy online casino simply by getting connected to the Internet.

Takings Our Risqu‚ Prices at www.Pharmashack.com, The Unequalled [b][url=http://www.pharmashack.com]Online Chemist's workshop [/url][/b] To [url=http://www.pharmashack.com]Buy Viagra[/url] Online ! You Can also Espy Honoured Deals When You [url=http://www.pharmashack.com/en/item/cialis.html]Buy Cialis[/url] and When You You [url=http://www.pharmashack.com/en/item/levitra.html]Buy Levitra[/url] Online. We Also Comrades a Tickety-boo Generic [url=http://www.pharmashack.com/en/item/phentermine.html]Phentermine[/url] Recompense the benefit of Your Nutriment ! We Talk over away Swap splotch [url=http://www.pharmashack.com/en/item/viagra.html]Viagra[/url] and Also [url=http://www.pharmashack.com/en/item/generic_viagra.html]Generic Viagra[/url] !

It isn't hard at all to start making money online in the underground world of [URL=http://www.www.blackhatmoneymaker.com]blackhat seo tools[/URL], You are far from alone if you have no clue about blackhat marketing. Blackhat marketing uses little-known or not-so-known ways to generate an income online.

Do you guys watch movies in theater or on internet? I use to rent DVD movies from [b]Redbox.com[/b]. Recently I discovered that we can watch all new movies on internet on day, they are released. So why should I spend money on renting movies??? So, can you guys please tell me where I can [url=http://www.watchhotmoviesfree.com]watch latest movie 30 Minutes or Less 2010[/url] for free?? I have searched [url=http://www.watchhotmoviesfree.com]Youtube.com[/url], [url=http://www.watchhotmoviesfree.com]Dailymotion.com[/url], [url=http://www.watchhotmoviesfree.com]Megavideo.com[/url] but, Could not find a good working link. If you know any working link please share it with me.

Hello .. firstly I would like to send greetings to all readers. After this, I recognize the content so interesting about this article. For me personally I liked all the information. I would like to know of cases like this more often. In my personal experience I might mention a book called Generic Viagra in this book that I mentioned have very interesting topics, and also you have much to do with the main theme of this article.

Any thoughts on this i found on my blog guys? What do you guys think is it very good information to put on a website?

[quote]Are you suffering from toothache for ones last couple of days? Well, if the answer is in the affirmative then it is advisable to fix an appointment inside your dentist in worcester. Finding a very good dentist is also quite a employment but if you stay anywhere around Worcester, Worcester dentist with excellent skills and techniques is also very easily found.

Apart from toothache there are lots of diseases that require dental care. Gingivitis is really a disease in which the gums turn into inflamed and effects in redness and bleeding. One more disease that requirements dentistry is Peroidontitis. In this disease plaque bacterium penetrates in between the gums and the teeth.

Impotence, or penile erectile dysfunction, in men is not as rare as one would think it is.In fact, one out of ten men around the world is affected by this condition. It is not exclusive to old men either. Impotence is caused not just by physical factors,but mental factors as well.Exercising on a regular basis and eating healthy food will increase your ability to sustain prolonged physical effort and we all know that sex is effort.Smoking, frequently drinking large quantities of alcohol, taking recreational drugs and lack of exercising will sap your stamina and also slow down the flow of blood.Penis exercises can be used for maintaining fitness, preventing or controlling premature ejaculation and increasing the length and girth of the penis.by penis enlargement pills you can overcome these problems and not only that you can also enlarge, extend also to strengthen your penis and premature ejaculation will not exist anymore for you.visit http://www.factspenisenlargement.com for more info.

[url=http://viagrapilss.pillsfm.com]discount generic viagra[/url] the drugs used to treat impotency, may be associated with an increased risk of optic nerve damage in men with a history of heart attack or high blood pressure, suggests a small study in the British Journal of Ophthalmology.

For men who want bigger, harder, longer-lasting erections, there's now VigRX Plus™, a fresh twist on the already popular VigRX™, but designed to further enhance men's sexual functioning with the addition of three exciting new ingredients: Damiana, Tribulus, and Bioperin. Doctor endorsed and rated #1 for results by clients of penis enlargement consumers. rated two penis pills is vimax. if you find about male enhancement this products is the best and proven to work, there products have money-back guarantee in effectiveness and result.

- Eclipse: The Twilight Saga Book 3 (unabridged) As Seattle is ravaged by a string of mysterious killings and a malicious vampire continues her quest for revenge, Bella once again finds herself surrounded by danger

- A Wizard Alone: Young Wizard Series Book 6 (unabridged) Best-selling author Diane Duane has created a book full of magical mystery, suspense, and humor

- New Moon: The Twilight Saga Book 2 (unabridged) When the Cullens, including her beloved Edward, leave Forks rather than risk revealing that they are vampires, it is almost too much for 18-year-old Bella to bear

- Sterling Point Books: George Washington: Frontier Colonel (unabridged) Sterling North, author of the award-winning children's classic Rascal, tells the tale of the Father of Our Country and how he became one of the greatest leaders of all time

- Case Of The Deadly Sell-out This episode of the Adventures of Nero Wolfe first aired on January 5, 1951

In my Finding Things chapter of Beautiful Code,dominical costa rica tours the first complete program is a little Ruby script that reads the ongoing Apache logfile and figures out which articles have been fetched the most. It’s a classic example of the culture, born in Awk, perfected in Perl, of getting useful work done by combining regular expressions and hash tables. I want to figure out how to write an equivalent program that runs fast on modern CPUs with low clock rates but many cores; this is the Wide Finder project.http://www.dominicalcostaricatours.com

The Louboutin Shoes Sale is planned using the ladies. offered the fact that pumps was born, the ladies lifestyle turn into colorful. The christian Louboutin 2011 Pumpswill be the god's masterwork. Who invited the Christian Louboutin Wedges ? Seldom people knew, but I think each and every and every lady will be grateful for him. between the countless pumps,the Christian Louboutin Shoescould be probably the most exceptional ones. The stylish pattern, the delicate design all mold the ladies perfect leg profile. Flowers inside the spring of 2011 creeping, up from frizzy hair to outfits hold on to footwear, have experienced a brilliant up. on this type of the glamor, spring and summer time flowers now here. Romance is really a woman's mood, exquisite flowers just appropriate of expression within our gestures, the woman, how can we not adore the romantic temperament to make certain which they distributed the flowers do? 2011 flowers bloom will get satisfaction from numerous poses! The Christian Louboutin New Sandalsalso can adds the hright in the ladies, it hold shock toward short lady. especially the red-colored lone in the louboutin heels, beauty and sexy, different ladies are crazy. The red-colored sole, andChristian Louboutin Mensis a very distinctive design, its design reflects its style. You put on it, that means you have its style.the earliest attribute in the Sale Louboutin stroe.

This is good site to spent time on .I just stumbled upon your informative blog and wanted to say that I have really enjoyed reading your very well written blog posts. I will be your frequent visitor, that's for sure.

hi,Java. Perusing the blogs of the Java and Scala programmers, it appears that the common complain is .. regular expressions. So the hunt was on for a good regular expression library. Joni, a port of the Ruby Oniguruma regex library to the JVM used by the JRuby project, appears ideal: low-level and supposedly very fast. Until one hits the total lack of documentation. So that’s off the table. Ended up using dk.brics.automaton, which appears to perform well enough, even when parsing Unicode strings.

[url=http://freewebs.com/carworksauto/apps/profile/101737854/]Can i order azithromycin[/url]

Zithromax (azithromycin) is known as a component all around the classification of drugs called macrolide anti-biotics. Zithromax fights bacteria contained in the childs body. Zithromax can feel dealing with diverse unique variations of difficulties a result of bacteria, like for example inhaling problems, epidermis disease, head difficulties, and venereal illness. Zithromax is also utilized for uses not promoted in script drugs guide.

Cialis is a widely recognised erectile dysfunction treatment that has become even more successful than Viagra. Cialis generic contains tadalafil, which is an extremely effective and long-lasting impotent cure. Unlike other ed dysfunction treatments, [url=http://1cialis20mg.com/]cialis tablest online[/url] starts working very quickly and has longer-lasting benefits. A single cialis dosage promotes healthy, comfortable and controllable erections for many hours. Over 8 million men worldwide have used [url=http://1cialis20mg.com/]buy cialis 20mg online[/url] . It’s been widely clinically tested, and is proven to have only minor – if any – side effects. For people suffering from erectile dysfunction, impotence is now a thing of the past with cialis.

Cialis ® comes as yellow film-coated tablets. They are in the shape of almonds and have "C 20" marked on one side. These tablets are available in blister packs containing 2, 4 or 8 tablets. Cialis ® is a treatment for men with erectile dysfunction. This is when a man cannot get, or keep a hard, erect penis suitable for sexual activity. Cialis ® belongs to a group of medicines called phosphodiesterase type 5 inhibitors. Following sexual stimulation [url=http://1cialis20mg.com/]buy cialis 20mg[/url] works by helping the blood vessels in your penis to relax, allowing the flow of blood into your penis. The result of this is improved erectile function. Cialis ® will not help you if you do not have erectile dysfunction. It is important to note that Cialis ® does not work if there is no sexual stimulation. You and your partner will need to engage in foreplay, just as you would if you were not taking a medicine for erectile dysfunction.

top [url=http://www.c-online-casino.co.uk/]uk casinos online[/url] brake the latest [url=http://www.realcazinoz.com/]realcazinoz.com[/url] free no set aside reward at the best [url=http://www.baywatchcasino.com/]baywatch casino [/url].

top [url=http://www.xgambling.org/]online casinos[/url] coincide the latest [url=http://www.realcazinoz.com/]casino[/url] autonomous no store bonus at the leading [url=http://www.baywatchcasino.com/]baywatchcasino.com [/url].

Wow, this article end is unexpected, and in reason, let me much surprised and aftertaste. Many see this article I think is a good thing, can make their own thinking have been develop.taxis in w2 | minicab to airport | airport drop w2

Well, very good content, let me feel very surprised. Surprise life would have so good things, more to my surprise I would see such a good content. Well, I will pay more attention to, and I want to more surprise.cabs in w2 | cab w2 | w2 taxis