Friday, May 30, 2014

New data show that California will be starting the summer dry season with a snowpack around the lowest levels since recordkeeping began nearly a century ago. The data were collected by hand over the last week as part of California’s annual snowpack survey across the vast Sierra Nevada, an update to the automated numbers released a week ago.

Of course, record-keeping, in some sense, began quite a bit more than "nearly a century ago".

All around the house it looks desolate. Where there were green pastures when we camped here two years ago, now all is dry, dusty, bare ground. Three hundred cattle have died by the miserable water hole back of the house, where we get water to drink, and their stench pollutes the air.

Here is a notice released by the Seattle National Weather Service office:

.CLIMATE...THE RAINFALL TOTAL AT SEATTLE-TACOMA AIRPORT WAS 0.22
INCHES SUNDAY. THIS MAKES THE RAINFALL TOTAL SINCE FEBRUARY 1ST
22.87 INCHES. THIS BREAKS THE RECORD FOR THE WETTEST FEBRUARY
THROUGH JULY IN SEATTLE. THE OLD RECORD WAS 22.81 INCHES SET IN
1972. FELTON/MCDONNAL

You knew this was a wet late winter/spring, particularly mid-February through mid-March. But to beat the Feb-July record in MAY is really notable.

Playback is accomplished by binary searching a 2-level index of pointers into the WARC data. The second level of this index is a 20TB compressed sorted list of (url, date, pointer) tuples called CDX records[2]. The first level fits in core, and is a 13GB sorted list of every 3000th entry in the CDX index, with a pointer to larger CDX block.

Index lookup works by binary searching the first level list stored in core, then HTTP range-request loading the appropriate second-level blocks from the CDX index. Finally, web page data is loaded by range-requesting WARC data pointed to by the CDX records. Before final output, link re-writing and other transforms are applied to make playback work correctly in the browser.

For an island of only 24 square miles, Manhattan sure has a lot of neighborhoods. Many have distinct monikers that might not seem intuitive to the lay-tourist, or even to a lifelong New Yorker. Here's where the names of New York's most famous 'hoods came from.

Chambers goes even further than Cisco's counsel, decrying the NSA's tactics and the damage they're doing to his company's reputation.

“We simply cannot operate this way; our customers trust us to be able to deliver to their doorsteps products that meet the highest standards of integrity and security,” Chambers wrote. “We understand the real and significant threats that exist in this world, but we must also respect the industry’s relationship of trust with our customers.”

A group of Riverbed TAC engineers have worked on an internal troubleshooting document to kick start new TAC engineers. It describes the design of the Steelhead appliance, the working of the optimization service and the setup of optimized TCP sessions, installation and operation related issues, various latency optimization related issues, on how to use the various CLI tools to troubleshoot and how you can deal with the contents of the system dump.

When Alchin offered him the job, Russinovich didn’t take it. But after several more years spent running his Sysinternals site–where he published a steady stream of exposés that, in his words, “pissed off” Microsoft and other tech outfits–he did join the software giant. The company made him a Microsoft Technical Fellow–one of the highest honors it can bestow–and today, he’s one of the principal architects of Microsoft Azure, the cloud computing service that’s leading the company’s push into the modern world.

Although I think everyone expected this, after Donovan's unusual "recess" from soccer a year ago, it was still a shock. Here's what Sporting Newshas to say:

The U.S. has minted only one Donovan. He is the greatest field player the United States has produced; a starter at three World Cups; a player with nearly every important offensive record in his nation’s history: goals, assists, World Cup goals.

Frankly, it's hard to see our roster terrifying anyone, but Klinsmann has been a miracle worker so far, so I'm just going to hold my breath and wait to see them play.

Meanwhile, of the players that did get picked, many are very young and are unfamiliar to me. Of the players I do know, my favorites are Bradley and Dempsey, but I like watching all of them.

It's hard to believe, but the opening match is 3 weeks from today! Here it comes!

It was on sale, and I had been wanting it for a while (it's about a year old I think), so I jumped when I saw the sale.

It's very simple: this is totally my sort of game, and TOTALLY a fun game.

Shadowrun Returns is sort of an RPG: you have a character, and your character goes on quests, and your character develops, and you can make choices about what to do next.

Shadowrun Returns is sort of a tactical turn-based combat game: when you are in a fight, you and your enemy alterate turns, and on each turn you make a tactical decision about where to move, what cover to seek, what weapon to fire, which enemy to fire upon, etc. In that way, Shadowrun Returns kind of reminds me of XCOM.

And Shadowrun Returns is sort of an interactive novel: it's kind of a mystery story, in which the overall plot device is that of a "who-done-it", and as you go traveling around you have various conversations with characters and sub-plots and twists and turns.

That is, it's sort of a mystery story if Raymond Chandler wrote detective stories involving cyborgs, trolls, orcs, and shamanistic mages, and set it in a post-apocalyptic version of Seattle.

Yeah, Godwin's law, whatever, whatever. My point is NOT that the NSA is the same as the Nazi party (in fact, my argument has the NSA on the opposite side). My point is that the government now treats ordinary civilians as worthy of the same sort of tactics that they once used against the Nazis.

It is most likely that Vernam came up with the need for non-repetition; Mauborgne, though, apparently contributed materially to the invention of the two-tape variant. Furthermore, there is reason to suspect that he suggested the need for randomness to Vernam. However, neither Mauborgne, Herbert Yardley, nor anyone at AT&T really understood the security advantages of the true one-time tape. Col. Parker Hitt may have; William Friedman definitely did.

All of the weird trades occurred at about the same time, just over 10 minutes before the close, and that explanation makes sense. There you are at 3:49 p.m., entering your market-on-close order to buy and sell a bunch of shares in the closing auction. You enter the number of shares you want, you go to the drop-down box on your computer system, you pick "Market on Close," your mouse slips, you actually click on "Market," and whoosh, off your order goes to be filled in the market.

A time-in-force limitation that can be placed on the execution of an order. This limitation requires that the order is executed as close as possible to the closing price for a security. All or any part of the order that cannot be executed at the closing price is canceled.

For those not in the know, feature flags are a way of adding a conditional to your code that lets a configurable number of users or requests through, originally designed for restricting new features to internal users before rolling them out to the rest of the userbase.

Each like task is grouped into a stage having a queue and thread pool (ScheduledThreadPoolExecutor more specifically for the Java folks). Some stages skip the messaging service and queue tasks immediately on a different stage if it exists on the same node. Each of these queues can be backed up if execution at a stage is being over run.

LMDB is an ultra-fast, ultra-compact key-value embedded data store developed by Symas for the OpenLDAP Project. It uses memory-mapped files, so it has the read performance of a pure in-memory database while still offering the persistence of standard disk-based databases, and is only limited to the size of the virtual address space, (it is not limited to the size of physical RAM).

Friday, May 16, 2014

So, what could be better than a story by Maggie Koerth-Baker about opals: The Opal's Fire.

Scientists know what makes the difference between precious opal, common opal, and regular silica. But why those differences exist is kind of a mystery. A new theory has an answer for that, while simultaneously explaining why most of the precious opals in the world come from a single place. The solution depends on one surprising fact. Turns out, Central Australia has a lot in common with the surface of Mars.

One thing is clear (besides the fact that opals make pretty jewelry): the next time I visit Mars, I'm bringing my old rock pick.

Wednesday, May 14, 2014

Beats headphones weren't tuned evenly, like the usual high-end headphones. They were tuned to make the music sound more dramatic. Iovine adds: "We wanted to recreate that excitement of being in the studio. That's why people listen." But skeptics also wondered why anybody would pay $200 for headphones when you got the earbuds for nothing. "I was like, 'Bad audio is free,' " Iovine says. "When you believe in something, the last thing you say to yourself is, Well, no one's doing this, so there must be no good reason to do it."

Farmers here are turning to groundwater to make up the difference—and that’s where things get worse. The shocking truth is, California is the last state that doesn’t regulate groundwater pumping, even as supplies are dwindling. That means the motto around here right now is, to borrow another Mother Jones headline: “Drill baby drill (for water, that is).”

The CJEU’s embrace of a double standard with respect to the information (i.e., it was permissible for the newspaper to publish it in 1998 but not for Google to link to it in 2014) raises a difficult question about the law’s role in empowering people to sculpt their public information profiles to conform with an image of themselves that they want the world to see. There is no question in Mr. Costeja’s case that the information at issue was true or that it was appropriately publicized by the newspaper. Does damaging public information become private simply by virtue of the passage of time?

Anybody introducing a new API will have to answer the ‘copyright’ question: “Do you claim copyright on your API?” In practice a very very small percentage of APIs ever get copied/cloned, because most fail or the competition comes up with what they think is a better API.

When growth slows and the market is divided between major players a collusionary pricing phase will take over. Cloud customers are sticky customers. It's hard to move off a cloud. The need for higher margins to justify the cash flow drain during the customer acquisition phase will reverse the favorable trends we are seeing now.

Skyrim’s towns have very few people. Wowwiki says World of Warcraft’s Stormwind City has 200,000 residents. Walk around and you’ll see fewer than 100 buildings and people. Both games also shrink wilderness areas relative to cities. Stormwind City is as large as Elwynn Forest. Towns are much smaller than realistic towns would be; wilderness areas are extremely small compared to realistic counterparts.

Over time, the load keeps going up and the system moves further from the initial test conditions when it was designed, developed, and tested. This happens to many highly complex systems and some end up operating at an order of magnitude higher load or different load mix than originally anticipated – this is a common cause for complex system failure. We know that systems eventually go non-linear as the load increases so we need to constantly probe 10x beyond what possible today to ensure that there remains adequate headroom between possible operating modes and the system failure point.

You’d have to be crazy not to pay 15 cents for that! Programmers are well paid and the amount of time this saves is so enormous it should be one of the easiest decisions ever.

There have been times in my career where I’d have gladly paid thousands of dollars to trap a crash in the debugger. Now that we live in The Year of the Cloud that knowledge can be stored and accessed for a mere 15 cents. This, my friends, is the future.

Breakpad is a library and tool suite that allows you to distribute an application to users with compiler-provided debugging information removed, record crashes in compact "minidump" files, send them back to your server, and produce C and C++ stack traces from these minidumps. Breakpad can also write minidumps on request for programs that have not crashed.

Breakpad is currently used by Google Chrome, Firefox, Google Picasa, Camino, Google Earth, and other projects.

I'm less than a rank amateur at Go: I know the rules, and can read and enjoy beginner's books on simple Tesuji, and can somewhat follow a game as it is played, but really I only know enough to know how little I know.

But no matter how much you know about Go, I think you'll really enjoy this article.

The trouble is that identifying Go moves that deserve attention is often a mysterious process. “You’ll be looking at the board and just know,” Redmond told me, as we stood in front of the projector screen watching Crazy Stone take back Nomitan’s initial lead. “It’s something subconscious, that you train through years and years of playing. I’ll see a move and be sure it’s the right one, but won’t be able to tell you exactly how I know. I just see it.”

Similarly inscrutable is the process of evaluating a particular board configuration. In chess, there are some obvious rules. If, ten moves down the line, one side is missing a knight and the other isn’t, generally it’s clear who’s ahead. Not so in Go, where there’s no easy way to prove why Black’s moyo is large but vulnerable, and White has bad aji. Such things may be obvious to an expert player, but without a good way to quantify them, they will be invisible to computers. And if there’s no good way to evaluate intermediate game positions, an alpha-beta algorithm that engages in global board searches has no way of deciding which move leads to the best outcome.

On my recent trip to South Korea, we found ourselves waiting a little while in Incheon Airport for our plane home. While we waited for our group of rows to board, my attention drifted over to one end of the waiting room, where a group of people were gathered around a big screen TV watching a professional sporting event.

I wandered a little bit closer, and it turned out that they were watching a professional match between two female Go players, one Korean, one Chinese, who were two of the top players in the world.

I stood, rapt, with dozens of other spectators, engrossed in the beauty of the game, and the complexity of trying to understand what moves they were playing, and why.

Saturday, May 10, 2014

Honestly, I'm often between projects. I rip through projects like a circular saw through pine. But I'm not just between projects, I'm between PROJECTS: with my team, we've arrived at the culmination of a five year effort to build and deliver a coordinated set of functionality that completely raised the bar for the level of usage that our product can support.

When I arrived at this company, we would talk about a 2,000 user installation as a large installation, and those installations were continually struggling; now we routinely talk about the problems of (simulated) 50,000 user installations, because our existing large installations, though they are five times as large as they were when I arrived, are just humming along, servicing massive user bases easily. In my field, there are currently no 50,000 user installations; an immense installation is 10,000 users, so we can support 5-fold expansion of our current installations, even if hardware doesn't improve in the meantime to save our bacon.

So, not only am I between projects, but the whole company is between projects. We're thinking about entirely different directions to take the product, which is interesting, but leaves me without a lot of concrete work to do in the meantime.

I want to talk a little bit about how you know that a project is done.

In my view, there are a number of indicators that you can use:

You've gone through all your own checklists, the ones you keep on your whiteboard, and scribbled on pieces of paper and notebooks, and you keep finding that you've checked everything off.

You've trawled your internal bug database over and over and over. While there are still items left, they are minor, and you look at the open bug reports again and again and notice that they were reported by you, and not ever observed by any customers

Then, even after looking at those bugs for the tenth time, the risk of fixing them still seems higher than the reward of having it gone.

You're continuing to write new tests and extend your existing tests, but the new tests that you're writing aren't finding any new bugs. Someone asks you a question and you think, "Interesting; I ought to write some tests in that area," but when you go to look, you've already got a bunch of tests in that area.

People stop asking you questions about the project; importantly, they stop asking you "when will it do this?" because it already does that.

People say, "software is never done." This is true, but there is definitely a time when software is DONE ENOUGH; at which point, it's time to find a new project (but, I think, not a new company; just a new project).

Meanwhile, in my personal life, I'm kind of "in between" right now as well. My kids are thoroughly grown up. As recently as 6 months ago, they still came to ask me for help, or had problems I could help solve. But now they just take care of stuff on their own. In fact, they're starting to ask me if they can help me!

Another way that I've noticed that I'm "in between" is that I've reached that age when the medical industry is starting to view me as a growth opportunity. I'm still in fine health, but the doctors, and my co-workers, are warning me that I'm about to start devoting an excessive amount of time to allowing various specialists to poke and prod me (and charge my insurance company obscene sums). In fact, these "old man" pokings and proddings begin this month, sigh.

When you're "in between," you have to choose which direction to face. You can look backwards, because now there is a lot of backwards to look at, and a lot of things to think about in that direction. In fact, it's easy to look backwards, because you know what there is to see and you can choose things that suit your mood.

In software, this takes the form of sticking with an existing project for too long, over-engineering it, adding unnecessary extra frills and decorations, fixing extremely minor problems that just churn the software without improving it. It's, unfortunately, a very easy trap to fall into: you know A LOT about the existing software: you know where it has warts, little rough spots that you could smooth over, imperfections that could be removed.

But there's much more ahead. The brand new projects not yet embarked upon have much more potential, at long last, than the "finish coat" on the work left behind. I have ideas, big ideas, for projects to do, and places to go, and accomplishments to achieve.

The Belfast Project, as it came to be known, was unique in focus and design. But it is one of a growing number of oral histories undertaken at colleges across the United States. The field has expanded and professionalized in recent decades and now claims its own association, with about 900 members, along with several degree-granting programs. Its popularity is driven by the interest among contemporary historians in the lives of ordinary people and also by digital advances. Simply put, it has become much easier to conduct oral histories and to disseminate them.

It is the ultimate of Zero-World Problems: what do you do when your billion-dollar company is hemorrhaging cash, yet can’t be sold, can’t go public, and can’t raise funding? For founders facing the Series A crunch, such questions may seem irrelevant, perhaps even a tad obnoxious. Yet, in the tale of Square and Box lies a grave message for all of us about the ability of certain founders to defy gravity, even in the face of overwhelming evidence of the difficulty of building startups targeting commodity technology markets.

So far, the only feature that Google and its Android device makers have not been able to work around without losing the benefit of the invention is rubberbanding -- after more than 50 months of Apple litigation against Android, this fact shows the limits of Apple's intellectual property.

Jaywalking pedestrians. Cars lurching out of hidden driveways. Double-parked delivery trucks blocking your lane and your view. At a busy time of day, a typical city street can leave even experienced drivers sweaty-palmed and irritable. We all dream of a world in which city centers are freed of congestion from cars circling for parking (PDF) and have fewer intersections made dangerous by distracted drivers. That’s why over the last year we’ve shifted the focus of the Google self-driving car project onto mastering city street driving.

When a developer thinks, "I want to find the source for package XYZ," why do they go to the GitHub search bar instead of Google? Do people really so believe that GitHub is the only place for code that it has supplanted Google as the way to find things?

But finally, I think, it fits in with an issue that keeps coming up for me: reviewers are too arrogant. If they don't see why the problem is important, then the issue must be with the research or the writing; it couldn't be with their reading or understanding of the problem space.

While you're watching the presentation, listen to the justifications for the design. They're all about testing! It's about having faster tests, without touching the database, and it's about being able to test controller logic without dependent context.

To achieve this, the simple controller is forbidden from talking directly to Active Record, now it has to go through the Repository. And the action itself is hollowed out to extract a Command object, which then has to call back into the controller through the Listener pattern.

There are some really cool properties that emerge from the blockchain system that I think are quite elegant. But there also exists manifestations of decentralized networks and technologies (Tor, Darknets, etc.) that are also interesting. Looking at the blockchain from a realist’s standpoint, it is not obvious that there is a need for a worse-performing database, that an unregulated oligarchy has disproportionate power over, that isn’t improved with administrator arbitration. It looks like a technology looking for a problem to solve, rather than a technology created to solve a problem.

The problem with calling out over-testing is that it’s hard to boil down to a catchy phrase. There’s nothing succinct like test-first, red-green, or other sexy terms that helped propel test-driven development to its rightful place on the center stage. Testing just what’s useful takes nuance, experience, and dozens of fine-grained heuristics.

I am not claiming Apple is not long for this world. They make great kit. But at the same time, like all other companies, sometimes they mess up. And do so in profound ways. Ways that completely open the door to competition, and effectively exclude them from playing there. I hope it was unintentional. They need to rethink the no-PCIe card design.

The speed with which a CIRT and constituent take containment actions is the subject of hot debate in the security world. Some argue for fast containment in order to limit risk; others argue for slower containment, providing more time to learn about an adversary. The best answer is to contain incidents as quickly as possible, as long as the CIRT can scope the incident to the best of its capability.

Project Secor was born from the need to persist messages logged to Kafka to S3 for long-term storage. Data lost or corrupted at this stage isn’t recoverable so the greatest design objective for Secor is data integrity.

Mechanisms built into Secor assure that as long as Kafka won’t drop messages before Secor can extract them (such as an aggressive retention policy), every single message will persist on S3.

This opinionated, introductory guide is intended for the Java programmer (all 9 million of them) who wants to learn how to write modern, lean Java, or for the Python/Ruby/Javascript programmer who’s heard (or may have experienced) bad things about Java and is curious to see how things have changed and how they can get Java’s awesome performance, flexibility and monitoring without sacrificing too much coolness.

When I first installed the game, it had all of its video options set to "maximum", and it played kind of sluggishly. But after I adjusted the options to a lower level, the game plays quite responsively and it's quite fun to play.

It's totally turn-based, no real-time elements at all.

Completely a fantasy theme.

Anyway, just passing it on for the other gamers I know, to think about.