If you want to predict what people will do in most financial situations, then figure out what is in their narrow self-interest to do, and assume they will sooner or later figure it out, too. But that doesn't work when people's behavior isn't confined to the narrow channel of self-interest.

For the first few decades after the war, it worked very well. But today the different European countries are deeply suspicious of each other. There’s an especially deep divide between the rich/creditor northern, Germanic countries and the poorer/debtor Mediterranean countries. Europe as a coherent political entity exists more in theory than in practice. And without political will, the economics simply doesn’t work.

On pensions, the EU wants Greece to stop people from retiring early, as 75% of its public-sector employees do. Europe wants Greece to raise the retirement age to 67 years old — the same as Germany's — which would be a big change from Greece's current retirement age of 61 years old.

Although unwelcome, stiffing the IMF is not considered a default by the main credit ratings agencies, which are focused on debt held by private-sector lenders. The IMF also gives late payers warnings and some time to clear their arrears before publishing an official notice of default.

Clauses in Greece’s debts to the euro zone’s rescue fund and the ECB include provisions to call in loans if the country misses a payment to other creditors, but officials suggest that they wouldn’t be quick to enact these clauses as long as there is hope of reaching some sort of deal that would keep Athens afloat.

European Union President Donald Tusk has warned the Greek people that voting no in the referendum will not give their government more leverage to seek a better bailout deal: "Every government has a right to hold a referendum, therefore we respect the Greek decision. However, one thing should be very clear: if someone says that the government will have a stronger negotiating position with the `no' vote, it is simply not true. I'm afraid that which such a result of referendum, there will be even less space for negotiation."

Now that the Supreme Court has denied Google's petition and appellate attorney Joshua Rosenkranz (of Orrick Herrington Sutcliffe) has once again shown why he was dubbed the "Defibrillator" (for bringing cases back to life that appeared to have been lost), the sizable litigation caravan that had gone from California to Washington DC for the appellate proceedings--where an amazing reversal of fortunes occurred, with Oracle now having the upper hand--can finally head back all the way to the West.

But Github’s pending emergence as Silicon Valley’s latest unicorn holds a certain irony. The ideals of open source software center on freedom, sharing, and collective benefit—the polar opposite of venture capitalists seeking a multibillion-dollar exit. Whatever its stated principles, Github is under immense pressure to be more than just a sustainable business. When profit motives and community ideals clash, especially in the software world, the end result isn’t always pretty.

A request from Emilien Macchi for more collaboration between Fuel and the Puppet OpenStack project kicked off the discussion. Puppet is a configuration management utility that is used by Fuel to assist in deploying OpenStack. But Fuel has forked some of the Puppet modules it uses from the Puppet OpenStack project—which creates Puppet modules for OpenStack components—into its Fuel Library repository. Macchi noted a number of problems with how that has been handled over the last two years.

To be sure, it is a maxim of economics that the returns to innovation are often higher for new entrants than incumbents precisely because new entrant’s don’t care about the sales of existing products. But it is equally important to note that that maxim only arises if the incumbent expects entrants not to win with those new products. As soon as they expect that, because the incumbent has other assets — namely its core brand — the incumbent has a more powerful incentive to develop those new products than entrants. This is one way of looking at Facebook’s acquisitions. They are not so much about becoming a family of brands but instead, after uncertainty is resolved, continuing to fund innovation on products where the incentives to fund entrant’s doing them has fallen, not risen.

The site, If I Knew Then, is actually also a book written by Artie Buerk, a member of the Harvard Business School (HBS) class of 1963 and contains collected wisdom — all in quotation form — from his classmates, gathered in preparation for their 50th reunion.

A salesperson’s domain knowledge isn’t just a boon to sales, it’s an important component when sales brings back customer feedback. Engineers know when they’re dealing with a “coin-operated” salesperson, and they have no respect for the species. ‘Ah, he doesn’t know what he’s talking about!’. Actually, engineers don’t have much respect for sales, period, but they’ll listen to a technically competent salesperson, particularly if he or she is bringing back an esoteric bug report.

Work began in 2007 to raise the capacity of Gatun Lake and build two new sets of locks, which would accommodate ships carrying up to 14,000 containers of freight, tripling the size limit. Sixteen massive steel gates, weighing an average of 3,100 tons each, were built in Italy and shipped to Panama to be installed in the new locks. Eight years and $5.2 billion later, the expansion project is nearing completion. The initial stages of flooding the canals have begun and the projected opening date has been set for April of 2016.

It sounds like a lot more singing, and a much more accessible show, with songs plucked from both new and old. I'm sad I missed Wharf Rat and Eyes of The World, two of my favorites, but we had such a wonderful time on Saturday that I'm certainly not complaining!

Now, on to Chicago; I wonder what additional wonders they will pull out of that hat...

Sunday, June 28, 2015

Roughly 25 years after the last show I attended (New Year's at the Kaiser Auditorium in 1990, I think), we made it down to Levi's Stadium in Santa Clara last night for Fare Thee Well, a reunion concert of the Grateful Dead, consisting of the 4 surviving original band members (Bob Weir, Phil Lesh, Mickey Hart, Bill Kreutzman) joined by guitarist Trey Anastasio and keyboardists Bruce Hornsby and Jeff Chimenti.

We arrived early, and were able to navigate the legendary Levi's Stadium traffic (the stadium is built in the middle of a high tech office park, and leases the parking lots of the nearby office buildings to serve as stadium parking). 2 blocks away was the giant dirt lot that is normally the tailgate area for 49ers games, and it had been turned into a scene from Golden Gate Park in 1967.

After about 90 minutes of enjoying the scene, we made our way over to the stadium, where we had comfortable seats with a beautiful view in delightful near-perfect weather.

Even when night arrived, and a (very) light rain began to fall at about 9:00 PM, the weather remained perfect, as we were treated to a glorious rainbow above the stadium, perfectly framing the stage.

The band dug WAY back into the old days, and although they played two of my favorite songs (Uncle John's Band and Cumberland Blues), the bulk of the show drew on some of their oldest, freakiest, least accessible material, with pieces like Alligator, Born Cross-Eyed, and Cryptical Envelopment.

These were not sing-along crowd favorites (though they did open and close the show with those: Trucking to start and Casey Jones to end), but rather were dreamy "jam" pieces, stretching out for 10 or 15 minutes at a time, complete with light show playing across the enormous stage.

These long, instrumental pieces fit well with the composition of the band, as Hornsby and Anastasio are really more jazz musicians than folk artists, but really my favorite work by the Grateful Dead was always the folk and bluegrass sensibility that was truly Jerry Garcia's thing.

And, yes, they did indeed play Dark Star, and 75,000 grizzled old codgers (and a surprising number of what, 3rd generation Dead Heads?) sung along with "the transitive nightfall of diamonds".

So while it's no surprise that the show went the way it did, it's also probably no surprise to hear that it left me content, yet simultaneously even more wistful about the terrible loss that was Garcia's vastly premature death in 1995.

After 2 hours of driving, 2 hours of wandering around through the pre-show scene, and 5 hours (yes, they started at 7:00 and didn't finish until after midnight; a typical legendary Grateful Dead set list) of show, we were both exhausted, yet pleased that we were there.

It was a memorable event, and even though I'm not the lad I was 42 years ago when I heard my first Grateful Dead song (Tennessee Jed, as I recall), you can clearly take the Bryan out of the Grateful Dead, but you can't ever quite take the Grateful Dead out of Bryan.

Friday, June 26, 2015

Creating digital representations of people who aren't white indicates a culture and industry who view us as people. It counters the status quo that dehumanizes us by erasing us or casting us as non-humans. We want to be seen as people, too. There's little more to it, for me.

But seeing angry responses to this simple request speaks volumes about the kind of culture we're creating by not diversifying races, genders and so on. Consider: In The Witcher 3, all humans are white and every other being is non-human. That's not exactly friendly or inclusive of people of color. A game can include a diverse variety of monsters, but not a diverse variety of skin colors or races for humans?

Wednesday, June 24, 2015

Maybe it was the sheer exhaustion from being in the middle of a 19-day climb of the Dawn Wall, but when the guys at Google Maps and Yosemite National Park asked if I wanted to help them with their first-ever vertical Street View collection of El Capitan in Yosemite, I didn’t hesitate. Yosemite has been such an important part of my life that telling the story of El Capitan through Street View was right up my alley—especially when it meant working with the Google engineers to figure out some absurd challenges.

The Witcher 3: Wild Hunt is a daunting game: a huge, open world RPG and the end of a trilogy starring monster hunter Geralt of Rivia. What do you need to be prepared? First comes brushing up on the events of The Witcher and The Witcher 2, Geralt’s previous video game adventures. Both are big games, dense with characters and political machinations. But that’s not the beginning and end of the Witcher-verse. Geralt’s origins lie in a series of novels by author Andrzej Sapkowski, and guess what? Some important characters from those novels will be showing up in The Witcher 3.

Tuesday, June 23, 2015

(Yeah, yeah, yeah, all I can talk about nowadays is The Witcher 3: Wild Hunt.)

One of the things I particularly enjoy about the game is the nuance and shading of the moral decisions that our hero has to make during quests.

For example, at one point he comes across a grandmother caring for a number of orphan children and has to decide whether to help them or not.

But it turns out (trying to avoid too many spoilers here) that she might not actually be a grandmother, and although the children are indeed orphans, it's not clear that she's legitimately in their charge.

In another quest, an evil wraith is haunting the cemetery and attacking the townspeople, who plead with our hero to destroy the wraith.

Only it turns out that the wraith is the spirit of a young girl from the town, who was stoned to death by the townspeople some time ago for daring to date a boy from the other side of the lake...

And for a third example, a fairly complicated quest involves a wealthy woman who was killed by peasants who went on a rampage.

Only it turns out that the reason they went on the rampage is because the woman employed a rogue herbalist who was concocting experimental new potions and was testing them on peasants imprisoned for not paying their debts.

And the list goes on. The Witcher 3 is full of situations like this, where people plead with our hero to help them, while concealing or slanting the information they provide in various ways so that it's not obvious who is deserving of our help, and why.

It's not uncommon to complete a quest, only to learn that while some problems have been lifted from the population, others have been created, or made worse, and it's completely unclear whether the decisions that I made during the game actually made things better, or worse.

I almost NEVER do this in games, but several times I've chosen to back up and re-play sections in order to change my responses to certain quests.

Only to find that, in several cases, there was NO good answer to the problems. Both sides were at fault, both sides were unjust, and there is no simple happy ending for the problems that are raised.

It's so refreshing and addictive to interact with a game that draws you in so deeply, that makes you care so much about the fictional characters of the story, that makes you think carefully about why you're doing what you're doing, what impacts you will have, and whether there might be another way to solve the problem.

Now, where was I? Oh, yes, this group of farmers feels that the lord has unfairly raised their rents. Hmmm, let's see...

And Stephenson wants to talk about everything, so the book is full of topical subjects. There are discussions of Fukushima and of Facebook, there is an Elon Musk character and a Hillary Clinton character, and on and on.

Stephenson is so erudite, and has so many ideas about so many subjects, and so much imagination, that his book can barely contain it all. It is over-stuffed, with bits and pieces dribbling out everwhere.

It is encyclopedic.

There is even a real Encyclopedia Britannica in the book, and a character who spends her (young adult) life reading the encyclopedia.

Now, if you've ever actually sat down and read an encyclopedia, you know that it's not a lot of fun. Sometimes you find yourself reading a bunch of stuff that you're not terribly interested in.

Reading Seveneves is like that. A co-worker and I joked that, when you find yourself hitting one of those patches, you should just swipe "next page" on your Kindle twice, and resume reading a few pages later.

(Though actually, I didn't do that, I powered through every page.)

And Stephenson often seems to be almost deliberately encyclopedic, reaching over backwards for some sort of strange combination of balance and political correctness. You can almost hear him thinking out loud to himself: "let's make this next character an Eastern European woman who is an athlete and a lesbian. Then, let's see, I'll need to counteract that with a Chinese scientist who is an expert in nuclear science."

And to make sure we know he's an equal opportunity writer, Stephenson frequently reaches out and slaps us in the face with it:

"No," Luisa said. "Let's be clear, Markus, I have baggage too. I'm a brown Spanish speaker from South America. I devoted years of my life to hanging out with refugees on boats. And I'm a Jew. That's my baggage, okay"?

In his "Author's Notes", at the end of Seveneves, Stephenson observes that it took him nearly ten years to write the book. Obviously he had a lot to say, and in his ten years of labor he covered a lot of ground. But if we try to peel back the layers and layers of words and detect the individual themes, I think we might find something like this:

What would it be like for humans to live in space, not a few humans for a few days or months, but thousands and thousands of humans living in space for decades and centuries?

What would happen if humans tried to actively control and accelerate the course of evolution, using not just selective breeding but more aggressive techniques such as genetic engineering?

How can we get people thinking about big-picture questions, things that affect not just individuals but all people, and over time periods that span hundreds or thousands of years?

Clearly, these are some big questions, and I think Stephenson would be the first to admit that he isn't trying to answer these questions, just trying to actively explore them.

Obviously, the "life in space" theme is what people come to Stephenson to read: "hard" science fiction, realistic speculation about what lies ahead. And Seveneves is jam-packed with this material: page after page after page of discussion of vehicles, gear, propulsion, materials, techniques. If you want to spend some time (a LOT of time, actually) thinking about how you might accomodate different docking systems, or transition between orbits, or how the L1, L2, and L3 points are relevant to your trajectory and burn planning, Stephenson has you covered.

In spades.

For me, the most interesting part of this material was the discussion of robotics. Stephenson has obviously spent a HUGE amount of time thinking about how robotics will play out in giving humans a home in space, and the detailed descriptions of different types of robots, different ways to use robots, and different ways of living as a result of robots was fascinating.

On the other hand, the "life in space" aspects of Seveneves will be immediately (and justifiably) compared to this year's hit phenomenon, The Martian, and I'm sad to say that Seveneves comes up wanting.

It's not for lack of trying, it's something more basic. Stephenson's material just isn't fun, like The Martian is.

Rather, you get rubbish like this:

For now, her attention was captured by the giant machines in which the view was framed. Surrounding her, and just visible in her peripheral vision, was another of those ubiquitous tori, spinning around to provide simulated gravity for the staff who lived here with their families, looking after the tether and the elevator terminal. Inward of that were the sixteen orifices where the tether's primary cables were routed into the frame of the Eye. Each of those cables, though it looked solid from a distance, was actually made of sixteen more cables, and so on and so forth down to a few fractal iterations. All of these ran parallel between the Eye and Cradle. Webbing them together was a network of smaller diagonal tendons, arranged so that if one cable broke, neighboring ones would take the force until a robot could be sent out to repair it. Cables broke all the time, because they'd been hit by bolides or simply because they had "aged out," and so if you squinted your eyes and looked closely enough at the tether, you could see that it was alive with robots. Some of these were the size of buildings, and clambered up and down the largest cables simply to act as mother ships for swarms of smaller robots that would actually effect the repairs.

There's not the same thrill, the same pace, the same adrenaline.

And, really, there should be, because Stephenson's story is just as exciting, plot-wise; he just doesn't manage to carry it off like Andy Weir does. Which, I think, is more a matter of praising Weir than condemning Stephenson; I think it will be a while before another book as strong as The Martian comes along.

Meanwhile, the part of Seveneves which I found simultaneously most disturbing and yet most intriguing, is the substantial amount of time Stephenson spends exploring evolution and human biology.

Stephenson dances around the topic for a while, backing into it with a discussion of how dogs, wolves, and coyotes vary, and yet are really the same, and then discussing black-footed ferrets.

Later, after taking many hundreds of pages to demonstrate, at times quite vividly, just how messed up humans are, just how many flaws we have, and just how horribly those flaws lead us to perform the most terrible actions, Stephenson finally declares that it is, in effect, a bug (as we software engineers would term it):

When that trait ran out of control and sought dark paths it led to depression, paranoia, and other forms of mental illness. The challenge then was to find a way of combining that trait with a more positive mentality. Julia's research -- and she did a lot of research -- therefore tended to center on the history of sages, seers, ecstatics, shamans, artists, depressives, and paranoiacs throughout history, and the extent to which those traits could be localized to specific base pairs in their genomes and fostered by acculteration.

A bug? Well, let's fix it!

"Correction" was the name given to the phase that had begun after the first round of Gestations, when Eve Moira had fixed errors that had led to several nonviable infants. In a sense, Correction went on continuously all through the first round of Gestations and began to taper off as the daughters of the Eves began to produce second-generation children. It faded into a next stage, Stabilization, which lasted through the following ten generations or so as Y chromosomes were patched up, lingering genetic mistakes were fixed, and members of different Strains began to interbreed to produce hybrids within their own racial groups.

...

Selective mating had the power to wreak impressive changes over time, without any artificial meddling. In some cases, though, racial isolates had acquired genetic labs of their own. These had been used for many purposes, usally considered benign. In some cases, they had been used for Enhancement, which meant deliberate genetic manipulation for the purpose of rendering racial characteristics more pronounced -- the artifical acceleration of what was happening "naturally" in the way of Caricaturization.

...

their Eve had spawned not just one race but a "race of races," a mosaic, as proof that her children could do all that those of the other Eves could, and more.

What?

I mean, really, what the heck?

As you can see, this is some pretty bizarre stuff, and I really don't know what Stephenson is driving at with it. At times it seems to be a cautionary horror story, sort of a modern version of H. G. Wells's tale of the Eloi and the Morlocks.

At other times it appears to be an engineer's fantasy, in which those annoying politicians and managers stop running the show, and the engineers are free to retire to their labs and fix all the bugs in humanity. There's a lot of interest nowadays in the notion of the "Lifehacker," and Stephenson seems to be exploring just how far that idea could go.

Which, in my reading, isn't very far, and isn't very successful, though I'm not sure Stephenson would be pleased to hear me say that; I think he has (much) higher hopes for this than I do.

Regarding Stephenson's third theme, which is that of taking "the long view," it's clear that Stephenson has been deeply affected by the well-known Long Now Foundation, and if you're familiar with their work, you'll know how this aspect of Seveneves plays out. Here, Stephenson is sober, cautionary, and, I'm sorry, dull as dishwater.

Some of this is the inevitable consequence of feeling like you're being lectured by the dentist to floss more regularly, but I think some of it can be charged to the uneven pacing of Stephenson's story line. Stephenson will sometimes spend a hundred pages describing the passage of 3 days' time, then in the next hundred pages he'll cover 3 months' time, or 3 years' time, or even 3 centuries. It's not easy to try to tell a story that consumes five thousand years, but the feeling of fits and starts seems, in my opinion, to make it even harder.

Not that I think this is really possible. Writers like Michener have tried, but I think it's just an insuperable problem. You can be epic, but can you really be this epic?

There is, of course, one well-known book which covers something around 5,000 years of history, and it's regularly read and taught in Christian churches around the world. And certainly there are plenty of allusions to the Bible in Seveneves, starting with the seven Eves of the title, and continuing with the core of the story's plot: human beings are raised up into the sky, and live beyond the death of people down on the Earth's surface, surviving in a Noah's Ark of sorts until the time comes for (seven) Eve(s) to repopulate the world from the Garden of Eden.

But I can't follow that line of thought very far, and I'm not sure Stephenson wants me to, either. There is room for many professions and many skills in his future world, but there isn't a cleric in sight. When he wanders closest to the subject, this is the result:

It was a meditative piece, a secular eulogy for those who had fallen, but in retrospect it seemed to mark an inflection point in the survivors' thinking. The Swarm had always had a sort of quasi-divine status to some, who had perhaps read too much chaos theory too superficially and were prone to believing that its collective decisions, lying beyond human understanding, partook of the supernatural.

The mishmash of techno-mystical ideation that had grown out of that one blog post was unreadable and incomprehensible to Luisa or to anyone else who read it after the fact, with a clear mind, but it seemed to have offered hope and comfort to many terrified young people trapped in arklets.

So if Stephenson isn't interested much in religion in his history of the extra-terrestrial future, what replaces it?

The result seems to be rather deadening, in my view; it's a highly technocratic triumvirate of engineering, politics, and warfare. Stephenson seems to be saying that, many centuries in the future, certains things will still be true:

We'll be always be tool-builders

We'll always be divided by differences of opinion, causing us to form groups and hierarchies

And we'll always find things to fight over

So let's get going building robots and practicing our genetic engineering skills!

There's a lot to read in Seveneves, and a lot of it is fairly interesting.

But in the software world, it's not uncommon to find a software project which has been worked on for many years, which is big and powerful and complex and filled with all sorts of ideas, each developed more or less thoroughly, and yet, when you consider the result, it isn't really worth keeping or using.

In the software world, we call those "failed projects," and we don't release them (hopefully), and we start over.

Both of these organizations strongly support "cybersecurity" legislation, claiming that it's necessary so that the US government can "help" companies dealing with "critical infrastructure." And yet, here we are, with the government's own personnel files being held in a system without encryption that was hacked and copied by (likely) foreign hackers. And we're supposed to trust two government agencies who have been going around cursing encryption, that we should give them more access to "protect us" when another government agency's attack likely could have been prevented if they'd just used encryption?

If you're getting a 72% return rate on your game under those conditions, it sounds like nobody liked the first 2 hours of your game. I guess you can blame the refund policy for that, if you want, or you can simply make better games.

When I look around, I see the culture we’ve built turning from a liberating revolution into a repressive incumbency. We’ve built magical devices, but we don’t care enough about protecting ordinary people from harm when they use them. We don’t care that a lot of the children out there with the potential to become amazing hackers are driven away at every stage in the larval process. We don’t care about the people who lose out when we disrupt the world, just the winners (who tend to look a lot like us).

I’ve paid attention to people in groups and collaborating on projects. I’ve had occasion to do this as a team member and a team lead, as a boss and a line employee, as a consultant and as a team member collaborating with consultants, and just about everything else you can think of. And what I’ve observed is that this phenomenon is not a function of the people who have been fooled but the person doing the fooling. When you look at people who wind up being highly overrated, they share certain common habits.

Now is the time we in IT need to think about some kind of code of conduct. We desperately need something we can refer to when we are not sure what should be happening, how we should be responding to some event in the world. But for that to work, we also need to accept that we are a political group with some real power and not just a bunch of kids playing with bytes.

I'm not arguing for legally enforced rules and I don't want the debate to go in this direction. We are not lawyers, we are hackers, and we know that any kind of rules can, and will, be bypassed. It is our job.

The laws are comparable to the technical limitations that a developer puts on a web page to forbid you to read it without entering your email address: disable JavaScript and it just works. Our job, and most of the time also our hobby, is to bypass those limitations. Not to do anything bad, but because it is fun. Sometimes we even tell developers that they should do it differently, to be safer.

There's a lot of details, and I recommend reading them. There was probably a Kerberos zero-day vulnerability involved, allowing the attackers to send updates to Kaspersky's clients. There's code specifically targeting anti-virus software, both Kaspersky and others. The system includes anti-sniffer defense, and packet-injection code. It's designed to reside in RAM so that it better avoids detection. This is all very sophisticated.

From the moment humans discovered the benefit of staying in one spot, growing plants and raising animals, they started to devise ways of bringing water to the settlement. They developed wells and rainwater channels, dug irrigation ditches, and created aqueducts to pull water a long way from its sources. Their engineering paid off; villages, towns, cities flourished; eventually, complex, urban civilizations dominated. It smacked of success.

Except that great solutions to water needs also begat great problems. When water is drawn more quickly than it is replenished naturally, a water source can dry out, endangering people who based their lives around it. Some of our engineered structures, like lead pipes, have led to serious health problems. And when more than one nation draws from a single water source—as in the case of the Jordan River, which feeds into Israel, Jordan, and Syria—tensions can escalate into conflict.

We need water to survive. But the ways we get it have changed our ways of life, driven us from our homelands, and in some cases, poisoned those who’ve stayed. Here are some examples of problems we’ve created by being too good at what we do: surviving.

You can’t actually pin down real-world time, because you can’t measure an actual instant in time. We like to think that we talk about an instant of time, but what we’re really talking about is a duration. 6:14 PM isn’t an instant, it’s a duration of time between 6:14 PM and 6:15 PM. Increasing your precision doesn’t change this. A second is always a second long. A nanosecond is still just a duration. No matter how finely you slice it, multiple events can happen within a single second, nanosecond, femtosecond, and so forth.

Friday, June 12, 2015

The conference had accepted an apolitical presentation on a fairly obscure project by a software engineer named Curtis Yarvin, only to reject it last week after it received complaints about political views Yarvin espoused on his blog.

Once you control Harvard Law, you control the courts. Once you control the courts, you control the laws. Once you control the laws, you control the people.

Or, alternatively: once you control the technology conferences you control the team leads, once you control the team leads, you control the engineers, once you control the engineers, you control the tech industry, once you control the tech industry, you control the 21st century economy.

Where things were done wrong was not Strange Loop 2015, but about PyCon 2013. That was no doubt part of a larger pattern, but it is the place to focus.

PyCon 2013 was of course the venue for DongleGate. The accusations and responses over Urbit, Moldbug and Strange Loop are all quite beside the point, and can be harmlessly ignored, but I urge everyone to read about DongleGate.

One of the wonderful things about the modern Internet is the rich library of recorded music that is easily available.

I spent several fine hours this week listening to as many different performances of Beethoven op 57 as I could find.

And there are a lot of them out there to find!

Daniel Barenboim

Jonathan Biss

Arthur Rubenstein

Vladimir Horowitz

Claudio Arrau

Alfred Brendel

And on and on and on...

I was inspired to do this by listening to Biss's wonderful lecture series on Beethoven's piano sonatas; Mr. Biss has just released a new set of lectures including one on the 23rd sonata.

This, frankly, wasn't one of my most-loved sonatas the first few times I listened to it.

But Biss motivated me to spend more time with it, and to seek out different interpretations.

And being able to hear so many different performances, in such a short period of time, is a truly wonderful way to appreciate the richness of the work and the astonishing amount of depth and detail that each performer and each performance is able to find within it.

Even the same pianist reacts to the work differently over time. For example, there are several Barenboim recordings easily available, over a span of 30-plus years, from when Barenboim was still young and just emerging onto the world scene, through recent performances when he had a lifetime of experiences and reflections to draw on.

It's really a joy to live in such times, to have such things available.

A real-time streaming system demands certain systemic qualities to analyze data at a large scale. Among other things, it needs to: process of billions of events per minute; have sub-second latency and predictable behavior at scale; in failure scenarios, have high data accuracy, resiliency under temporary traffic spikes and pipeline congestions; be easy to debug; and simple to deploy in a shared infrastructure.

To get a feel for message latency, we’ll use the analogy of a train going through a tunnel. This setup is shown in Figure 1. The train represents the message; longer trains stand for larger messages. The tunnel represents the link connecting the sender and receiver. The message latency is the time it takes for the train to enter the tunnel and leave the other side.

This is a personal view of the first dozen years of the history of the field of concurrency—a view from today, based on 40 years of hindsight. It reflects my biased perspective, so despite covering only the very beginning of what was then an esoteric field, it is far from complete. The geneses of my own contributions are described in comments in my publications web page.

Collaborative filtering (CF) is one of the important areas where this applies. CF is a recommender systems technique that helps people discover items that are most relevant to them. At Facebook, this might include pages, groups, events, games, and more. CF is based on the idea that the best recommendations come from people who have similar tastes. In other words, it uses historical item ratings of like-minded people to predict how someone would rate an item.

Specifically, we focus our study on the common use of feral, or application-level, mechanisms for maintaining database integrity, which, across a range of ORM systems, often take the form of declarative correctness criteria, or invariants. We quantitatively analyze the use of these mechanisms in a range of open source applications written using the Ruby on Rails ORM and find that feral invariants are the most popular means of ensuring integrity (and, by usage, are over 37 times more popular than transactions).

If applications do not partition perfectly, then transactions accessing multiple partitions end up being distributed, which has well-known scalability challenges. To address them, we describe a high-performance transaction mechanism that uses optimistic concurrency control on a multi-versioned tree-structured database stored in a shared log. The system scales out by adding servers, without partitioning the database.

These new NVM devices are almost as fast as DRAM, but all writes to it are potentially persistent even after power loss. Existing DBMSs are unable to take full advantage of this technology because their internal architectures are predicated on the assumption that memory is volatile. With NVM, many of the components of legacy DBMSs are unnecessary and will degrade the performance of data intensive applications.

We present a novel MVCC implementation for main-memory database systems that has very little overhead compared to serial execution with single-version concurrency control, even when maintaining serializability guarantees. Updating data in-place and storing versions as before-image deltas in undo buffers not only allows us to retain the high scan performance of single-version systems but also forms the basis of our cheap and fine-grained serializability validation mechanism.

In this paper, we describe a system that can compute efficiently complex join queries, including queries with cyclic joins, on a massively parallel architecture. We build on two independent lines of work for multi-join query evaluation: a communication-optimal algorithm for distributed evaluation, and a worst-case optimal algorithm for sequential evaluation.

The annual ACM SIGMOD/PODS conference is a leading international forum for database researchers, practitioners, developers, and users to explore cutting-edge ideas and results, and to exchange techniques, tools, and experiences.

My regular home machine is a plain old Dell Inspiron 660. It's a low-end machine, but it's been just fine for several years now.

One weakness of the 660, though, is the video card I got with it: the nVidia GT 620 is definitely under-powered for modern software.

So I pulled the trigger on a nVidia GTX 750 Ti. The 750 series is a low-power version of the GTX line, definitely not the most powerful, but able to fit cleanly into my small-format case, and able to survive on the power supply delivered through the PCI-Express slot without needing special power cabling (and an upgrade to my Dell 300w power supply).

The physical installation of the card was straightforward, but then I had the oddest symptom: the nVidia device drivers wouldn't install, telling me that there was no nVidia device in my machine.

Well, of course there was! My monitor was plugged into it!

A lot of web searching took me in a lot of wrong directions, with a lot of poor advice dating from pre-Windows-7 times (I run Windows 8.1), and I came near to bricking the machine several times (though I did successfully upgrade my BIOS, which was worthwhile).

Luckily, I avoided the worst of the poor suggestions, and instead kept staring at the Device Manager display.

Two things then occurred, almost simultaneously:

I found View -> Show Hidden Devices, which told me that there were ghosted devices in my machine which were being blocked for software reasons

I found a device buried at the bottom of the Device Manager display with the crucial yellow question mark next to it:

After several more tries, that device finally agreed to install updated device drivers from the net, the question mark went away, and "GTX 750 Ti" finally appeared in my Display Adapters section of the Device Manager display.

Cluster schedulers have existed long before big data. There's a rich literature on scheduling on 1000s of cores in the HPC world, but their problem domain is simpler than what is addressed by datacenter schedulers, meaning Mesos/Borg and their ilk. Let's compare and contrast on a few dimensions.

Mesos decides how many resources to offer each framework, based on an organizational policy such as fair sharing, while frameworks decide which resources to accept and which tasks to run on them. While this decentralized scheduling model may not always lead to globally optimal scheduling, we have found that it performs surprisingly well in practice, allowing frameworks to meet goals such as data locality nearly perfectly. In addition, resource offers are simple and efficient to implement, allowing Mesos to be highly scalable and robust to failures.

The replication protocol in ZooKeeper assumes that servers can crash and recover, and the ensemble can make progress as long as f + 1 members are up and running, where f is a bound on the number of faulty servers determined by the size of the ensemble. If the ensemble has 5 servers, then the ensemble can make progress as long as 3 are up (f = 2). It additionally assumes that the disk state before a server crash is there during recovery. This post is pointing out that this isn’t always the case, and it is an issue to be aware of. Note that this isn’t unique to ZooKeeper, but I’ll focus on ZooKeeper because I know it well.

We can think that the total processing capacity of a system will increase when the capacity of a part of the system increases, and so we expect improvements in performance objectives accordingly in that case. The famous Braess paradox tells us that this is not always the case; i.e., increased capacity of a part of the system may sometimes lead to the degradation in the benefits of all users in a Wardrop equilibrium. We can expect that, in the Nash equilibrium, a similar type of paradox occurs (with large N), i.e., increased capacity of a part of the system may lead to the degradation in the benefits of all classes in a Nash equilibrium, whenever it occurs for the Wardrop equilibrium. We call it the Braess-like paradox.

The processors of the future can be mindblowingly, blazingly quick and it won't make the slightest bit of difference if memory can't keep up. A CPU/GPU needs quick access to its addressable memory banks—RAM, generally—because, otherwise, speed is just a number.

Adding technology to your company comes with a cost. As an abstract statement this is obvious: if we're already using Ruby, adding Python to the mix doesn't feel sensible because the resulting complexity would outweigh Python's marginal utility. But somehow when we're talking about Python and Scala or MySQL and Redis people lose their minds, discard all constraints, and start raving about using the best tool for the job.

Microservices are a useful architecture, but even their advocates say that using them incurs a significant MicroservicePremium, which means they are only useful with more complex systems. This premium, essentially the cost of managing a suite of services, will slow down a team, favoring a monolith for simpler applications. This leads to a powerful argument for a monolith-first strategy, where you should build a new application as a monolith initially, even if you think it's likely that it will benefit from a microservices architecture later on.

Rather than wasting time ranting and railing at the Elephants (although he did some of that as well), he just built successful companies that showed the ideas worked well enough that they actually could sell successfully against the elephants. Not only did he build companies but he also helped break the lock of the big three on database innovation and many database startups have subsequently flourished. We’re again going through a golden age of innovation in the database world. And, to a large part, this new period of innovation has been made possible by work Stonebraker did.

Distributed systems affect the user. We need to shift the focus from system properties and guarantees to business rules and application behavior. We need to understand the limitations and trade-offs at each level in the stack and why they exist. We need to assume failure and plan for recovery. We need to start thinking of distributed systems as a UX problem.

I had to read that a couple of times. Turcke’s daughter had used a paid service to get content and her parent was telling her that it was stealing. I am not sure what it is but it is not stealing. If you pay for something for the same amount as an equivalent human across the border, that is not stealing. The message Turcke was teaching her daughter and now was trying to tell all Canadians was that they were second class. They couldn’t get the content that others could get because companies like Bell Media had paid for them not to do so.

Can they find a way to make PGP easy to use? That encompasses a wide range of activities: composing encrypted and/or signed email, receiving it and immediately realizing its status, being able to search encrypted messages—and doing all this without undue mental effort. Even for sophisticated users, it's really easy to make operational mistakes with encrypted email, mistakes that gut the security.

Some way of asking the generated code to perform dynamic overload resolution at execution time… not based on dynamic values, but on the basis that the code we’re compiling against may have changed since we compiled. This resolution only needs to be performed once, on first execution (or class load, or whatever) as by the time we’re executing, everything is fixed (the parameter names and types, and the argument names and types). It could be efficient.

Some way of forcing any call sites to use named arguments for any optional parameters. (Even though in our case all the parameters are optional, I can easily imagine a case where there are a few required parameters and then the optional ones. Using positional arguments for those required parameters is fine.)

"On a high level, what we’ve determined as of this week is that — unless the lobbyists derail our efforts – we’re going to ask for different authentication measures on a new customer, and different on returning customer, and then we’re going to ask for whole bunch of data elements that we’re not getting now that will allow us to filter the returns on receipt and will allow us to put the returns in various buckets of scores for possible fraud."

For example, one telltale sign of a fraudulent return is one that takes the filer a very short time to fill out.

"If someone takes two minutes or less to fill out a tax return, that’s pretty much fraud 100 percent of the time, because they’re just cutting and pasting information from somewhere else," said Magee’s deputy Garrett. "So we said, okay, send us information about how long it takes them to fill out a return."

When the money was deposited into the Sunrise account, TPG extracted three fees: $35 for handling the federal refund, $10 for state refunds and $10 fee for TurboTax (since thieves had used TurboTax to fraudulently file his request.

Another $2,000 from the refund was diverted to an Amazon gift card. For thieves, diverting some of the funds to Amazon hedges their bets in case somehow the prepaid card that receives the bulk of the funds gets canceled by authorities cracking down on tax return fraud. These gift cards also are easily resold for cash.

"For Amazon, it guarantees a flow of future purchases in the Amazon system, and potentially generates more profit as consumers often forget to use all the value on their gift cards," Garrett said.

Ever since it assumed its hegemonic role in the ashes of World War II, America has played the part of what Josef Joffe calls the “default power” in maintaining global peace and security. It’s why Israelis and Palestinians both trust America, and not, say, Russia or Saudi Arabia, as an “honest broker” in peace negotiations, why the Europeans long ago consented to American security dominance on their continent, and why nearly all Asian countries appeal to us in containing an expansionist China.

To my wonderment, a check for $16 arrived in the mail about a week later. No explanation; no apology; and obviously no need for Bill Graham’s lawyers to waste their time arguing with a disappointed hippie law student. That was not good enough for me, however, as there was still the matter of my $2.25 filing fee, which I was entitled by law to recover.

Tuesday, June 2, 2015

Blatter, who has been in charge of the organization since 1998 and won re-election for a fifth term Friday despite allegations of mass corruption within FIFA, announced at a press conference in Zurich that he would step down after a special congress is called between December and March to appoint his replacement.

Meanwhile, so I'm not just tweeting, I find myself wondering, since this Saturday one team or another will win their triple, which is the most prestigious triple in the world?

Treble is used in association football to refer to a team winning three trophies in a single season. Honours usually considered to contribute to a treble are the top-tier domestic league competition, primary domestic cup competition and most prestigious continental cup competition, although this depends to some extent on the football competitions of a particular country.

It seems to me, given that it's only occurred in South America once, and that was over 50 years ago, and oh, by the way, that team had Edson Arantes do Nascimento on the roster, the answer must clearly be that the South American triple is the most prestigious of them all.