Thursday, April 30, 2015

This is the kind of "bug report" that makes programming for users so frustrating. You hear a user is losing data, but have absolutely no idea how to reproduce the problem. It's something you've never seen the software do, can't imagine it actually doing. The description sounds like magic, but nothing in software happens by magic, all programmers will tell you. I believe it's happening but I have no idea why.

And then, after plenty of work and sweat and tears

we had arrived at the Holy Moment in debugging -- reproducibility. I told him "reproducible" is the programmer's favorite word. If you can tell me the steps to reproduce the problem, then I can find it and fix it. Until it's reproducible all I can do is share your frustration.

But the NBA is about matchups, and not all matchups are equal. Just as the Grizzlies will have to solve, or at least partially impede, Curry and Thompson, the Warriors will have to slow down Memphis' post play.

And while it's easy to game-plan rotations and plan for various scenarios, on-court situations usually lend themselves to chaos. Will the Grizzlies' perimeter players be able to knock down shots and punish Golden State for overhelping? Will Green and Bogut handle their business enough to prevent the need for help in the first place?

Tuesday, April 28, 2015

Now, tonight, I turn on the news and I see politicians calling for young people in Baltimore to remain peaceful and "nonviolent." These well-intended pleas strike me as the right answer to the wrong question.

...

When nonviolence is preached as an attempt to evade the repercussions of political brutality, it betrays itself. When nonviolence begins halfway through the war with the aggressor calling time out, it exposes itself as a ruse. When nonviolence is preached by the representatives of the state, while the state doles out heaps of violence to its citizens, it reveals itself to be a con. And none of this can mean that rioting or violence is "correct" or "wise," any more than a forest fire can be "correct" or "wise."

Sunday, April 26, 2015

Programmers have an expression for a particular type of flaw in program code.

That's fragile, they say.

When a programmer points to a segment of code and calls it fragile, they mean something very specific.

They mean that a minor, innocent-looking, apparently-unrelated change to the overall program at some later date would cause this bit of code to unexpectedly fail without warning.

It might be because this code is in fact linked to the behavior of some other part of the system, but the linkage is not made explicit. For example, you might have an array which needs to be the same size as some other data structure elsewhere in the system, but there is nothing that is checking that those two conditions match, so changing the other part of the system would break an assumption that this code doesn't check.

The essay talks about the financial system, and compares it to various other situations in which an otherwise minor event had catastrophic consequences:

This is why asking whether the Waddell & Reed sale, or the behavior of a manipulative trader, really caused the crash is a mistake. The disparate “causal” explanations of the crash can’t be reconciled with each other for a simple reason: They aren’t in conflict. The Flash Crash was an emergent phenomenon. Just as any grain of sand might cause the sandpile to collapse, and as Three Mile Island’s meltdown could be attributed to a failed pump, stuck valve, or operator error, the trigger for the Flash Crash could have been related to Sarao, Waddell & Reed, or something else entirely. The true roots were in the complexity of the system itself.

In large-scale systems programming (database systems, distributed systems, web servers, network file systems, etc.), there is a problem that arises when the system reaches a certain size and complexity: you can no longer hold the whole thing in your head. If you aren't careful, when your system reaches that size, you will find it completely breaking down: bugs crop up left and right, unexpectedly, faster than you can fix them; you feel like you're playing "Whack-a-Mole" with the breakages in your software.

The only way out of this is to graduate to a whole new type of system design and implementation. Your overall system must be componentized; individual modules must have clear responsibilities and clean interfaces; the boundaries between modules must be well-known to all the teams; extra care must be taken with code paths which cross module boundaries; system interfaces must check their parameters, assert their pre-constraints, and generally validate that the overall rules of the system are being obeyed.

Is the modern international financial system effectively a giant software system? The authors of the Nautilus essay clearly think so:

In the years since the Flash Crash, the SEC has implemented measures to reduce tight coupling in the markets, and exchanges now pause trading if there are drastic price moves in individual securities. These measures help, but are they enough? The fundamental interactive complexity of the market and the unpredictable and difficult-to-observe interactions between software components, trading models, and market participants remain in place.

Saturday, April 25, 2015

The straightforwardly-named Google Testing Blog is an instance of the "group blog" category, where a collection of people, all of them Google employees (as far as I know), publish articles about the art and science of software testing.

I read the Google Testing Blog faithfully, partly because I'm rather obsessive about software testing.

Computer programming is a profession that appeals to and rewards the obsessive, and within that broader group of people, the still more obsessive sorts like me often obsess about testing.

Computer programmers obsess about testing the way that car lovers obsess about oil changes, the way that NBA athletes obsess about free throws, the way that sushi chefs obsess about knife selection. Testing is a tool in the programmer's toolkit, but you when you see it wielded with experience and training, it is an amazingly powerful tool.

Testers can invest their time in writing many types of automated tests, including unit tests, integration tests, and end-to-end tests, but this strategy invests mostly in end-to-end tests that verify the product or service as a whole. Typically, these tests simulate real user scenarios.

I'm not sure where the author came up with this strawman, frankly. In all my decades of professional software development, it's been a long, long time since I've been around anyone who's suggested that we invest "mostly in end-to-end tests that verify the product or service as a whole." However, I'm sure there could be such people, and indeed we see that there are entire books on the subject (of course: there are entire books on any subject).

Anyway, back to the Google Testing Blog. The author then proceeds to relate a

composite sketch based on a collection of real experiences familiar to both myself and other testers

The article uses these "real experiences" to demolish the idea of end-to-end testing with a scenario so bizarre, so fanciful, so implausible that I can hardly believe it:

Let's assume the team already has some fantastic test infrastructure in place. Every night:

The latest version of the service is built.

This version is then deployed to the team's testing environment.

All end-to-end tests then run against this testing environment.

An email report summarizing the test results is sent to the team.

The deadline is approaching fast as our team codes new features for their next release. To maintain a high bar for product quality, they also require that at least 90% of their end-to-end tests pass before features are considered complete. Currently, that deadline is one day away:

Days Left Pass % Notes
1 5% Everything is broken! Signing in to the service is broken. Almost all tests sign in a user, so almost all tests failed.

Uhm, what?

This is wrong on so many levels that it's hard to know where to start.

Did any of these tests ever pass?

When was the last time they passed?

Did they all pass with 2 days left?

What did the team do on the day when the "email report summarizing the test results" first reported that "almost all tests failed"?

Whatever went wrong with this project, though, one thing is very clear to me:

The testing strategy is not the problem here.

Any team which allowed themselves to think they were at a point where the "deadline is one day away" and yet "almost all tests failed" is so poorly managed, so inexperienced, so lacking in commonsense that they aren't going to solve their problems by blaming the testers or their test strategy.

Tests are one barometer of project progress, but there are many other metrics that any successful software development project uses, combined with that innate sense that experienced software developers acquire that tells them just how close they are to something that is actually ready.

And if you choose to ignore the information that's available to you, that's your fault. If the testers had followed a different strategy, and produced a different set of tests, the team could just as well have ignored that data entirely, too.

Trying to put aside the invective of the article, and its caricatured depiction of a project wildly out of touch with reality, what is the article really trying to say?

I think the points they wish to make are:

The sooner you can receive feedback from your tests, the sooner you can act on it

Smaller, more focused tests are cheaper to write and faster to run

If your tests run fast, you can run them very often

If you run your tests very often, they will more clearly point to the instant when a problem was introduced into the code

But if you test only individual components or modules, problems can creep in where the modules and components must be assembled into larger software systems, so don't entirely omit complete system tests.

Really, this is well-trodden ground. Every time I see an article like this, I bemoan the fact that Martin Fowler's work on Continuous Integration is now 15 years old, yet seems to be so little-known to so many people.

An important thing to decide is what makes a successful build. It may seem obvious, but it's remarkable how this can get muddy. Martin once reviewed a project. He asked if the project did a daily build and was answered in the affirmative. Fortunately Ron Jeffries was there to probe further. He asked the question "what do you do with build errors?" The response was "we send an e-mail to the relevant person". In fact the project hadn't succeeded in a build for months. That's not a daily build, that's a daily build attempt.

And Martin Fowler isn't the only one who's been talking about these basic principles for decades. For example, consider Joel Spolsky's Daily Builds Are Your Friend (again, nearly 15 years old):

If a daily build is broken, you run the risk of stopping the whole team. Stop everything and keep rebuilding until it's fixed. Some days, you may have multiple daily builds.

On large teams, one good way to insure that breakages are fixed right away is to do the daily build every afternoon at, say, lunchtime. Everyone does as many checkins as possible before lunch. When they come back, the build is done. If it worked, great! Everybody checks out the latest version of the source and goes on working. If the build failed, you fix it.

I think that the Google Testing Team are primarily trying to convey the notion that different types of tests are useful for different purposes, and you need to have a complete collection of tests, using lots of different testing approaches, to consider your testing strategy complete.

In fact, they even discuss the notion of the "testing pyramid" at the end of their article, though it's a shame that they don't point to the original source of the "testing pyramid" notion, Mike Cohn's The Forgotten Layer of the Test Automation Pyramid

I'm pleased that the Google Testing Blog is publishing articles on testing, I just wish they'd dig a bit deeper into the history of the field, and take a more modern approach, rather then putting up strawmen that haven't been in favor in decades and then tearing them down as if they'd just had a bold new vision of how to build quality software.

Hello. We have been trying to reach you. This call is officially a final notice from IRS, Internal Revenue Services. The reason of this call is to inform you that IRS is filing lawsuit against you. To get more information about this case file, please call immediately on our department number NNN NNN NNNN.

It was chilling. Lawsuits? The IRS is filing a lawsuit against me?

But something about the call didn't sound right.

Well, actually, MANY things about the call didn't sound right:

It was a robot, not a person

It didn't greet me by name

It was full of awkward, incorrect English ("Internal Revenue Services"?!!)

The new Tax Scams video describes some basic tips to help protect taxpayers from tax scams.

These callers may demand money or may say you have a refund due and try to trick you into sharing private information. These con artists can sound convincing when they call. They may know a lot about you, and they usually alter the caller ID to make it look like the IRS is calling. They use fake names and bogus IRS identification badge numbers. If you don’t answer, they often leave an “urgent” callback request.

Yep, that matched, quite well.

A nice article at the South Bend Tribune was helpful, too, as it even included the same fake phone number that had appeared on my phone.
Credit 'charge' appears very real

I have repeatedly written about IRS scam telephone calls but I am doing so again as your BBB continues to receive many questions from area residents who are concerned about receiving such calls. Caller IDs are showing all kinds of phone numbers, which pretty much indicates the numbers are being spoofed. Some have reported their Caller ID shows 585-310-3870, 725-422-5697 and 726-597-6584, but the IRS impersonator provides different numbers on the message.

Most recipients are being told “this is your final notice from the IRS” and “a lawsuit is being filed against you for failure to pay taxes.” Some are saying if the taxes are not paid at once, a warrant will be issued for your arrest and the police will be coming after you. Consumers are then told taxes must be paid “immediately.” Instructions are given to wire the money via Western Union or get an advance cash card such as Green Dot MoneyPak from your local drugstore or retailer namely Wal-Mart, Kmart or Target.

I read about lots of scary, annoying stuff, but rarely do I actually get one of these myself.

In a weird way, it was good to get one; it kind of was a tune-up, a practice exam, a drill.

A good reminder that it's a strange world out there, and you should stay on your toes and not fall for the nasty scam.

Oh, and yes: I simply deleted the voicemail (though I did file a complaint on the FTC's website for reporting telescams, and I did re-check that my phone is on the do-not-call list, which it has been for years).

Wednesday, April 22, 2015

Tim Massad, the head of the Commodity Futures Trading Commission, which oversees the trading of futures and swaps, said on Wednesday that it took so long to charge Sarao because of the size and complexity of U.S. derivatives markets. "These are huge markets," he said. "There's a lot going on."

The faster your trading system, the quicker you can take advantage of those discrepancies. But that is not all the high-frequency traders can do. They can “front-run” news events, jumping on anything that might move a price, and get in and out of a stock before anyone else knows that something has happened. By trading at such speed, they can even get into an equity in the gap between the placing of a large buy order and the execution of it, and thereby buy just before the price rises.

Sherrod Brown, the top Democrat on the Senate banking committee, told the Financial Times: “It’s encouraging that the Justice Department and [Commodity Futures Trading Commission] are pursuing this case, but troubling that it has only come to light now with the help of a whistleblower who invested substantial time in putting the pieces together.”

Adding to concerns, the CFTC was alerted to Sarao's alleged misdeeds by a whistle-blower, who has not been identified, according to Shayne Stevenson, who represents the whistle-blower through Hagens Berman law firm in Seattle. Stevenson said his client brought "high-quality information" about "market manipulation" to the CFTC, which alerted the DOJ.

A British man accused of market manipulation that contributed to the May 2010 Wall Street "flash crash" said he opposed being extradited to the United States, while the operator of the market where he traded sought to rebut prosecutors' suggestion that futures helped cause the crash.

More than a year before the May 6, 2010 "flash crash," CME Group noticed questionable trading in its E-mini market by a particular electronic trader who was placing orders and cancelling them.

As the crash whipsawed the futures and stock markets in 2010, CME saw the suspicious activity again and warned the trader that day that orders must be placed “in good faith,” without an intent to cancel. The trader responded two weeks later: “Kiss my ass.”

He seems to have executed his trades out of a modest, semi-detached house under the Heathrow flight path that he shared with his parents. He used off-the-shelf software that he souped up to make his bets. He named one shell company "Nav Sarao Milking Markets Ltd." The guy showed up to court in a pair of white sweatpants.

But surely, since then regulators went through the action that took place on and around the day of the Flash Crash with a very fine comb? And if so, why did Sarao's alleged actions not jump at them as strange at the time?

The CME Group declined to comment, citing the ongoing investigation. But analysts worry that the CME’s revenue model interferes with its motivation to police trading. The more trades that zip through the exchange, the more money it makes. That means it could be disincentivized from tackling manipulative traders who still bring valuable liquidity to the market.

Junk Debt collapsed a full hour before the stock market Flash Crash took place (and I would add that utilities and Treasurys were leading prior, indicating a VIX spike was possible). If junk debt collapses, how can stocks not collapse afterward when in a bankruptcy proceeding, junk debt has a higher claim on assets than equity? What caused the Flash Crash in stocks was a Flash Crash in junk debt, which was like a 1987 style crash in credit.

in or about June 2009, Defendants modified a commonly used off-the-shelf trading platform to automatically simultaneously “layer” four to six exceptionally large sell orders into the visible E-mini S&P central limit order book (the Layering Algorithm), with each sell order one price level from the other. As the E-mini S&P futures price moved, the Layering Algorithm allegedly modified the price of the sell orders to ensure that they remained at least three or four price levels from the best asking price; thus, remaining visible to other traders, but staying safely away from the best asking price. Eventually, the vast majority of the Layering Algorithm orders were canceled without resulting in any transactions. According to the Complaint, between April 2010 and April 2015, Defendants utilized the Layering Algorithm on over 400 trading days.

The Complaint alleges that Defendants often cycled the Layering Algorithm on and off several times during a typical trading day to create large imbalances in the E-mini S&P visible order book to affect the prevailing E-mini S&P price. Defendants then allegedly traded in a manner designed to profit from this temporary artificial volatility. According to the Complaint, from April 2010 to present, Defendants have profited over $40 million, in total, from E-mini S&P trading.

The SEC has been seeking evidence of abuse of order types, as well as traditional forms of abusive trading like "layering" or "spoofing" and other issues relating to high-frequency trading that might be violations of the law, SEC Director of Enforcement Andrew Ceresney told Reuters in May (reut.rs/1kwSqF5).

Spoofing and layering are tactics where traders places orders that they cancel before they are executed to create the false impression of demand, aiming to trick others into buying or selling a stock at the artificial price.

I'm pleased that investigators continue to investigate.

On the other hand, even after 5 years the investigators still appear to be uncertain as to exactly what happened and why.

Saturday, April 18, 2015

“Fault creep” is a condition that results when the underlying geology is too soft to get stuck or to accumulate tectonic stress: in other words, the deep rocks beneath Hollister are slippery, more pliable, and behave a bit like talc. Wonderfully but unsurprisingly, the mechanism used to study creep is called a creepmeter.

The ground sort of oozes past itself, in other words, a slow-motion landslide at a pace that would be all but imperceptible if it weren’t for the gridded streets and property lines being bent out of shape above it.

But "Hurr durr I'ma sheep" trounced "I like online polls" by a 62-to-38% margin, in a poll that people weren't even supposed to participate in. Who can argue with solid numbers like that? 5,796 votes from people who can't even follow the most basic directions?

Looking at just the statistics in git, this release is not just when we cross half a million commits total, but also cross the 4 million git object limit. Interestingly (if you look for numeric patterns), Linux 3.0 was when we crossed a quarter million commits and 2 million git objects, so there's a nice (and completely unintentional) pattern there when it comes to the kernel git repository.

The problem is that, thanks to the current drought, the water supply is going away faster than expected. The almond industry is an indicator of how difficult it might be to adapt to climate change, economically and environmentally.

What we’re witnessing in California right now is a glimpse into the future. California has now endured drought in 11 of the last 15 years, and there’s every reason to believe this is just the beginning.

This morning I had the pleasure to be a mathematical guest in my daughter’s third-grade class, full of inquisitive eight- and nine-year-old girls, and we had a wonderful interaction. Following up on my visit last year (math for seven-year-olds), I wanted to explore with them some elementary ideas in graph theory, which I view as mathematically rich, yet accessible to children.

I decided to build a distributed replication layer using the Raft consensus protocol, which gives me effective replication without the hassle of running a much heavier solution like MySQL. It provides all the advantages of replication, with the data modelling functionality of a relational database, but with the convenience of a single-file database.

While crossword construction is unavoidably linguistic, constructors these days rely heavily on data and programming. Take, for instance, Puzzle 5, always the hardest of the tournament. This year, Jeff Chen was the sadist behind the curtain. An aspiring novelist from Seattle, he runs the crossword database XWord Info and has authored or co-authored 37 New York Times puzzles.

Chen’s Puzzle 5 was titled “Attention, Newbies!” The conceit was adding new B’s (get it?) to familiar phrases. “Vocalist” became “vocab list,” “alloy” became “ballboy,” and so on. To find workable and interesting phrases like this, Chen wrote a Python script and applied it to his master word and phrase list. Using this program, he realized “caroms” could become “car bombs,” for example.

To give an idea of the scope of the demo, Engelbart demonstrated an early look at word processing, windowing, hypertext, and dynamic file linking, as well as using graphics in a computer program. It was also the first time many of the attendees had seen a mouse, although work on the mouse began in 1963.

Soleri taught architecture at ASU, but his real classroom was Arcosanti: "an experimental town and molten bronze bell casting community" a few hours from Phoenix, which was supposed to have 5,000 residents by now, demonstrating "lean" city living in high-density complexes that combined super-efficient usage of space with stylized, naturalistic exteriors that made each building part of the landscape.

Tuesday, April 14, 2015

One of the things about computing that hasn't changed much over the decades is that it is often best to learn by doing. Many of the original computing pioneers were hobbyists, who built their own computers for themselves, programmed them for fun and for mental stimulation, and who traded notes about what they were doing, what worked and what didn't, and what they had learned along the way.

So along comes Edward Capriolo, who has entertained himself over the winter writing his own NoSQL data store, and has entertained and educated us, along the way, with his series of blog posts:

Sunday, April 12, 2015

A statement by Weir, Hart, Kreutzmann and Lesh said “it has become clear to us that we first need to return to our beginnings, where we first said hello — to each other and to all of you. And so it is that we have decided to plug in for two additional shows on June 27 and 28 at Levi’s Stadium in Santa Clara, California — just a dozen miles south of where Dana Morgan’s Music Store once stood.”

Fans wanting tickets for the two shows, priced at $59.50-$199.50, can sign up for an online ticket lottery that the band installed on its website, www.dead50.net. The sign-up will be in place through April 14 and those who place requests — with a maximum of four tickets per person, per show — will learn if they have received tickets on April 19, according to the band’s website. The announcement said 65,000 tickets would be sold for each show.

Saturday, April 11, 2015

I was fascinated by the subject matter; I love to read historical fiction; I love to be swept away into another world.

I just didn't expect that it would be so much effort to read this book.

Griffith decided to aim for a high level of authenticity, which is wonderful, and is one of the reasons I wanted so much to read Hild. Griffith is thorough in her descriptions, accurate in her dialogue, and detailed in her choice of things like the names of the characters and the vocabulary used to describe the things they are doing in their day to day lives.

And, as this book is set one thousand five hundred years ago, her decision means that there is a considerable amount of distance between the characters, the objects, the events, and the language of that time, and of this one.

But that means you end up with a book which has a Dictionary and a Guide to Pronunciation at the end, and is packed with passages like this:

The high men of the isle gathered to seek favour and pay homage to Edwin and his new queen. Every evening, arriving at the beat of a drum or the ripple of a lyre, a handful of brightly cloaked men, wearing enough gold to dazzle a jay, would swing into the hall and bend their proud heads to the high table. Bryneich from the north, with their short hair, red mouths, and enamelled brooches, under Coledauc king -- who bowed to Hild and gave best wishes from Prince Morcant. The piglet, Hild remembered. Men from Rheged, under Rhoedd the Lesser, Rhoedd's sister-son and little Uinniau's older brother, styling himself prince and bearing gifts from Rhoedd for the king and queen -- and a beautiful double pin inlaid with garnet from the princess Rhianmelldt for the princess Hild. Coelgar, returned from Lindsey, with half a dozen Lindsey thegns at his back and a kinglike bearing. And Dunod, lord of Craven, whom some called king.

Do you see what I mean? It's somehow wonderful and exhausting at the same time.

Eventually you become, if not fluent, at least competent in the language, so when every other sentence talks about a "seax", or a "gesith", or a "wealh", or a "gemaecce", or a "cyrtel", or a "haegtes", or an "aethling", you can start, slowly, to follow the underlying story and events.

But the overall effect, although brilliant, fascinating, and immersive, is also jarring, otherworldly, and uninviting.

Friday, April 10, 2015

Krebs points to a superb, long, detailed, in-depth analysis at the CitizenLab website: China’s Great Cannon.

Several previous technical reports have suggested that the Great Firewall of China orchestrated these attacks by injecting malicious Javascript into Baidu connections. This post describes our analysis of the attack, which we were able to observe until April 8, 2015.

We show that, while the attack infrastructure is co-located with the Great Firewall, the attack was carried out by a separate offensive system, with different capabilities and design, that we term the “Great Cannon.” The Great Cannon is not simply an extension of the Great Firewall, but a distinct attack tool that hijacks traffic to (or presumably from) individual IP addresses, and can arbitrarily replace unencrypted content as a man-in-the-middle.

The operational deployment of the Great Cannon represents a significant escalation in state-level information control: the normalization of widespread use of an attack tool to enforce censorship by weaponizing users. Specifically, the Cannon manipulates the traffic of “bystander” systems outside China, silently programming their browsers to create a massive DDoS attack. While employed for a highly visible attack in this case, the Great Cannon clearly has the capability for use in a manner similar to the NSA’s QUANTUM system, affording China the opportunity to deliver exploits targeting any foreign computer that communicates with any China-based website not fully utilizing HTTPS.

There's lots to dig into here. Thanks, CitizenLab, for sharing what you've learned.

California’s water-rights system is almost incomprehensibly complicated, but the result is that many farms that have depended for decades on an allotment of water from the state supply — their share of a public commodity — are now receiving nothing. Not a drop. Because there’s not enough to go around.

We meet in a little café and the two of them offer me a crash course in California Water 101. Even in non-drought years the logistics are complex. Snowpack runoff is captured in reservoirs. Rivers and lakes are dammed. Canals snake across the state. Some water is managed at the federal level, some at the state. There are 500 public water districts, each with local ordinances. There are senior water rights, junior rights, riparian rights.

Part of the problem is that none of these articles were co-written by a Liberian scientist. The investigators collected their samples, returned home and published the startling results in European medical journals. Few Liberians were then trained in laboratory or epidemiological methods. Even today, downloading one of the papers would cost a physician here $45, about half a week’s salary.

Falconry essentially is the art of managing a falcon’s appetite, he said. “You’re standing here on the beach, when the falcon starts disappearing around the back of the hotel, you’ve now lost visual contact, your communication is gone, your mobile signal is gone. If the bird now sees a pigeon across the road, what’s stopping the bird from deciding to chase that pigeon? Nothing really, other than the bird’s loyalty to fly around the building and seek comfort in seeing Pete again. In an urban environment, the margin for error is huge, you’re radically increasing the possibility of something going wrong.”

So at first What a Terrible World, What a Beautiful World freaked me out a little bit, because it's considerably different from The King Is Dead. But over the last few months I've grown to love WATWWABW at least as much as TKID, if not more.

Besides just their music, one of the interesting things about The Decemberists is how much people like to talk about them as a way of talking about the world at large. This is true of many popular artists, but it is particularly true about The Decemberists, perhaps because their songs get people thinking about larger topics.

That relationship between bands or singers and their audience, it's kind of a funny relationship and abusive in its own right, going both ways. I shouldn't say abusive, but it can be antagonistic. I think that it's an odd relationship, and it's just that particular singer trying to come to terms with that aspect of it. Having an audience, you may want to continue doing things on your own terms, but that becomes more challenging when there are expectations. And audiences have more of a voice than ever with the advent of the Internet.

Other music listeners might ask if bands of the Decemberists’ vintage can change enough to feel pertinent in 2015. A decade ago, music blogs, film and TV music supervisors, Pitchfork, and other new media outlets boosted “indie” to a rare visibility. Now, many of those acts are returning from long absences to quite an altered atmosphere.

Wilson goes on to explain why he uses the powerful term "racist" in this situation:

Few of them claim to be fighting any kind of battle against pop anymore—fans are almost always worse than artists on that count. But this decade has also seen a more widespread suspicion and critique of the workings of social privilege, and “indie” has a problem there—because its creators and listeners seem so disproportionately white, male, and upper-middle-class.

Later, Wilson more directly skewers The Decemberists for what he sees as their failings:

Likewise I am a bit skeptical that without “indie,” the Decemberists could even exist. If there were then still a call for a post-modern folk-rock Gilbert and Sullivan, it would have to have more of the courage of its strangeness. The band’s hiatus has done it some good, and the songwriting is more grounded on this year’s What a Terrible World, What a Beautiful World. But I still find Meloy’s unrelenting streams of conceits wearying, like a prog concept album from 1975 without even the gonzo musicianship to liven up the occasion.

More than any other band, they bring me back to the self-regarding turn that America made in the 2000s—the post-9/11 world-wariness and self-soothing. It would be too much to say that’s what made it an ideal period for “indie.” But when I listen to the Decemberists, I’m tempted.

I am, indeed, white, male, and upper-middle-class. So, guilty as charged. But does that mean I'm somehow committing a social offense by being a Decemberists fan?

I'll have to spend more time listening to their music before I can come to a more considered opinion about whether they are letting us down.

But it also seems like Wilson is asking Meloy and company to fight Wilson's battles, which is unfair. As Meloy says,

I just like stories. I like people telling stories.

Seems fair enough, to me. I'll keep listening, and hopefully I'll enjoy going to their show and meeting the people I meet there.

Sometimes art can just be entertainment, after all; it doesn't always have to change the world. That's a lot to ask, of anybody.

Monday, April 6, 2015

Q: Does Git last forever, or do you foresee another revision control system in another 10 years? Will you be the one to write it?

Torvalds: I'm not going to be the one writing it, no. And maybe we'll see something new in ten years, but I guarantee that it will be pretty "git-like." It's not like git got everything right, but it got all the really basic issues right in a way that no other SCM had ever done before.

The files in the /etc/xinetd.d/ directory contains the configuration files for each service managed by xinetd and the names of the files correlate to the service. As with xinetd.conf, this file is read only when the xinetd service is started. For any changes to take effect, the administrator must restart the xinetd service.

I was a little bit confused about "the names of the files correlate to the service", so I read further in the RedHat docs:

service — Defines the service name, usually one listed in the /etc/services file.

I looked in my /etc/xinetd.d directory, and sure enough I had some existing files:

$ ls /etc/xinetd.d
chargen daytime discard echo time

Sounds pretty good, so I created two new files:

$ sudo vim /etc/xinetd.d/server.1 /etc/xinetd.d/server.2

I put in some simple configuration, restarted xinetd, and ...

...

... nothing happened.

After a bunch of flailing around, I found /var/log/syslog, where xinetd reported that it loaded the configuration files /etc/xinetd.d/chargen, /etc/xinetd.d/daytime, /etc/xinetd.d/discard, /etc/xinetd.d/echo, and /etc/xinetd.d/time.

There weren't any other error messages or complaints.

It just quietly omitted my service configuration files.

A colleague, stopping by (mostly to stop the full-volume stream of profanities I was directing at my computer screen), looked over my shoulder and said:

Hmm.. I wonder if period is an illegal character in a service name?

WHAT?

Well, sure enough, I renamed those two files from "server.1" and "server.2" to "server1" and "server2", and restarted xinetd, ...

I guess Real Linux System Administrators don't need things like this documented, and don't need their tools to print syntax error messages when they make simple syntax errors in naming their network services.

I'll go back to getting useful work done. Bummer about those 90 minutes of my life; I could have maybe used them for something useful.

Sunday, April 5, 2015

Roads & Kingdoms is an independent journal of food, politics, travel and culture. It was recently voted the Gold Winner for Best Travel Journalism Site by the Society of American Travel Writers. The magazine first launched in Myanmar as a Tumblr that became a home for reports on everything from Burmese civil war to dissident MCs to the perils of rancid crab. R&K is now a fulltime digital magazine based in New York and Barcelona, publishing longform dispatches, interviews and global ephemera daily.

I love it all: I love the longform dispatches; I love the interviews; I love the global ephemera.

Everything about the site is well-done. The writing is clear and vivid (and well-edited). The photography and presentation is marvelous. And the topics are both fascinating and relevant.

I'm not sure where they've found the funding for the site, which is blissfully short on advertising and other annoyances. Perhaps this will change, as all great websites seem to eventually succumb to reality.

But for now, I'll keep loving Roads & Kingdoms, and I'll keep reading them as often as I can.

Wednesday, April 1, 2015

We are currently experiencing the largest DDoS (distributed denial of service) attack in github.com's history. The attack began around 2AM UTC on Thursday, March 26, and involves a wide combination of attack vectors. These include every vector we've seen in previous attacks as well as some sophisticated new techniques that use the web browsers of unsuspecting, uninvolved people to flood github.com with high levels of traffic. Based on reports we've received, we believe the intent of this attack is to convince us to remove a specific class of content.

the two GitHub pages are constantly loaded and reloaded by millions of computer users inside and outside of China, an endless loop that left unmitigated outages not just on the two targeted pages but throughout GitHub's entire network. Exhibit A in the case in which China is involved are the two specific GitHub pages targeted: one hosts anti-censorship service GreatFire.org while the other hosts a mirror site of The New York Times' Chinese edition. The targets suggest the attackers are sympathetic to the vast censorship apparatus known as the Great Firewall of China.

One website the user visits loads a javascript from a server in China, for example the Badiu Analytics script that often is used by web admins to track visitor statistics (much like Google Analytics).

The web browser's request for the Baidu javascript is detected by the Chinese passive infrastructure.

A fake response is sent out from within China instead of the actual Baidu Analytics script. This fake response is a malicious javascript that tells the user's browser to continuously reload two specific pages on GitHub.com.

On March 17th 2015, our websites and partner websites came under a DDoS attack. We had never been subjected to an attack of this magnitude before. This attack was unusual in nature as we discovered that the Chinese authorities were steering millions of unsuspecting internet users worldwide to launch the attack. We believe this is a major cyber-security and economic threat for the people of China.

GreatFire.org’s mirroring services provide unrestricted access within China to a range of websites, including itself and the Chinese language version of The New York Times, which has been regularly blocked in China. Some of the others are Deutsche Welle, BBC News, China Digital Times, Google.com, and Boxun, a Chinese-language news website. GreatFire.org says it does not mirror The Wall Street Journal. GreatFire.org works directly with some, but not all, of the websites it mirrors.

GreatFire.org is partly funded by Open Technology Fund, a United States government-financed initiative under Radio Free Asia. Last year it provided $114,000 in funding, according to its website. Mr. Smith declined to comment on any financial backing.

What is happening here is pretty clear now:
A certain device at the border of China’s inner network and the Internet has hijacked the HTTP connections went into China, replaced some javascript files from Baidu with malicious ones that would load

Github’s whole site uses HTTPS encryption, so when a Chinese netizen visits content hosted on the site, Chinese censors can only see that the user is visiting github.com, but not the full URL address within GitHub. So China can't selectively block just some content on GitHub without blocking the entire site.

Instead of completely controlling a network node as in a man-in-the-middle attack, the attacker only has regular access to the communication channel, which allows him to read the traffic and insert new messages, but not to modify or delete messages sent by other participants. The attacker relies on a timing advantage to make sure that the response he sends to the request of a victim arrives before the legitimate response.

The attackers have implemented a sneaky mechanism that allows them to manipulate a part of the “legitimate traffic” from inside and outside China to launch and steer Denial of Service attacks against Cloudfront and the Greatfire.org's anti censorship project.

Because the overall internet is too important to block, and because some sites are necessary (like Github) there are always holes in the system. Add in a useful dose of encryption (yay!) and the ability to control everything that's read in one particular country becomes increasingly difficult. You might hope the response would be to give up attempts to censor, but China isn't likely to give up just like that. So, instead, it's basically trying to censor the global internet, by launching a high powered attack on the site that is the problem, while basically saying "get rid of these projects and we'll stop the attack."

Recently, KrebsOnSecurity heard from Michael Kasper, a 35-year-old reader who tried to obtain a copy of his most recent tax transcript with the Internal Revenue Service (IRS). Kasper said he sought the transcript after trying to file his taxes through the desktop version of TurboTax, and being informed by TurboTax that the IRS had rejected the request because his return had already been filed.

In this increasingly common crime, thieves steal or purchase Social Security numbers and other data on consumers, and then electronically file fraudulent tax returns claiming a large refund. The thieves instruct the IRS to send the refund to a bank account that is tied to a prepaid debit card, which the fraudster can then use to withdraw cash at an ATM

“We were later notified that the breach was much deeper than originally suspected, which included all of our beneficiaries, their personal information, my life insurance policy, 401-K stuff, and our taxes,” Doug said. “My sister-in-law is an accountant, so I raced to her and asked her to help us file our taxes immediately. She pushed them through quickly but the IRS came back and said someone had already filed our taxes a few days before us.”

Doug has since spent many hours filling out countless forms with a variety of organizations, including the Federal Trade Commission, the FBI, the local police department, and of course the Internal Revenue Service.

“During this tax season, Intuit and some states have seen an increase in suspicious filings and attempts by criminals to use stolen identity information to file fraudulent state tax returns and claim tax refunds,” the company said in a statement.

States allow unlinked returns because most taxpayers owe taxes at the federal level but are due refunds from their state. Thus, unlinked returns allow taxpayers who owe money to the IRS to pay some or all of that off with state refund money.

It's great of Krebs to keep a light on this subject, because it's complex and frustrating.

Happily, slow progress appears to be being made, as Krebs covers in his latest column:

those involved in tax refund fraud shifted more of their activities away from the Internal Revenue Service and toward state tax filings. This shift is broadly reflected in discussions on several fraud forums from 2014, in which members lament the apparent introduction of new fraud “filters” by the IRS that reportedly made perpetrating this crime at the federal level more challenging for some scammers.

...

According to a January 2015 GAO report (PDF), the IRS estimated it prevented $24.2 billion in fraudulent identity theft refunds in 2013. Unfortunately, the IRS also paid $5.8 billion that year for refund requests later determined to be fraud. The GAO noted that because of the difficulties in knowing the amount of undetected fraud, the actual amount could far exceed those estimates.