The idea behind big data is that it allows context-free inference: Preponderance of data can compensate for lack of causality determination or subject matter expertise. There are limited circumstances for which that can work. My favorite example is in financial markets trading. Think of sphexishness from Hofstadter’s Godel, Escher, Bach book. In that situation, it doesn’t matter why one thing triggers another, as much as that it does, and in a highly predictable way, for a long enough interval of days, weeks or months to be useful for profitable trading.

Machine learning is great for applications for which the cost of spurious correlation is very low.

Crypto.cat is an encrypted chat application developed by Nadim Kobeissi. Cryptocat received a lot of publicity in the technology press at the time of its initial release in 2012. Some of it was hyperbole. There was even talk of integrating Cryptocat with the Tor browser, or with Tails.

I was browsing through Github issue comments looking for something else when I found a most surprising Cryptocat feature discussion! All the way back in March of 2013, Nadim was planning for secure messaging among Internet of Things things, specifically, refrigerators!

Airbnb is widely touted as a bastion of mobile app-enabled egalitarianism, however, a report issued by New York State Attorney General Eric Schneiderman [PDF] indicates quite the opposite to be true. Commercial enterprises use Airbnb to operate million dollar businesses. For example, a single commercial user made $6.8 million in less than five years in New York City.

Airbnb In The City

Widespread illegality across New York City listings was documented on Airbnb’s website. 72% of Airbnb reservations violated New York law.

Tenaments or flop houses? Illegal hostels in NYC

Airbnb units serve as illegal hostels, despite New York law prohibiting commercial enterprises from operating hostels. In 2013, multiple, unrelated guests shared the same unit on the same night, as they would in a hostel, with one top listing average 13 reservations per unit per night.

Bookings in three Manhattan neighborhoods – Greenwich Village/SoHo, Chelsea/Hell’s Kitchen and Lower East Side/Chinatown – accounted for more than 40% of hosts’ revenue. All reservations in Queens, the Bronx and Staten Island combined brought in less than 3% of the New York City total.

According to a 2014 Pew Research survey about public perceptions of privacy, people believe landline phones are the “most secure”.

Americans lack confidence that they have control over their personal information. That concern applies to everyday communications and to collection of their information, by government and corporations.
Network World had a good article, summarizing the results:

Feeling like you live in Orwell’s 1984 is not a good thing, but hopefully this feeling of discontent will urge people into taking action to better control their privacy online. Don’t hold your breath for that, though.

Testifying at a U.S. Senate Armed Services Committee hearing, Henry Kissinger, an ardent Cold Warrior who was Richard Nixon’s main foreign policy advisor, stopped short of endorsing a call by the committee chairman, Republican Senator John McCain of Arizona, to provide defensive weapons to Ukraine’s military as it battles Russian-backed separatists. “I’m uneasy about beginning a process of military engagement without knowing where it will lead us and what we’ll do to sustain it,” the 91-year-old Mr. Kissinger said.

Mr. Gorbachev was one of the architects of the peaceful dismantling of communist rule in Eastern Europe. He has increasingly sounded ominous warnings about the path events are taking in Ukraine. Earlier in January, he reportedly told a German magazine that he feared a nuclear confrontation was possible if things begin to escalate. “The statements and propaganda on both sides make me fear the worst. If anyone loses their nerve in this charged atmosphere, we will not survive the next few years,” he said. “I do not say such things lightly. I am a man with a conscience. But that’s how it is. I’m really extremely worried.”

United States vaccination requirements have been in the news, particularly following what the Centers for Disease Control and Prevention (CDC) has described as “a large, ongoing multi-state outbreak linked to an amusement park in California.” As such, many questions have arisen as to the laws regarding vaccination and immunization in the United States. The answers to these questions are not as simple as one might think.

For Bitcoin only? No!

The blockchain might be applicable for more than cryptocurrencies! IBM has figured out a way to use the blockchain for keeping track of Internet of Thing things. This is just a single page from the recent IBM whitepaper. Scott Bell is my Twitter friend. I am EllieAsksWhy.

More specifically, ADEPT is an IBM system developed in partnership with Samsung that builds a distributed network of devices – a decentralized Internet of Things. ADEPT is an acronym for Autonomous Decentralized Peer-to-Peer Telemetry. It uses the blockchain, with a mix of proof-of-work and proof-of-stake, to enable secure transactions.

Supplysideliberal is a professor of economics. I am not, nevertheless, I think this is a bad idea, as it will undermine trust:

Give the central bank the authority to make interest rates negative, including making the interest rate on paper currency negative when necessary, by using the fact that a paper dollar is no longer guaranteed to be worth the same amount as an electronic dollar.

moving to hold down the value of the Swiss franc amid turmoil in global currency markets and expectations that deflation is at hand. The bank acted as the crisis in Russia and plummeting oil prices have caused a run on emerging market currencies.

This means that Switzerland will be charging a fee to anyone that wants to keep Swiss francs on deposit. This safe haven currency will now charge for the privilege.

Hi there, I have heard that some material related to the GamerGate controversy in the gaming world has been removed. Can you give me any details on what your polices are on removing material and who might have requested this? Thanks a lot

<p><a class="tumblr_blog" href="http://blog.archive.is/post/98379557621/hi-there-i-have-heard-that-some-material-related">archive-is</a>:</p>
<blockquote><p>Hi.</p>
<p>I have never heard about “GamerGate” controversy and I have not received any related deletion requests.</p></blockquote><p><p><strong>Never heard of GamerGate?</strong> Wow, fortunate soul!</p></p>

As for being disease-free, Ebola can hide in reservoirs in the body. It has been found in the urine and semen of people from one to two months following recovery. Eventually the virus does go away. Ebola victims who survive – the virus kills 60 to 90 percent of those it infects – first must spend weeks, if not months, regaining their strength and body weight following the ravages of Ebola … and ought to emerge from their struggle with immune resistance against future Ebola infection.

Visa announced a roadmap for U.S. adoption of EMV chip cards and NFC-enabled mobile payment devices…adding a layer of safety to transactions, through the use of dynamic authentication… There’s a lot of confusion around the myth that EMV means “chip-and-PIN.” It doesn’t in many countries, including the U.S. That’s because we can rely on online processing…transmitted in real-time to the issuer for approval. With that in place, there’s no need for the offline authentication that was the genesis of chip-and-PIN. Mercator Advisory Group recently wrote about this…The key is to implement a streamlined, online-only version of EMV…

I C Q

This image reminds me of a mantrap. I was reading about industrial strength security measures last week. That’s how I learned about mantraps, also called “security vestibules”. Mantraps can satisfy both physical AND electronic security requirements. They have nothing to do with capturing a husband. They function more like the carnivorous Venus fly trap, however, Venus fly traps are a final destination. Mantraps are not.

The most recognizable analogy is an airlock. Airlocks are familiar from science fiction movies. The genuine article was used by NASA and former Soviet manned earth orbit and lunar missions. Terrestrial airlocks are uncommon. The only uses that I can think of are high risk bio-containment facilities and perhaps for hyperbaric chambers.

To a large extent, the mainframe’s longevity is a result of two major architectural innovations introduced with S/360. The first was the notion of a family of computers, from low to high performance, all based on the same instruction set which allowed customers to upgrade to larger systems as well as to future models without having to rewrite their applications.

The second was OS/360, a common operating system that supported the various members of the S/360 family except for the smaller ones which ran a subset with more limited capabilities. Today’s z/Architecture and z/OS are direct descendants of the original S/360 and OS/360. —Irving Wladawsky-Berger, The Wall Street Journalibm.com/mainframe50

If that seems too much like native advertising (Irving Wladawsky-Berger is such an IBM fanboi ;O) then pay a visit to Seeking Alpha with me. They live and breathe capitalism over there. Regarding IBM in general and the mainframe in particular, I found A Moat Full of Immortal Dinosaurs. Warren Buffet looks for “economic castles protected by unbreachable moats.” Good moats protect businesses from their competitors. They are made of brand power, scale/ cost advantages or intellectual property. The mainframe business is IBM’s very strong moat.

All the code that’s fit to print!

WordPress does something similar, “If you are reading this, contact us. We’re hiring!” WordPress doesn’t have the surprisingly lush New York Times typography themed ASCII art though. Limeduck even has a doppelganger of sorts!

@limeduck still reading hardcovers? I hear those who can’t write, read.

Secret Neoreactionary Island

Duck Enlightenment a.k.a. Jokeocracy is my most recent duck discovery on Twitter, though he seems to have found an online home. He is a humorous version of the Dark Enlightenment ideology that I otherwise find so disturbing. Duck Enlightenment is not a parody account, so there is an edge. He is a more youthful sort than LimeDuck. It is impossible to see that sweet little duck body, suitably darker hued, with petite nubby horns, and not feel a rush of empathetic loving kindness.

Any data scientist worth their salary will tell you that you should start with a question, NOT the data. Unfortunately, data hackathons often lack clear problem definitions. Most companies think that if you can just get hackers, pizza, and data together in a room, magic will happen.

This reminds me of Windows XP Solitaire whenever I won a game. I think the trajectory of the bouncing playing cards was a simple exponential decay in two dimensions, but I liked it, a lot! I like this too. Click the URL and watch it animate!

Network diagrams are a popular way of visualizing social and corporate relationships. Network theory has been used to model telecommunications performance and especially, the Internet. Communications networks increase in value as the number of connections increases. Metcalfe’s Law attempts to quantify the increased value.

Optimizing Metcalfe’s Law

For a network with n members, Metcalfe’s Law posits that the total value of that network is proportional to n * (n-1). Metcalfe’s Law as applied to the Internet, and even to the telephone network, is only valid if all connections have equal value. This is incorrect. Some internet connections are hardly used and contribute limited value. Of course, there are reasons to connect everyone that are not based on monetary value! Rural electrification is an example.

Andrew Odlyzko’s article about Metcalfe’s Law (IEEE Spectrum, 2006) was written with a keen awareness of the 2000 dotcom bubble. Odlyzko demonstrated how Metcalfe’s Law’s applicability could be limited by the equal value assumption, among others. I read it, and wondered: What is the Internet’s optimal number of nodes and connections? When did the value of a larger Internet network start diminishing?

At some point, ISPs (Internet Service Providers) stopped charging users for access, as the business of delivering content became more valuable than providing greater network connectivity. AOL charged for service until 2002 or so.

I thought it would be helpful to begin with a timeline of Internet growth, by number of sites connected and corresponding events, as a starting point for determining incremental value. I searched for a streamlined history, but the best that I could find is provided by The Computer History Museum, and it isn’t quite linear. It also has a lot of technical detail that isn’t relevant for verifying Metcalfe’s Law. I decided to construct a timeline of dates and nodes, from which connectivity can be determined. I am writing this partly for myself, for reference purposes. (I don’t know how to value connectivity, not yet.)

In the beginning

In the beginning, the Internet had only two nodes. It was called the ARPANET.

Two-node ARPANET

One node was a computer at UCLA in Los Angeles, run by researchers known as the Network Working Group. Their job was to develop a protocol, which eventually became known as the collection of programs comprising NCP, or Network Control Protocol. (IMP is Interface Message Processor.)

The other node on the two-node Internet was a computer at the Stanford Research Institute (SRI) in Palo Alto.

On October 29, 1969, the first Internet transmission occurred, i.e. the first ever login to a remote host via the ARPANET. It was documented in a handwritten log.

First connection to a remote host via the ARPANET on October 29, 1969

In 1970, two more nodes were added to the ARPANET, at the University of California, Santa Barbara. This was done in order to deal with the problem of screen refresh over the net.

1971

The ARPANET began the year with 14 nodes in operation.

14 node ARPANET of 1971

UCLA’s Network Working Group completed the Telnet protocol and made progress on the file transfer protocol (FTP) standard. By the end of the year, the ARPANET had 19 nodes.

September 1973

Two years later, the ARPANET had grown. 30 institutions were connected to the network. Users included corporations and consulting firms (e.g. Xerox PARC and MITRE) as well as government sites like NASA’s Ames Research Laboratories, the National Bureau of Standards, and the U.S. Air Force’s research facilities. Packet-switching was found to be a viable technology.

Other network connectivity programs were developed, such as Packet Radio sites and a satellite connection enabling sites in Norway and the UK to connect. ARPANET, PRnet (Packet Radio), and SATNET all had different interfaces, packet sizes, conventions and transmission rates. Linking them together was a problem. In response, Robert Kahn and Vint Cerf designed a net-to-net connection protocol. In September 1973, they presented their first paper on the new Transmission Control Protocol (TCP).

30 node ARPANET of September 1973

At Xerox PARC, Bob Metcalfe (remember Metcalfe’s Law?) was working on a wire-based system for Local Area Networks (LANs). It became Ethernet.

January 1975

The ARPANET geographical map now had 61 nodes. Daily traffic was about 3 million packets.

ARPANET with 61 nodes in 1975

January 1983

The Internet in 1983

The Domain Name System (DNS) was developed and recommended for the user@host.domain addressing system. The number of computers connected via these hosts was much larger. Growth accelerated with the commercialization of Metcalfe’s Ethernet.

1984

DNS was introduced across the Internet, with the domains of .gov, .mil, .edu, .org, .net, and .com.

The 56 Kbps backbone between NSF centers led to the creation of regional feeder networks. With the backbone, these regionals started to build a hub and spoke infrastructure.

1986

In the beginning of 1986, there were 2000 networks.

USENET newsgroups such as ‘alt.sex’ and ‘alt.drugs’ were still not allowed.

Internet 1987

1987

The NSF, realizing the rate and commercial significance of the growth of the Internet, signed an agreement with Merit Networks, IBM and MCI. The NSF started to implement its T1 backbone between super-computing centers.

USENET newsgroups became available for the PC.

In early 1987, the number of hosts passed 10,000. Network management started to become a major issue. SNMP was chosen as a protocol for remote management between routers.

NSF Connectivity Map, April 1987

1988

As of January 1988, there were 30,000 hosts. The upgrade of the NSF backbone to T1 was completed. The Internet started to become more international with the connection of Canada, Denmark, Finland, France, Iceland, Norway and Sweden.

Later in 1988, the Morris worm burrowed into 6,000 of the 60,000 hosts now on the network. DARPA formed the Computer Emergency Response Team (CERT) to deal with future incidents.

1989

MCI Mail and CompuServe connected their commercial email systems to the Internet and each other for the first time. This was the start of commercial Internet services in the United States.

In Switzerland at CERN, Tim Berners-Lee proposed a hypertext system that would run across the Internet on different operating systems. This was the World Wide Web. I stopped here.

Light, sweet crude is the most desirable grade of crude oil because it requires minimal refining while producing the most gasoline. This chart is useful when considering geopolitical risk and commodities prices, as it illustrates where the “best” oil is.

Oil types by country (click image to view full-sized)

The chart was produced by the U.S. Department of Energy in 2012. Perhaps that is why West Texas Intermediate (WTI) crude is not listed, as the U.S. was not an oil exporting nation until recently, under President Obama’s administration, although we are not a net oil exporter.

OPEC

The Organization of Petroleum Exporting Countries has had varying levels of power since its founding in 1960 and heyday in the 1970s. Saudi Arabia has always been the controlling producer in the cartel. OPEC’s objective is to control oil price by either freezing production or cutting production.

Iran is an OPEC member. Due to the U.S.-Iran nuclear deal, finalized in 2015 and effective as of early 2016, sanctions on Iranian products and services have been lifted. This includes Iran’s heavy, sulfurous crude oil. Unfortunately for Iran, crude oil won’t be quite the source of hard currency revenue that it might have been in the past. Saudi Arabia will freeze production at January 2016 levels, with other OPEC members expected to fall in line:

Not surprisingly, Iran is unwilling to play ball, with oil minister Bijan Zanganeh last week saying it was “a joke” that countries pumping more than 10 million b/d should expect Iran to freeze its production at the low level imposed by the sanctions.

OPEC’s ability to control market prices has been diminished by the presence of U.S. shale oil producers as an additional global supply source. This was not an OPEC concern until U.S. shale production increased for U.S. domestic use (lessening dependence on foreign oil) and even some limited exports. OPEC is likely to stick to its resolve to freeze but not cut production, as Riyadh could survive $20/barrel oil according to Platts.

Cheap oil trends

Given prominent policy-maker concern about anthropogenic climate change and reducing dependence on fossil fuels, it is odd that the U.S. government would decide to allow greater levels of extraction (e.g. from strategic reserves) as well as building out infrastructure for more production now, for the first time in 40 years. Increased supplies make gasoline less, not more, expensive, thus encouraging greater use.

The plummeting price of oil has had unsurprising consequences: SUVs and other fuel-inefficient vehicles have surged in popularity. Hybrids are being put up for sale. There is a possible impact on high-end exotic electric vehicles such as those manufactured by Tesla. (Since Tesla vehicle owners are motivated by factors other than cost of acquisition or use, the effect of lower gasoline prices is unknown.)

Many stations are eliminating premium gasoline. That also means 89 octane is gone at those stations, too, because you can’t have the 89 without the 93. At most fuel pumps, 87 and 93 are mixed just seconds before they reach your car to make 89.

Unavailability of higher-grade fuel will cause problems for the many new vehicles which require it for optimal performance.

Texas almost joined OPEC

The Texas state organization most responsible for bringing crude oil to market as refined gasoline was the Texas Railroad Commission. It was created in 1891 and empowered to restrict production to avoid waste, but its role expanded over time. The Commission set global oil prices until it was dethroned by OPEC in 1973.

OPEC’s influence was waning by the 1980s. In 1988, a member of the Commission decided that Texas had some common objectives with OPEC. The commissioner went to Vienna to discuss cutting Texas oil production, in order to raise prices and avoid financial waste, if not to express solidarity with OPEC members. His actions were viewed with skepticism, or worse, by many.

With daily production of about 2 million barrels, Texas pumps more oil than all but eight nations and often sees itself more in league with exporters such as Saudi Arabia and other OPEC members than with net consumers such as Illinois and most other states.

This image was originally developed as the focal point of the Math Awareness Month poster of April 2000. The theme was “Math Spans All Dimensions”. It was too lovely to be retired, and appeared again at TFBCON2003.

According to the artist, Brown University professor Thomas Banchoff, it is suffused with joy, just as I had hoped!

Energy market pricing behavior seems contrary to the relationship between supply and demand. The oddly behaving RIN market is an intermediate factor that influences gasoline prices for automobiles. RIN (Renewable Identification Numbers) should be decreasing. Instead, they are too high.

Bio-fuel pricing anomaly

RIN establish compliance with standards for non-fossil fuel usage, specifically, for corn-based ethanol as a blend in gasoline. In 2007, legislation was passed to encourage greater use of ethanol. The percentage requirement of ethanol is set by the EPA. It increases annually, and is calculated at an aggregate level, measured volumetrically, over all U.S. domestic consumption.

My favorite energy blog, Platt’s Oil Barrel, featured a guest post* by former Special Assistant to President Obama and Senior Director for Energy and Climate Change of the National Security Council Jason Bordoff, explaining anomalous RIN price behavior, and what the EPA is doing about it. He noted two reasons for the seemingly anomalous pricing.

Hitting the blend wall

Renewable Fuel Standards (RFS) were revised in 2007, based on the assumption that gasoline usage would increase over time. In fact, it has not done so, not consistently. Instead, it decreased during 2011-2013, yet the schedule of increasing amounts of ethanol has remained, as legislated. As a result, according to Bordoff, we are now hitting the “blend wall”, when blenders physically cannot put enough ethanol into the gas supply to comply with RFS law.

Bordoff identified a second reason:

broad-based skepticism in the market that EPA will use its waiver authority to avoid the blend wall—even though EPA just went to unusual lengths to signal precisely that it will.

Federal Reserve v. EPA: Powers and purpose

The bio-fuel situation bears an odd resemblance to the rational expectations based logic of monetary policy. It is difficult for the Federal Reserve to effectively signal to markets, e.g. the anticipated (and appropriate!) end of quantitative easing. The Federal Reserve System has taken measures to increase transparency. Fed Governors Bernanke and Yellen hold scheduled press conferences. Bernanke was the first Federal Reserve governor to do so. The Fed was audited by the GAO in 2012. Federal Open Markets Committee (FOMC) meeting notes are published and posted online.

Despite all of the above, the “job creators” aren’t investing, and the Fed is now contemplating QE4.

Let’s get back to bio-fuels!

RIN prices were intended to incentivize the build-out of ethanol-blended fuel infrastructure. I tried, but still can’t quite figure out how RINs work. Are RINs a levy on petroleum refiners? Go have a look at this post about RINsanity from Ethanol Producers. I do understand enough to believe that it is inappropriate to expect the EPA to function like the fuel market version of the Federal Reserve. The Environmental Protection Agency is a regulatory authority. It does not have a fuel market oversight mandate, nor does it have the means to effectively influence fuel markets.

Economic Model of Consumer Behavior In the Time of Tranquility

Behavioral economics is NOT the solution to everything. It seems arrogant to expect it to be. Building out biofuel infrastructure will only happen if manufacturers and consumers want it. The EPA is good for enforcement of specific criminal acts of environmental waste and endangerment. The EPA is good for setting standards that provide guidance as part of normal day-to-day life and business. The EPA shouldn’t be responsible for implementing massive, sweeping public policy that spans manufacturing, transportation and consumer necessities!

More than one-third of our corn crop is used to feed livestock. Another 13% is exported, much of it to feed livestock as well. Another 40% is used to produce ethanol. The remainder goes toward food and beverage production.

There isn’t as much corn for animal feed. Next, it becomes uneconomic for farmers to grow organic crops, or anything with higher risk, instead of corn.

The EPA signaled but without any effect. That’s due to a lack of trust, which makes sense, as the EPA is not a market participant. The EPA establishes standards and enforces compliance. They shouldn’t force blends of E10 to E15 if manufacturers say it will damage engines though! If manufacturers, consumers and fuel stations aren’t convinced that the EPA is acting in the public and industry’s best interests, then no amount of so-called incentivizing through RIN prices will be effective.

For newer Porsches only

The EPA can’t force the public to behave irrationally. Irrationally? Yes. Everyone except the ultra wealthy ruling class (say, greater than $1 million average yearly income or greater than $10 mil net worth) is worried about money. Of course no one will want to use E15 fuel if it could ruin his car engine (American Automobile Association, November 2012):

The only vehicles currently approved by automakers to use E15 are flex-fuel models, 2001 model-year and newer Porsches, 2012 model-year and newer GM vehicles and 2013 model-year Ford vehicles.

People won’t worry about the environment if they are afraid of losing their jobs, due to needing a new engine for the vehicle used to get to work. Without that engine, vehicle and job, families will be hungry and homeless. Public transportation is an excellent alternative, but it isn’t available for most of us.

Auto companies should have developed more bio-fuel accommodating automobile engines a long time ago. Most recently, the government has been obsessed with the JOBS Act (general solicitation for hedge funds), gutting Dodd-Frank while adding bulk, the America Invents Act (opaque euphemism for more patents, less invention) and funding not-so-high-tech, so-called social media entrepreneurs and cloud computing, instead of chemistry, physics and engineering research by Midwestern auto companies and big land grant universities of the states of Iowa, Michigan, Texas, New Mexico etc.

Does the EPA classify information too?

Unless one has spent the past year in a sensory deprivation tank, it would be impossible NOT to be aware of the disclosures of former defense contractor Edward Snowden, and a slew of revelations about surveillance. The concept of classified or restricted information is applied in few government contexts. Most U.S. federal, state, county and municipal documents are available for public review, as they are funded with our tax dollars. Thus, I was rather surprised to learn that the EPA issued its 2013 version of bio-fuel related updates, but redacted a key item and resisted Wall Street Journal requests for disclosure.

* I drafted most of this on 13 Oct 2013, so there may have been changes in the interim.

Some years ago, I studied mathematics and statistics. At that time, there was only one statistician among the mathematics department members, maybe the entire Swarthmore College faculty, Gudmund R. Iversen. He was my academic adviser. Professor Iversen was grey, tweedy and Norwegian. He always addressed me as Miss Kesselman, which helped alleviate my shyness at the time.

Lunch with Tufte

Professor Iversen had a group of colleagues, all statisticians from other academic institutions. They would visit Swarthmore to give lunchtime talks, or more typically, late Friday afternoon presentations to mathematical statistics students.

Napoleon’s March to Moscow. Charles J. Minard, 1869

I recall one particular guest statistician. Edward Tufte was on the faculty of Princeton University, and had recently written his first book, The Visual Display of Quantitative Information. The venue was a small private room in Sharples dining hall. I was one of maybe 20 attending.

Tufte was high-strung and slightly fussy, with occasional flashes of humor. He handed out hardback copies of his book, admonishing us “not to dip them in the gravy” from lunch (there was no gravy at lunch). Tufte explained that he had to take out a third mortgage on his house to finance the production and publication of Visual Display. The book was gorgeous. The statistical graphs were unlike anything I had ever seen before. Tufte spoke at length about Charles Minard’s famous map representing Napoleon Bonaparte’s doomed Russian campaign. In the summer of 1812, Napoleon set out for Moscow with 440,000 troops. Only 10,000 returned.

Tufte spoke well. After a mild question and answer session, he retrieved copies of his book from us. I badly wanted to keep mine. For a little more Tuftese see my Chart Art post.

Statistics moves up in the world

During my time at Swarthmore College, statistics was considered a marginal field of study, at best. The current math department chairman, James England, referred to it as “cocktail party math”. Professor Iversen had tenure by the time I arrived, yet he didn’t have an office with the rest of the mathematics department. Instead, he still occupied the same room in the 1st level basement of the engineering building as he had since 1973, and a ten minute walk from the rest of the department. It was an almost windowless room, with woven wool rugs on the floors and hung on the walls, which kept the air warm and dry. Naturally, the furniture was mostly mid-century Scandinavian modern.

Given that background, I was surprised and happy when Professor Iversen became the new department head in 1992! In 1993, the department name changed. Now it is the Swarthmore College Mathematics and Statistics Department. As far as I can tell, Professor Iversen kept his original office even while he was department chair. After twenty years, it was their turn to come to him.

The book grew out of activities supported by what is known as the Radical Statistics group, a twenty-five year old group unknown to me before reading this book. Radical Statistics is “a group of statisticians and others who share a common concern about the political assumptions implicit in the process of compiling and using statistics, and an awareness of the actual and potential misuses of statistics and its techniques.”

There has been so much tumult in bitcoin and crypto currencies over the past few days! Interest and concern extends beyond online communities. Motives vary.

Mining with Windows 7

Decentralized and anonymous

There are two conceptual pillars of trust that uphold bitcoin as being superior to fiat currency. The first is decentralization. The fiat currency of reference is primarily the US dollar, for the time being. Why? Because the $US is the world’s reserve currency, for now. It is highly centralized. As ideological (but not market) confidence in the $US has diminished, the appeal of an apolitical, alternative currency increases, especially one that is a fungible, stable store of value.

The second conceptual pillar of bitcoin is anonymity. US dollars held as cash will be anonymous until one wants to use them for exchange for commercial transactions of size. Bitcoin has some anonymity shortcomings, but there may be tractable remedies. That is a detailed discussion, widely covered elsewhere.

Centralization of bitcoin

All markets are game theoretic. Bitcoin is more transparently so. I really wish we could ask Professor John Nash what he thinks of bitcoin! Nash wrote a pleasant, accessible article that described bitcoin-like currency, titled “Ideal Money” a few years ago.

Bitcoin’s most acute concern right now is loss of decentralization. It is due to the documented, persistent existence of a 51% majority mining pool controlled by gHash.io. gHash is owned and operated by a private entity, cex.io. gHash’s market dominant behavior was noted in March 2014, but the situation was transient, unlike now. See How a mining monopoly can attack bitcoin for a chart of strategies that mining pools can pursue as a function of hash power. As a follow-up, Ed Felten confirmed that Bitcoin mining is now dominated by one pool.

Production and transaction costs

In theory, bitcoin is a perfectly smooth, zero transaction cost medium of exchange. In reality, this is possible but involves a modicum of effort.

Some bitcoin miners and many non-mining users keep their holdings in custody of a clearinghouse such as Mt. Gox or Silk Road. This does generate small transaction costs, for holding user e-wallets. In return, users benefit from the greater convenience in making purchases and sales. The clearinghouse may have an additional appeal to miners, as it offers the option of participating in a shared mining pool. Bitcoin was designed to reward early adopters; as more bitcoins are mined, more computational effort is required.

Home bitcoin mining before ASICs

Mining Bitcoin requires processing power and electricity. In 2010, a PC with an NVIDIA or ATI GPU would have been adequate, but no longer. A new crypto currency-specific manufacturing industry has evolved for bitcoin mining equipment, using FPGAs which are more energy-efficient than graphics processing units. This was further improved by an application-specific integrated circuit (ASIC),

In other words, a chip designed from the ground up for the specific purpose of mining bitcoins. ASIC also represents the theoretical limit on the hardware capabilities of mining equipment.

The mining rigs cost tens of thousands of dollars, and create a high barrier to entry for many miners. The remedy had been to use mining pools in the cloud e.g. Amazon AWS or clearinghouse hardware.

There are costs for miners, but a monetary levy is not the worst of it. The same can be said for non-miners who have others hold their e-wallets. Trust is equally important as middle-man costs. Mt. Gox “lost” many customers’ Bitcoins, then declared bankruptcy. No depositor funds have been recovered.

Mt. Gox customer letter, click to view full size

Electricity cost and externalities

Mining bitcoin is costly due to mining equipment prices and the high power usage, which result in large amounts spent on electricity.

Perverse incentives motivate uneconomic choices. The most egregious and harmful behavior directly associated with bitcoin mining that I’ve seen to-date was unauthorized use of a National Science Foundation supported supercomputer to mine bitcoin. $150,000 in computing resources, e.g. electricity, were spent in order to mine the equivalent of $8000 in bitcoin. Another incident occurred at Harvard University in March 2014. The researcher used Harvard’s high-powered network of thousands of CPU cores to mine an unspecified number of dogecoins.

We apply the FLUSH+RELOAD side channel attack based on cache hits/misses to extract a small amount of data from OpenSSL ECDSA signature requests…This means we obtain private key recovery by observing a relatively small number of executions, and by expending a relatively small amount of post-processing via lattice reduction. We demonstrate our analysis using…the Bitcoin protocol.

Using the attack described above, 200 signatures were sufficient to recover the secret key for bitcoin encryption protocol. Now, the authors have refined their approach, reducing the number of signatures to 25, see Just a Little Bit More and IACR eprint 2014/434 [PDF], emphasis mine:

We extend the FLUSH+RELOAD side-channel attack to extract a significantly larger number of bits of information per observed signature when using OpenSSL. This means that by observing only 25 signatures, we can recover secret keys…used in the Bitcoin protocol, with a probability greater than 50% …We utilize all information obtained and not just that in the least significant or most significant bits…whereas previous work require direct information on ephemeral key bits, our attack utilizes indirect information…

Update

Nice Mr. OkTurtle @taoeffect said it was a server side issue, and not to worry.

Not an autodidact

My only education-related experience has been passive, as a recipient. From kindergarten through 12th grade, I attended public schools in Las Cruces, New Mexico. I loved learning algebra, calculus, chemistry, English literature, French, U.S. history, physics, drafting, home economics, and orchestra. I find it difficult to learn from self-instructional materials. Learning by doing is effective, but requires some guidance.

K-12 education

I perceive betrayal of public interest throughout the U.S.A., due to federal government educational policy. New York City is especially troubled. Exceptionally wealthy individuals with ZERO experience or training in education have decided that they know what is best for America’s children.

The 0.1% of the 0.1%

The Brookings Institute describes them as the 0.1% of 0.1% in assets. Assets held is a robust metric for gauging wealth. It is important to distinguish between wealth and income. Income fluctuates from year to year, even for the wealthy. Causes vary. Some have profound impact, such as significant reversals of fortune. Some are merely transitory, e.g. accounting losses reported in order to minimize impact of tax law changes. These 0.1% of 0.1% individuals choose to actively direct the projects that are beneficiaries of their philanthropy.

Philanthropy, education reform and charter schools

Most education reform activists, or perhaps investors, have no knowledge, nor experience in public school education. Rather, they are exceptionally capable leaders of global software conglomerates. Others have great prowess as hedge fund managers, venture capitalists or real-estate moguls. Several are Wal-Mart family scions. Education reformers who have children educate them at private schools like Philips Andover or Choate, yet they claim that charter school operators provide a superior education, compared to public schools in the U.S.A.

Charter schools are similar to private schools, yet they are financed by public, i.e. taxpayer funds. They are not fiscally accountable, unlike public schools. Charter schools do not resemble Exeter or Choate-Rosemary Hall as far as quality of instruction or facilities, not at all! An increasing body of empirical evidence and peer-reviewed research indicates that charter schools are inferior to public school education, in myriad ways. The same is true for the uniformly despised Common Core Standards, by everyone other than ALEC and those who hold copyright to them.

Common Core poll – KFYI AM Radio 550

Anti-Schumpeter

Schumpeter wrote about creative destruction. I think that the current education reform movement is better described as destructive disruption.

Unfortunately, the media tows the line of those who claim that teaching must be disrupted. This is because those who are wealthy are influential, and LOUD. They like to say that teaching must be disrupted, in order to keep pace with the inexorable path of scientific progress.

Why the need for disruption? Answer: We live in an era of technology! Existing pedagogy is allegedly archaic, resembling that which was used for the past 1000 years. Silicon Valley is especially fond of saying that. In fact, 20th century teaching methods were similar to those used for the past 1,000 years, and for good reason: Our brains haven’t changed in the past 10,000 years! Our cognitive processing and synthesis of information into knowledge has not evolved over such a short time span. Technology is great, but technocrats who want to replace teachers with robots and mobile phone apps will do great damage. That isn’t how THEY (the technocrats) learned math, reading or anything else! Yet most websites where programmers, PhD educated mathematicians or physicists gather, cannot say enough bad things about how repressive, stifling, corrupt and inadequate our public education system is. Where did most or all get their educations? Surprise: K-12 public schools, often followed by land grant universities! Most excel in their careers, in STEM fields.

Root-cause medley of societal malaise

So, just maybe, status quo pedagogical methods, as applied before 1990 (and Common Core), were exceptionally effective! And the problem, now, is not teacher inadequacy and the lack of iPad’s from kindergarten on. Instead, there are profound societal inefficiencies due to a decade or three of so-called Democrats, who are neither Democrat nor GOP nor libertarian. They are crony capitalists, neo-liberals, oligarchs in the wings. What’s happening now is a logical consequence of:

a Byzantine tax code full of loopholes

inconsistently applied, sometimes adversarial regulation

no protectionism or support of American products in global markets

anti-union federal government policy under the Obama Administration

dismantled immigration policy

never-ending wars, yet not calling it war, but rather “conflict”

anti-intellectual wisdom of the crowds and the sharing economy, euphemisms for exploitation of foreign workers/U.S. underemployed youth and feasting off The Commons, respectively

“questioning everything” including mainstream scientific thought, while blindly following life coaches, anti-vaccination celebrities and pseudo-religious demagogues

dissolution of local community, partly due to unfair competition from huge e-commerce retailers; note that small e-commerce is hurt by this too!

ridicule and intolerance of religion, of any sort

failure to value the independent 3rd party press, e.g. “news is a commodity, distributed freely by the internet”

failure to distinguish between intellectual property laws that oppress innovation e.g. nutty software patents, NPE’s a.k.a. trolls versus copyright as a basic human right, that is, the right to be paid for one’s original work

I’ll stop now.

Disclaimer

In the style of Seeking Alpha investment analyses, I disavow any agenda, nor will I benefit in any way from this post. In fact, the contrary is far more likely, particularly since I am in search of employment. My passive experience with public education, that I mentioned earlier, is based on observations made by my mother and two aunts, each of whom has 20 years experience as public school teachers. Also, I have tutored college students who had trouble with calculus. I do that at my kitchen table, free, yet many can’t make time in their busy schedules, until failing out and retaking the class.

How to generate random numbers from spam

Generating random numbers is pretty complicated if you need them for cryptographic algorithms. This software generates them based on spam comments…

It caught my eye as a sort of “spinning spam into RNG gold”, or more likely, PRNG (pseudo-random number generated) gold. Many WordPress blogs, whether self-hosted using WordPress.org or not, effectively use Akismet as a comment spam sieve. As I’ve learned during my time with WordPress, and with spam comments, Akismet will not publish comments that it identifies as probable spam. This provides a possibly crucial aspect of SecurityDump’s application:

no one will be able to see the source of your numbers, unless they hack into your database

All the details are available on the Google project site for WPRandom, Problems and Attack Vectors wiki. I have no idea if SecurityDump worked the bugs out of this yet or not. I found it an amusing idea, though, to squeeze some genuine value from the efforts of spammers.

A slightly cynical RNG

Randall Munroe, of xkcd web comic fame, seems to have a less than favorable perception of the internet standards authority, Internet Engineering Task Force (IETF). xkcd 221 predated the recent mess that is HTTP 2.0 by many years, see HTTP/2.0 — The IETF is Phoning It In for lurid details, so I can only imagine what he would say about the IETF now.

I never attended DEFCON, though it remains a dream I hope to realize one day, soon. It may soon become too logistically awkward due to increasing numbers of attendees.

Shodan is a remarkable search engine. Traditional search engines use “spiders” to crawl websites. Shodan culls data from ports. It was created by John Matherly in 2007. He continues to develop it.

Shodan is helpful for locating web server vulnerabilities. It is available as a free service, for up to 50 searches. Query syntax includes searches by country, host name, operating system and port. Shodan can search for software AND hardware. It has been acknowledged by mainstream media. The most prominent coverage was in early June, via The Washington Post, when Stuxnet received so much press attention.

Me and Shodan

Next is my Scribd infosec collection. It isn’t exclusively Shodan-related. This is why. I first noticed that Michael Schearer wrote excellent PowerPoint presentations, and kindly posted them on Scribd. I was curious, searched for more. This led me to Shodan HQ.

Shodan is NOT an acronym for Sentient Hyper Optimized Data Access Network. John Matherly kindly confirmed this, in response to my recent inquiry.

@ellieaskswhy nope not an acronym for my project, but it’s the acronym for the videogame character the name is based off of

Human knowledge belongs to the world, after all, and information ALWAYS wants to be free.

I wrote an information security and data privacy post, on my Tumblr blog. The (inline) PDF document from hal.inria.fr is the highlight. Along the way, I learned about quines! They are a cute construct, like palindromes, or self-replication, or polyglot programs; fun, maybe useful.

A collection of web application back doors and malware, in PHP, JSP, ASP, etc.

DEFCON 13 badge

Let’s conclude with a video, Hackers vs. Disasters Large and Small. The venue was DEFCON 13. Introductory remarks were by Michael Schearer, which is where this all began. Be forewarned, duration is 1 hour, 45 minutes:

Most often we’re “on the grid” and close to our precious electronics and high-speed internet. What would happen if you find yourself stranded in the middle of nowhere or in the midst of a natural disaster? This presentation will show you that your hacker ingenuity can help you survive the worst.

WE'RE DOCTORS. WE KNOW BIOTECH

This was prescient for May 2012. I've since read about an appalling fiduciary failure. A physician lost millions in public employee pension assets. He had just started as a portfolio manager for a biotech fund which was selected to manage a big chunk of that state employee retirement money.

The physician had no prior experience as an institutional money manager.

I wouldn't want a biotech portfolio manager as a medical care-giver either.

Defense Department leaders have decided that the best way to protect sensitive information from cybercriminals and internal leaks is to consolidate its 15,000 networks into a single “joint information environment.” JIE is a set of security protocols — which the Pentagon calls a single security architecture...Although the JIE is not a “program of record” with its own funding line, it will be financed under the Pentagon’s $23 billion cybersecurity budget. Leading the massive network integration effort is the Joint Staff, U.S. Cyber Command and Defense Information Systems Agency [DISA].

It is good that DISA will serve as the Joint Information Environment (JIE) development hub, with associated accountability. However, I noticed that the head of DISA, Air Force Lt. Gen. Ronnie D. Hawkins Jr., mentioned this:

I am familiar with a cloud services vendor, Amazon Web Services (AWS), who uses the same terminology, "identity access management" or IAM. AWS is a wholly owned subsidiary of Amazon.com, the e-commerce retailer.

FedRAMP

I do not know whether IAM is an Amazon.com trademark, i.e. if it is unique to AWS. I do know that Amazon was granted FedRAMP, a special government contracting credential, in May 2013.

FedRAMP is a is a cost-cutting initiative intended to do away with checking the safety of cloud services, if the products already have cleared the program’s boilerplate style security audit. It guarantees that a cloud service complies with U.S. laws for guarding federal information, but is not applicable to classified or high-security systems.

Outsourcing

In my opinion, something as vitally important as our national defense deserves to be handled internally, watched over by government employees and military servicemen whose loyalties and steadfastness is unwavering. Congress should put aside sufficient funds. I can't think of many other government initiatives more worthy of being done right! I truly hope that the new Joint Information Environment will be implemented by U.S. Department of Defense staff or civilian employees of the federal government, rather than being outsourced to Amazon Web Service's cloud.

Computing Reviews is the Association for Computing Machinery (ACM) publication review website. It isn't scholarly journal peer review, but rather, book reviews and critique of trends and new developments in computer science and computing applications.

Reviewer revue!

Computing Reviews runs a monthly series, Featured in Five. No, there's no singing or dancing, like a musical revue. The same five questions are asked of each month's featured reviewer, a "reviewer revue", so to speak. I recommend the series in general. Reviewers are computer science academicians or information technology professionals at various stages of their careers. The variation in answers to the same five questions* is fascinating.

My interest is in applied math, statistics and probability. One post dwelt on data analysis more than the rest, and caught my attention.

Big data and uncertainty

Modern technology allows us to generate, store, and process huge amounts of data... big data can (and will) lead to revolutionary breakthroughs in science, engineering, medicine, and so on. The biggest challenge comes from the fact that all this data comes with uncertainty...

It is essential to gauge data accuracy. Until that is well understood and accounted for, big data will be of limited usefulness.

Bounded accuracy

Why is the accuracy associated with big data different, or more difficult, to study than "medium" or "small" data uncertainty?

Traditional data analysis and data quality assessment was based sources or which error rates and causes of inaccuracy were known or could be measured, whether deterministically or using models. Kreinovich (computer science, University of Texas at El Paso) gives a specific example:

Processing techniques originated from processing data from well-calibrated sensors, sensors for which we know the probability distribution of measurement errors, for which we can use traditional probabilistic and statistical techniques. The point of big data is to... supplement these "perfect" measurements with numerous less perfect ones—for example, we use temperatures regularly measured by volunteers who use off-the-shelf non-perfectly calibrated sensors. For such data, we often only know the upper bound on the measurement error...

In addition to this bounded interval of data inaccuracy, we also have partial information about probabilities. That is helpful, but it isn't sufficient to solve the central problem.

Instead of knowing the exact values of the characteristics, we only know the boundary values

This was an April fool's day blog post!

Disclaimers aside, it is amusing yet informative. Cycle Computing succeeds in being light-hearted, whereas I must turn everything into a ponderous, weighty object lesson :o)

ZeroCompute™ : Eco-conscious, Instantaneous Utility Supercomputing

...by simply not running the science on any cores, we remove the computational challenges and data transfer bottlenecks of today’s BigCompute and BigData workloads. Because ZeroCompute "completes" the floating point in the nanosecond it takes the software to decide not to run anything... peak floating-point performance exceeds one billion petaflops, at a cost of $0.00 per flop.

The only way to avoid all negative consequences is to avoid any action, any innovation at all. That is associated with negative consequences as well. The numerous, vocal opponents of nuclear energy and modern agricultural methods would be wiser and more compassionate if they were to take such ideas to heart.

Vaccination to prevent the childhood diseases of diptheria, whooping cough, mumps and measles, as well as adult diseases such as tetanus, typhoid, rubella, polio and smallpox are a different matter. They are an unmitigated public good. There are no concomitant hazards such as ionizing radiation exposure nor GMO innuendo. Our brave, heroic, brilliant though now-deceased Surgeon General C. Everett Koop would have been able to get through to the anti-vaccination evil-do'er groups, if anyone could.

Myanmar is the world's second-largest producer of opium. Heroin abuse is widespread. The center's popularity is a testament both to the severity of Myanmar's drug problem and the lack of options in a poor country where modern treatment programs are rare. It offers prayer, Bible study and devotional singing, with football and weightlifting for those strong enough.

The text, above, accompanies an expressive, compassionately photographed gallery of ten images featured in Reuters Online, "Photos of the Week". Subjects were portrayed honestly and sensitively.

Myanmar versus Manhattan

Faith-healing has connotations of superstition and ignorance. Here, it should not.

Despite public perception, and some of the nuanced annotations accompanying the Reuters photographs, treatment for heroin addiction is comparable, regardless of wealth or poverty. "Modern treatment programs" are quite similar to what is offered by the Youth for Christ Center in Myanmar, specifically, bed rest, mild food, emotional and psychological support and the company of others in recovery.

In fact, the option of remaining in treatment for 40 days to as long as three months is rarely, if ever, available in the U.S.A.

To their credit, Reuters does not sensationalize the first week spent in confinement in Myanmar.

Other sources refer to it as "behind locked doors", "behind bars" or worse. Those who do so neglect to mention that heroin and most other addictive drug detoxification program in the U.S.A., Europe and Australia have the same protocol. If you leave the facility during the first five days, or week, you are choosing to leave the program. Should one wish to return, the entire intake process, including determination of eligibility, must be re-initiated. Given scarce resources, one may be denied because of prior behavior, in favor of someone who has never entered treatment.

In Myanmar, palliative measures e.g. over-the-counter remedies to ameliorate the symptoms of withdrawal such as Motrin (ibuprofen), Imodium (loperamide) and Benadryl (diphenhydramine) are not available to patients. These are important and helpful for relieving some of the misery of heroin detoxification, and are offered at Western treatment facilities.

Many heroin in-patient detoxification programs in the USA and Europe administer some combination of the following: clonidine for elevated blood pressure; pharmaceutical opiates in quickly tapering doses for 3 to 7 days; older (possibly not SSRI) anti-depressants such as Elavil or Trazadone to alleviate severe withdrawal symptoms. These measures, as well as a licensed medical caregiver's supervision, constitute the greater portion of the modern treatment to which the Reuters text referred.

Heroin addiction treatment in Myanmar is superior to detoxification experienced while incarcerated in any jail or prison in the world.

Recovery is not location dependent

Recovery is difficult for everyone. There are no pratical alternatives to the grueling process of heroin detoxification. Recidivism is a major obstacle to complete recovery. Offering the equivalent of in-patient care for up to three months in Myanmar is remarkable, and likely to improve long-term prognosis.