Instead of a script to embed, these firms could provide an API to identify users from the server side. The scripts that captures the profile would be served by the sites themselves rather than from third party services.

Toast.

A possible solution would be anonymize the browser fingerprint, at least in private mode, ie lie about the details of the system.

I'm skeptical of this unnamed company's actual abilities. In the initial email how are they able to identify anything about your visitors before you've installed the tracking code? Since they apparently can see search terms used to reach your site the only thing I can think of is their code is running on some site that links to you (perhaps an off-brand search engine?) and they're tracking outbound clicks. Or it's fake.

It's pretty easy to guess company name from IP address, especially if you don't care about accuracy. You can kinda sorta do this in Google Analytics under Audience > Technology > Network. That seems to be roughly what they're doing in the screenshots posted. IMHO, this is not the most serious privacy issue on the web.

I would be very curious to hear exactly what percentage of visitors it is able to supply Name and Email for (and how many of those fields look bogus). This sort of individual-level tracking across sites is obviously possible, but I don't think it's common. Google/DoubleClick do not, as far as I know, do any sort of tracking at the level of an individual's name or email address (And why would they? It's asking for regulatory problems and it doesn't really help them much -- they target ads to groups of similar people based on demographics, not to particular named individuals.)

HubSpot (and pretty much any other marketing automation tool) has this feature, too. They lookup company name and location by IP address and build an anonymous "prospect" record representing each visitor so that salespeople and marketers can detect whether prospects from a given company are hitting the site for information.

The second a prospect submits a web form, all that previous web activity is tied to their email address (and any other info you collected via the form). You now have a real lead.

I don't see any privacy issues with this.

What I would see an issue with is if the tracking company were sending the IP address and cookie back to a central database to query "Does anyone _else_ know who this visitor is?" and then provide PII any company who uses the tracking service.

The moment you start giving my PII to a company that I didn't voluntarily give it to is when I feel a line has been crossed.

This sounds eerily familiar. Around a decade ago, a data analytics company called Pharmatrak was actually found guilty of breaking federal wiretapping statutes for doing something very similar. [1] In their case, they had built a network tracking HTTP GET requests to pharmaceuticals companies websites with a web bug [2] and attached cookie. But because some of the pharmaceuticals companies were using GETs as the method on HTML forms (remember, this was ten years ago), the users actually ended up making GET requests with personally identifying information in the URL encoded parameters. Since these GET requests were logged by Pharmatrak, and neither party (the users nor the pharmaceuticals companies) had consented to giving away personal information to them, Pharmatrak was found guilty of wiretapping.

Pharmatrak eventually won on appeal though, arguing that they had no intention of collecting personal information, which exonerated them because only intentional eavesdropping is a crime.

The company in the OP's article could make no such arguments though. I suspect that their main difference is that they make no assurances of confidentiality to the websites using their software the way Pharmatrak did. Which 1) is just really creepy, and 2) sets them up for trouble with users in California, because California's wiretapping statutes say that it's a crime unless both parties agree to it. [3]

Dataium does this too, as covered by WSJ's recent article on the subject [1]

The article goes into depth about how much personal information is sent along to advertisers including a popular dating site's apparently anonymized information about drug use, and sexual orientation.

I think we need a non-profit service that defines a set of privacy licenses (akin to CreativeCommons' licenses) which companies can opt to label their websites/apps with. There would be no policing/auditing [2], but companies found to violate the privacy licenses would be obliged to donate a sum to an organization like the EFF.

That the privacy policies would be encompassed by one simple privacy licence badge would allow users to quickly and easily identify a company's privacy policies. I believe users would gravitate toward using services that display this license.

It wouldn't completely work here (e.g. EFF's panopticlick could still fairly uniquely identify me, or IP address would give away info if I'm not going through my VPN), but it improves things.

It feels kind of extreme, but it's worth it to me. My experience is not broken that much, and I feel like various sites are aggregating less about me. These tracking technologies not such an issue now, but I foresee at least the possibility of abuse in the future, so I figure I'll do what I can now if it's not too much hassle.

Lastly, at its heart most of this is about advertising, something I know I'm very susceptible to (try as I might to convince myself I'm not). So the better I am at blocking out these things, I think the less money I'll spend in the long run on frivolous nice-to-haves.

Going to site A, not providing any info, then going to site B, C and D and seeing ads to site A haunting you is one thing, capturing your name and email is a new level. If you don't use a tracking blocker, clearing cookies is not always going to work, these persistent trackers are quite sophisticated, they use local storage if possible, IP address, header information and whatever is possible to be able to identify someone, there is a huge industry behind it. But this one is taking it a little bit too far, scary.

On the other side, most startups including YC ones, use some sort of tracking for analytics to improve usability and internal flow, so advocating against all trackers and for all users installing a blocker is a double edge sword.

I get emailed by them for every startup I'm involved with and that first email is mostly the same every time as you can see in that screenshot. (Compare it with the one posted in the article and you'll see).

They seem to be targeting startups and make it look like some big VC firms are visiting your site to get you interested. I'm not sure how they come up with the 'search terms', but I guess they could just look at your META-tags or make them up.

In their email they do say it's a "mock example", but still I find it very deceptive.

This kind of thing is what I've always seen as the potential end result of things like google analytics and also facebook connect. Both products that have javascript running on a vast number of websites, with the potential to link to personally identifiable information, in a similar manner to that discussed in article.

Read the article, thought that it was something interesting but probably not that applicable to me because I clear cookies on (frequent) browser close, don't enter my details into many sketchy sites, use multiple different (isolated) instances of my browser for different purposes.

Today, I get an email from a site that I visited yesterday and haven't heard from in 6+ months. It's too much of a coincidence for me to assume it's random so I dig into their website a little and they're using one of these services.

TL;DR: even though I'm relatively paranoid with giving out details online, one of these networks seems to have successfully identified me and provided my email to a website that I visited, who then reached out and tried to sell me shit.

I had to give Dick Smith (A NZ retailer) my phone number before I bought an external the other day.

"Do I _have_ to give you my number before I buy this?"

"yes, but it's for return purposes only"

Of course I received 'promotional' txts the next week. I was hesitant to give it to them for just this reason, and because I acknowledged I had a phone number I felt obligated to give it to him. Dick Smith is a member of a larger chain it's no stretch of the imagination to hook up CCTV cameras to an OpenCV instance and send txts to customers when they walk in.

No matter the law, morals people hold, or customer wants large companies are always motivated by profit margins. The Consumer Guarantees Act, the Privacy Act, the Bill of Rights Act all become murky when you're dealing with new technology, and law will find it hard to keep up.

I may be one of the few and perhaps I've just been desensitized with all the social network invasion, but I don't find this stuff that reprehensible. At worst, its moderately annoying because its one more email that I have to archive but its definitely on the lowest totem pole of annoyances. Recruiters have been cold calling and emailing me for years based off of my LinkedIn and Github profiles and all I have to do is tell them "no thanks" and my life goes on.

I recognize these screenshots - it's definitely Leadlander. I'm not sure if they do what he claims they do, but they can identify by your IP which company you belong to (assuming you're connecting from the office). There are a lot of companies doing that right now actually.

That said since a (very) long time I'm using separate Linux user accounts to: check my professional email + G+, surf my personal email + G+ + FB (my FB is using a fake but plausible name) and a third one to surf the Web.

The one surfing the Web is linked to a fake online identity: entirely made up, with fake friends / fake G+ circles, fake StackOverflow / OpenID and basically fake everything.

I then only ever surf using a transparent proxy for anything "work related": the IP can't be linked to my fake IP.

It's not difficult to set up: I did set up the transparent company Web proxy (VPN would to too) myself and basically Linux user accounts take care of the rest.

Now I'll start using different browsers too and, why not, maybe Tor in one of the account.

I take it I could take all this a step further and whitelist websites that my "personal" account is allowed to connect to (using iptables' owner-uid mod).

How the article is designed - it took me a while to understand that 42floors was not the company performing the tracking. I initially went there to find a name of the company (to put it on permaban in my NoScript), yet the only organization popping up while skimming through the page was 42floors. I was a bit spooked when I checked the noscript-list for blocked resources and saw the url I thought was tracking me.

After that, I looked at the URL-bar and it took that long for me to click.

Advertising and marketing companies aren't the only ones that do this. Any corporation which owns more than a couple websites collects bits of information about them from each site and then builds profiles of its users, often then selling the information.

Say you own a sports website, a fashion website, a political website, and a gaming website. The user only specifies a tiny bit of information on each website. Each bit is collected into a single user profile from which they can refer to do things like figure out what product advertisements to show them. They use the same techniques to identify users that don't have accounts, and still collect their viewing/interacting habits and add them to the profile.

Sometimes they'll send you an e-mail telling you to check out their gaming website if you're not signed up, because the comments you write in their other websites' forums have to do with gaming. Sometimes they just sell the information to a gaming company. In the case of Target, they might send your teenage daughter a list of baby products for the little one you didn't know she was expecting.

This is not some horrifying violation of privacy. There is a price for all the free shit you get from the internet. Usually it's paid for by all the personal information you leak onto the net. They're just mopping it up and selling it back to you.

I had a similar experience with FlightFox. I entered my origin and destination and got distracted and closed the tab. I get an email several hours later asking me to start the contest with the exact two places. Creepy, much?

Shouldn't we be able to make these systems useless by filling them with loads of fake data/spam?

I.e. if I have to fill out a form somewhere, I would not only submit it once, but several times (ideally automated), ideally with realistic data, i.e. other businesses in my area (so geo-location won't raise a red flag then).

If I visit the next website which employs the same network, they can't really identify me - they have a big set of businesses I could possibly be (or they just take the last one, which would be fake).

At least currently, they do not seem to verify whether the filled out form can be properly validated, i.e. if the user clicked on a confirmation mail or similar.

I'm rather amazed that any company would put this on their website. What you may be doing, in fact, is likely identifying customers to your competitors. Cross-shopping is very common in most product categories - so it is quite possible that you're giving up your customer to a rival.

I find it interesting that this article should show up the very same day I found out about TAILS - The Amnesiac Incognito Live System ("https://tails.boum.org/), a live Linux distribution that uses TOR and other tools to enhance your online privacy. The more I read about online tracking efforts like this, the more I want to set up a wall around my computer.

So, how feasible is it these days to do all of your browsing through a VPN? Not that a VPN's going to save you from the attacks mentioned here, but hey, maybe it's time to start getting serious about my privacy.

Please... A clever use of GA + Wolfram Alpha can reveal a lot of potentially identifiable information already. You can't expect the Internet to become a big part of our society and, at the same time, remain a place for complete anonymity.

I am okay with companies displaying ads to me - this is what pays for the web to exist. If however things like this continue to exist then I will take up all options offered to opt out of identification and ad networks. Google and Microsoft etc should take note to shut this sort of behaviour down.

Some of them have been thought out - "how I mapped caps lock" by yakshaver, "derailed by a pedantic comment" by wellactually(I suffer from this; working on it), "apple's downfall" by armchairceo; the comment puns(stylish, worn-out, nebolous) are pretty meh.

Despite this being a parody, I would still like to point out "a labor of love you can say mean things about". I have seen it happen here way too often. Someone posts something and the crowd goes wild - "this is a feature not a product", "as a designer I can tell you you suck", "another cool aid drinkers pretending node.js is cool" etc. Someone posts a "Show HN" doesn't mean you get the right to walk all over it. And the worse part is, you pretend you were doing him a favor - "I was only giving feedback which the poster asked for". The poster asked for feedback, not for insults. It doesn't matter if you are a programming god(most of the people doing it aren't, but still) - there is a difference between feedback and "look at this pathetic shit thinking he is worth anything".

I am sure I am not the only one who thinks people go overboard with their so-called feedback. pg especially made a post about the flood of launches coming in and being nice to them.

Please be nice to them. For you their launch may be "yet another YC startup," but for each individual startup this is their big moment.

"My 5000 word assessment of (insert latest gadget here) that leaves no doubt about why absolutely (no-one|everyone) should buy this, conveniently ignoring that other geeks have their own opinion and the average user doesn't give a damn"

"10 mistakes that I made during my unsuccessful YC interview, 9 of which PG probably didn't even notice"

"Show HN: My first (read: I'm younger than you) app I hacked up in less than 20 mins (read: I'm smarter than you), because I had nothing else to do (read: don't feel too cozy in your job, cuz I'm dropping out of college any day [if my parents agree, that is])"

"How I made big bucks with something so trivial that I'm surprised my cleaning lady did not think of it before me"

"Apple is no longer as innovative as they were 5 minutes ago"

Plenty of discussions that mix a lot of fancy Latin expressions and Scotsmen

Recruiter: a derogatory term for somebody that knows less about a specific field than than the experts in that field they hire

I loved the genuinely parodic headlines ("How I remapped my caps lock key to be both ESC and Ctrl" was my favourite), but the metaheadlines like "A long rant about a YC company falling short in some regard" are simply done to death and IMO have not been funny in years.

I laughed at "How I bootstrapped my company in 6 hours (with breaks)" :) I always wondered why this was a thing on HN, it doesn't matter if something took one weekend, one month or one year to finish - it's the result that matters, and saying "I build it in 24h" is the same as saying "it has a lot of bugs, but I want to launch it anyway".

Congratulations on your neat website. I think you really should consider making you're page center aligned - i hurt my head looking left in my bigass screen. I am very much a fan of wearing the same shoe every 3 days, but i admit youre article about sexism has changed my life. Unfortunately, since you don't support openID, i refuse to join your site.

To complete the picture, i would really like to respond here with a very long-form comment of at least 8 paragraphs, in which i address one by one a number of points that irritated me in a boastful way that makes my arguments sound authoritative and deep, while in reality it's just my brain-fart of the moment. I should also add a couple of references in square brackets, because that's what scientists do[1] and people seem to believe scientists (what's more, the most respected scientific journals use the vancouver reference style, which makes me look even more knowledgable). Because of the outrageousness of my long-form comment (and because it takes up a large portion of the screen real estate), people will upvote me and respond with equally half-cooked comments , some of them applying Godwin's law, and others merely acknowledging this application of the law.

Unfortunately, i barely made it to 3 paragraphs and i 've more or less run out of stupid things to say.

Looking forward to the new version of your website that will be implemented in a single var javascript statement, because that's my favorite programming paradigm this afternoon.

I really hate it when companies try to sugar coat a decision that is all downside for their customers. It may be really hard to do, but Google should just come out and say it: there is no upside for this to anyone except Google. It is not about giving you a better experience or making things more straightforward. It is all about Google deciding to maximize their profits at the expense of their users. That's fine, it's what businesses (ultimately) do (even the ones that pretend they put their users first). Google should just say it - they no longer want to support a free version of their product because they can make more money another way.

When I see ridiculous sugar coating it breeds distrust and disbelief - congratulations Google, I now believe every single future thing you say a little less, well done.

Huh. Just want to point out that one of the main reasons we're using google apps at my company is that I use google apps for hosting my personal email. When it came time to make a decision on that, and I am the one who makes that decision, I chose google apps because it was already familiar to me.

Poor choice, imho. I'm curious what the actual overhead is for people like me. I have 1 account (as in: 1 email address) that is hosted by google apps. I was going to set up an account for one of my other domains, but not for $50/year. (Per account!)

So full snark here, but it was between google apps, and office 365. I chose GA because o my familiarity with it.

But look here, Microsoft's equivalent offering is free: https://domains.live.com/ I wonder what things will look like when we evaluate google apps next year?

by offering <50 email accounts for free, Google essentially destroyed the market for any other startups or companies to come into the market and offer non-enterprise B2B email services... thus limiting competition and innovation.

One could argue that the removal of the free tier at this point is simply because they've created an entrenched position, but one could also suggest that this creates a modicum of opportunity for another player to try to enter this space.

Certainly until today the $ size of the addressable market in the small business email space was practically $0 given Google's position.

I'll tell you what the free plan was most useful for: business experiments.

I can't tell you how many small forays into new business ideas I and other folks I know have used google apps to kick off. For a business at high risk of failure and no budget in the early idea stages it was so useful to be able to stitch together a team and workflow really quickly using google apps to see how things pan out. This puts a real damper on kicking off new collaborative ideas

Outlook.com's equivalent offering, https://domains.live.com, is free (Hotmail also offered it, but they've changed the branding). I've been running my email on there for the better part of a decade. To those who point at Office 365, that's similar but different, and it costs money.

Between the mistaken despair over having no good free alternatives to Google Apps and the platitudes and awe over the Gmail/Google Drive integration (which Hotmail/Skydrive have had the exact equivalent of for years) a couple weeks back, I'm starting to wonder why nobody here, when they are all quite tech-literate, seems to have any clue about Microsoft's honestly rather impressive cloud offering.

I'm not sure if I'm parsing this correctly, but it sounds like people who use Google Apps as an email backend for personal email on custom domains will now have to pay $50/year.

If so, that's a huge bummer. I only recently switched to running my email this way and I don't relish the thought of migrating elsewhere so soon.

I would happily pay $50/year (or perhaps more... I don't know how high I'd go) for email that's not only convenient and spam free, but also well protected from governments and the provider's employees, and not data mined.

I have absolutely no use for phone support or 99.9 uptime for my personal email. So the businessification of Google mail is not a win for me. Privacy and convenience or what I'm after.

No, I don't mind the end of free Google Apps. I use Google Apps for my business for five years, and frankly, it is mind-boggling what they have given me for free. Luckily us existing users still get the free service, but if I had to pay US$50 per year per user, I would still consider it exceptional value.

US$50 per year for a set of critical, heavily-used services is an inconsequential amount for all but the most penny-pinching operation.

I have no problem paying for a Google Apps account where I actually use Google apps, but at the moment you have to have a Google Apps account to link a domain to an Appengine app. Some of our apps have two or three domains showing the same app, and because you need to have an account for each email address that Appengine sends email from, we have three or four accounts (support, noreply, accounts per domain.

So this move is going to add $600 per year to our costs - all for virtual accounts that don't actually use Google Apps at all.

Hopefully this was unintended, and Google will continue to provide free accounts for domains linked to Appengine apps - or provide another mechanism for linking and authorising sending addresses.

As a Googler and for much longer, a Google Apps user for my family, this is sad news.

I wish they had just fixed the experience for "vanity domains" so that they didn't require all the enterprise features, and didn't let administrators have complete control of users accounts. Then they could roll out new features to vanity users without needing the enterprise controls.

I understand that this is a little tricky, and involves no paying customers, but Google Apps was by far the best experience for custom domains. I'm sure it attracted the type of influencers that pull even more users. I used it for my family, bands, friends with very small businesses and more, and the other people on those domains not already using Gmail, in turn migrated their personal accounts to Gmail.

Wow. Google Apps for Your Domain is a huge thing for a lot of hackers. I run my entire family's email using it. Is google just giving up on supporting custom domains? That would be a huge disruption for me =(

Google believes the cloud/apps have enough providers now, so they aren't needed to drive it.

Their strategy is to grow the web. This works because they make more money from the web being used. It also makes people love Google, which is important because switching search engines is easy. The love also helps in hiring.

However... they have introduced internal cost accounting, so that products must pay their way - a little internal market. IMHO, this is potentially dangerous, since there already is a market (the real one), and it ignores the advantages of a firm (http://en.wikipedia.org/wiki/The_Nature_of_the_Firm)But they don't follow it absolutely, e.g. android isn't paying its way. (NB: Apple is focussed on making better products, not on growing the web).

If google can see that other firms are better placed to do a better job of growing the web in some respect (or they can apply pressure on them to do so), they are happy to step aside.

On a related note could dear hn crowd suggest email provider (maybe paid, say up to 30-40$ annually) caring about my privacy a little bit more? Really delete my messages when I want to is a good start.

Google is all good but I feel I should not put all of my emails in one account..

With this decision, I have now come to believe that Google is not the company that it used to be in the past. The user-friendly company with a quirky personality.

Somewhere in the mindless fighting with becoming the dominant social and mobile force on the planet, it has forgotten about the very users it once strived to please.

This has been more apparent since Larry took over Google as the CEO. His "more wood behind fewer arrows" has somewhere down the line taken away the humor with which Google has largely operated. 20% time? Google Labs? Hugely popular Google products scrapped into oblivion.

The only products that matter beyond search are - Android, Google+, YouTube and Google Apps which are fairly mature by now and are a serious threat to competitors.

The thing is they didn't need to do it because they aren't starving of computing resources for more important products and the marginal costs of adding free users is almost nil (although it exists).

I somehow feel betrayed by this decision. Google, so far, has resisted the temptation to shut down products that were important to it's users unlike other companies like Microsoft and Yahoo! I have been using Google Apps since they launched it a long ago. It feels like bait and switch.

This is completely right and there is nothing wrong with them shutting down a free service as a business but somehow it feels so non-Googly.

At 200 free users, there was no room for a new entrant in the market. I am glad that didn't last long.

2) On Charging too low...

Back when they launched it, $50/user/year was extremely aggressive pricing. Vendors were charging $50/month. May be Google is realizing they were charging too low. I now see an additional plan with $10/user/month option (which didn't exist earlier).

I gave up on Google Apps for your Domain for email a while ago (security, bugs, and the general black box nature of the product).

Right now, my favorite solution is Kerio Connect, which you can either self-host or purchase as a cloud solution. There are hosting providers who will handle all of this for you, but having the option to bring mail fully in house is really nice.

It's essentially Exchange, but much easier to manage, and far cheaper.

They also have a Sharepoint/Box alternative, Workspace, that I now love.

I'm more than happy paying $555 for a server and then $45/user for license, $15/user/year maintenance, and hosting costs. I really don't think $50-100/mo/user (once you factor in admin/hosting costs...you could do it for $20-30/mo but $50-100 is a safer budget) is an unreasonable amount for top quality email and collaboration tools.

This might be painful at the beginning but it could be a win-win situation in long term for most parties:

1. Google: can focus on customer service,2. Google's share holders: more revenue,3. Competitors: more competitive advantages,4. Startups: time to disrupt,5. Users: Email should be decentralized, why rely on a single provider is a good thing?

I'm from a SouthAmerican nation, here the IT budget of small business is pretty limited and Google Apps (even with the 10 accounts limit) was a viable option for them. Google will have to adapt the prices for non USA markets (BRIC, EMEA, Latam) if they want to succeed. Also this movement bring space of innovation for email.

I wonder if the cause of this is the support costs required for the free accounts? Obviously personal GMail is plug-and-play, but for someone who can't spell DNS, setting up a domain account was never quite drop-dead-simple enough. I suspect the goal here is to create a barrier to entry and prevent the inevitable support load of free users.

There was an excellent post about this a while ago, where a developer reported much better treatment from users after charging a token fee for their app.

As a final thought, maybe Google is catching on to the 'charge what something is worth, not what it costs you' way of thinking. Hosted email solutions for enterprise aren't free, and it seems like Google has realized the real value of their product. Also, ~$4/seat/month is pretty well in line with this kind of SaaS offering.

Oh come on... can we just have the gmail piece for free? If you want limit it to 5 addresses so that only really poor hackers use free account and they become your paying customers as soon as they find some traction.

This is really a logical step. If you are an individual and don't really see the value in $50 a year, you can forward your own domain to your regular Gmail account and set the reply to email address to whatever you want.

If you run a business and need the Google Apps platform, then $50 a year really shouldn't be a barrier for you. If it is, time to rethink your business priorities.

In reading the comments generally I feel like there's two main groups being affected here: small businesses and small groups (families, geeks, whatever).

The first groupâ€"small businessesâ€"can of course afford $50/user/year, even if they would rather not spend it.

The second groupâ€"enthusiastsâ€"even if they could afford it, likely cannot justify $50/user/year. I personally have a custom email hosted by Google for a small group of my friends, and I know that there is NO WAY any of us would pay $50 each per year. The price would need to be an order of magnitude lower for us to consider it. (Yes, I know that for the moment we're grandfathered in.)

Seems like there should be a way for Google to distinguish between the two groups based on services needed and then price two tiers accordingly. For example, offer a Small Group Plan at $5/user/year with a limit of 10 users & no phone support; and another Small Business Plan that offers up to 50 users, limited phone support & whatever else help small businesses for a higher price.

What really struck me was that they're only offering three nines of uptime. I can't see many businesses ditching Excel or Word to go with something that will cost you half a day of productivity every year, especially not for $50 a license.

I use Google Apps with my personal domain. A lot of people are recommending alternatives but the main reason why I like Google Apps is that I can use it to easily sign into YouTube, Google Docs and various other services that they provide with my personal, official email address.

I use Google Apps for a small blog that I run and I gave out email addresses to all of the writers. Eventually we had 11 writers and they only allow 10 free so I upgraded that one. People can't do that now. It's either all or nothing.

Meh, their loss. May have to go with Microsoft's alternative or something.

Ugh. The worst thing is that I've been relying on my personal domain e-mail address _also_ being a valid google address. So I have to either pony up for my family's accounts or move us all and re-create logins on a page or so of "authorized apps."

Still, the huge amount of spam that's been getting through (10-15 per day, despite always logging in to the gmail web interface and clicking Report Spam) has been encouraging me to consider a move anyway. This change is just the nudge I needed to finally make it happen...

At the very least there will be less potential customers now. I imagine some people would have been willing to pay $0 to Google and $15 per user per month for some kind of project management offering, now it's $65 per month, which is really going to hurt new sign ups.

Also for anyone with an existing free account who wants to add new domains, don't forget you can add domain alias's. So you keep the same account but just make it so email from two+ different domains can come in. For tiny ideas that always have the same 1-2 employees it's almost preferable to a whole new account anyway.

If you use google apps already, nothing changes (if I'm reading this correctly?)

If you're thinking about using google apps, you've got to pay for it now.

Yes, this benefits no one but google. Then again, google has been providing benefit to millions of people with google apps. Running your own e-mail server is not trivial. There's a reason there are no good gmail alternatives--it's harder than it looks.

If I were them, I would have charged for it from the beginning--it's a really useful service. And they could have been real dicks by forcing all current free users to start coughing up dough.

I just did a little research to see what alternatives are out there for custom domain emails. I haven't used these Namecheap for email, though I have used them for domain hosting after the SOPA debacle.

Not to change the subject, but this decision just made my switch to Office 365 that much sweeter. If anything I love that I get push notifications (active sync) for emails and group calendars work much better.

This is actually expected development. As the market matures, it is time to start make money.

This is also means that probably less and less of online services will be free.

Of course, you will still have "Walmarts of the online services" but you will get what you pay for - nothing more.

And I don't think there will be "cheaper" alternatives: but I do expect emergence of more expensive and better alternatives (if you can fork $50/year than you can fork $100/year - free is different story).

For people who just need email forwarding https://www.nearlyfreespeech.net/ work great. They charge $0.02/forwarded domain/day. So ~7USD per year per domain. I'm a customer for 5+ years, no complaints.

Seems like a strange move by Google. Most small or starting businesses don't need a 25GB inbox (at least not every user) or 24/7 tech support. Why create the barrier to entry? Now people with businesses will be more likely to use personal Google accounts for business or look for alternative services. Why not just create a premium business account option that gives you this level of storage and support if you need it, instead of trying to convince your users that they need it, when most of them are happy without it.

Personally, I liked the direction Google was heading with their Google Drive product. You get up to 5GB for free and then pay for more if you need it. Makes me wonder how Google might try to monetize other products like Google+ once they have a substantial following.

Just the other day I was looking for ways to offer email@mydomain to people who request it. I found that GApps was no longer offering free email. So, I ended up using Forwarders in cPanel to do the thing for me.

Forwarders in cPanel simply copy all incoming mail to email@mydomain to myother@email. If I didn't already create an email account email@mydomain, the incoming mail WILL NOT be stored on server which makes this a feasible solution. If I had, incoming mail would be copied to myother@email.

I think it's time for a dedicated email service provider with full search functionality and tools to migrate the old inbox from Google Apps. I would gladly pay more than $50/year for a simple, rock solid service like this if it had an excellent web client, real customer support, and cared about my privacy.

I'm confused here. It says nothing changes for existing customers but when I log into my domain's Google App dashboard I see this now: "Free 30 day Google Apps for Business trial" with an upgrade button. Does anyone else see this?

I would even pay $50 a year if Google stopped making "improvements" to the UI. The new composer looks nice at first but it slows me down when I need to use different fonts, etc AND they removed background color AKA highlight. I do not use Evernote just because they do not have highlight... and now they decided to removed it from Gmail too. If it is not broken, don't fix it :-/

Wow, I actually just called Google Apps support the other day for one of my domains to get downgraded back to free G Apps (I had unintentionally signed up for a 30-day trial of Business, and it didn't auto-downgrade, it just suspended my account). Bummer to see this though, I have multiple domains (As I'm sure many do here) and it's nice having a quick setup for one or two @domain.com's.

I hope they don't do this with Google Analytics next. Not just because it is a great product to get for free (I'm a massive fan vs other systems). To me it would seem wrong to come into the market @ free, wipe out or reduce much of the competitors and then expect everyone to pay up now they have a market dominance.

I use Google Apps for my domain but forward the mail to another account which means I see no ads and get hosted mail for free. I didn't even realize until this announcement that I'm probably a net negative and probably not the only person doing this.

Google apps for education is still free (I wonder for how long.) At least google is focusing on charging for things that are revenue generating, however for some education enterprise, that is debatable.

An alternative I use for educational customers is Microsoft Live Domains, also have some comercial domains with more than 50 accounts. But given this situation with Google I`m expecting Microsoft to do the same

Could someone explain me why a business providing a service should ? give it for free ? are google engineers working for free ? is google running its servers for free ? so yes , google is rich , but so is your electricity company , most gaz companies and most supermarkets. Do they give you stuffs for free ? "freemium" model is dead , in the future you'll pay for every service you use on the web.

It sucks that they've removed the free version; it's definitely a step up from managing multiple individual gmail accounts or creating a shared account of some sort. (I use one for one of the side jobs I do; it's a pain in the ass.)

However, $50/year for premium is really friggin good, considering that the cheapest alternative is $10/mo ($100/yr) for hosted Exchange 2010 alone.

I signed up my domain for google apps as soon as I registed it about 3 hours ago. Must have been one of the last ones - I noticed they had taken the small text link on the apps homepage away and I had to click through to pricing to get the free version. Obviously Google had kicked off their deployment while I was registering.

I'll be very interested to find out what role Guido will take up at dropbox. When someone like Matz/ Guido/ etc gets hired, what do their job responsibilities, for the community include exactly? Are language writer hires such as this purely symbolic?

Would someone like this be paid based on their fame and/or positive PR value, or would they be paid based on position just like everyone else? I've always wondered if these "programming celebrities" make substantially more just based on their personal brand, or if their personal brand just affords them the opportunity to have any job they so desire (but with the "standard" pay)

I wonder, will Dropbox still allocate 50% of Guido's time toward Python development?

Further, donning our tinfoil hats, is it reasonable to suspect that Google is phasing out the use of Python internally? I've heard rumors that Python is no longer permitted for new projects within Google; hoping some Googlers here can confirm or deny this.

A decade ago, Python being widely used in Google and the creator of Python being employed by the company was a big endorsement for the language. Now Python is quite mainstream. Actually, Guido was allowed to devote 50% of his time at Google for Python. Hope the good work continues at Dropbox.

Python has been a backbone of Dropbox since its early days as it allowed the startup to write code once but deploy it across platforms.

Can anybody elaborate on this? Is the argument that Python is cross-platform because everybody uses GNU tools on every platform, or are there other reasons why Python is more cross-platform than other languages?

Speechless. This is definitely one of the best decisions that Dropbox will probably ever make. Not only will this mean that Dropbox can hire other equally great Python developers, but as a company you can't get any more humbling than, "hey we hired the guy who wrote the programming language this site is based on and makes its money from"

Guido is an exceptional engineer as well, not just a guy who knows Python really well. The dude is seriously one of the rare gems in the community.

FWIW, Guido's role at Google wasn't specific to Python. He worked on real product teams and contributed much more than his Python expertise.

The article seems to think that Dropbox hired him for his thorough knowledge of Python, which probably had some role in the hiring decision, but I expect that the primary motivation was to acquire an excellent engineer.

It always strikes me as odd when something like this happens. Guido is still working nine-to-fives and Drew never has to work again. Can someone tell me why a brilliant person like Guido isn't worth a billion dollars? Have tons of fancy cars and a fancy house? Has to work for another company?

IMHO, Python would have been in a far better place if Guido and his team would have been getting paid to maintain and evolve Python, the language, rather than apps and platforms based on Python. I think in the early days at Google he was able to do so, later, appengine took most of his time.

Dropbox is still a startup - despite the substantial investments rounds - since it is still relying on external money (investments) to grow.

And of course this thread fills up with more of the same negativism trying to justify itself.

PG nailed it with the term "middlebrow dismissal". Which is a politer term than what I think when I come across it.

I never think to myself "Wow, what a realistic, practical, well-informed comment that generic negative feedback was. My respect for that commenter has certainly increased!" Particularly when it's the 500th example of the same thing in the same thread. Why do people do that? You can't possibly imagine you're adding any value in that case.

It's a growing problem here, and for me at least it's completely contrary to why I visit this site.

In the end, I don't care if the dude is capable of starting his own cable company or not. For one thing, I'm smart enough to know it's not my place to judge that. It's completely irrelevant what I think about that. I can wallow in my own mediocrity without feeling compelled to assign it to someone else.

But the value of the post to me, and to the site in general, is to consider the question. How would you go about starting a cable company? The question of whether any particular individual is capable doing such is COMPLETELY lacking in interest for me. I don't care about that answer, and I especially don't care about some random internet commenter's opinon on that topic. But the question itself is of interest to me, and I hope to others on this site.

If it's not, then maybe there's less value in this community than I'd hoped.

Sorry, a little riled up here. The continuing attempt at justifying the attitude is a bad, bad sign.

Imagine, as a thought experiment, that HN declared a do-over and took another stab at answering the question. What, specifically, do you want in the new thread that was not present in the old thread, and what do you not want in the new thread that was present in the old thread?

Some potential options:

a) Keep all the facts. Remove conclusory statements which suggest that this is a bad idea.

b) Remove the discouraging facts. Add new facts which tend to not be discouraging, such as "Monthly cable rates are high relative to many B2C products and lifetime values can easily hit the thousands of dollars, allowing you to spend a lot on customer acquisition." Add conclusory statements suggesting that this makes founding a cable company a good idea.

c) Remove discouraging facts about the cable industry, replace with to-do action items which accurately describe the process of how one would set up a cable company (regulatory approvals you'll need, who to hire, how much to raise, etc etc), and very carefully attempt to phrase "Convince investors to stake you with $X0 million" in such a way that it does not sound discouraging.

d) Remove everything about starting a cable company. Treat the question as a springboard about "How to disrupt the cable industry?"

I didn't take part in the previous thread, but I kind of took umbrage with the question. It's really hard to take a question like that seriously when it appears that the OP has put zero effort into researching the subject. How can you not be pessimistic about that? If you're not the kind of person that can do even the most basic research of how cable companies operate, then I don't have one iota of confidence that you would be successful building one. I'm not trying to be mean - just realistic.

I love this idea. Not as something to realistically help you get started, but purely educational. There's so much knowledge tied up exclusively among people "in the industry", that would be great to get out into the public.

I would love to know all the things involved in starting an airline. How are planes actually bought? How are mechanics actually procured? What kind of safety regulations are followed? etc.

I once read the book "Starting & Running a Restaurant for Dummies" just to find out how it's done. It's fascinating!

But turning it into a Wiki, for anything, would be so much better. You can have summary sections, that link to fuller articles, you can cross-reference how to hire a chef (both for the restaurant, and for the airline catering), and so on.

You asked for advice and received some great information. Just because the overwhelming advice is "This is not a great idea, you will need a lot of money to fight established, entrenched businesses" doesn't mean the advice was unhelpful.

Did you want everyone to sugar coat their opinions with "Yeah! Go get them tiger! We believe in you!", or did you want serious advice that can lead you to making an informed, rational decision?

If you were a CEO of a company, would you want your employees to tell you whatever you wanted to hear ("yes-men"), or would you want people who actually advised you and helped make an appropriate decision (even when it may be counter to your opinion)?

I agree there is a very large snarky dick contingent on HN who pollute potentially productive conversations with "What a stupid idea" type of comments. (Unfortunately I think this is a symptom of the human race, not just HN.)

However, there are some exceptionally bright people who hang out here who I would love to just brainstorm with--throw ideas out and see what happens. This site has an above-average level of these people I think.

I would love to see or find a group of people who are just interested in getting together and banging heads together and see what comes out. Even if I have no interest in starting a new courier service or whatever, I find it quite enjoyable to take the lessons and skills that I've learned and apply them in a new problem space. I especially love hearing feedback on those ideas from people who've worked in those spaces.

The most promising ideas are the ones that might sound dumb to someone entrenched in tradition.

I'd say there is a big difference between 'asking for feedback' and 'fishing for ideas'.

"I want to launch a new courier service. How would I get started?"

This seems much more in the 'fishing for ideas' stage and hoping you can crowd source the process of creating your business while paying nothing for the privilege. Most people aren't going to help you.

"I've started planning my new courier service. It will be mostly in the healthcare industry as hospitals are over-paying on delivery costs of fragile medicine. I've worked out some ideas on how I can better fit and transport fragile medicine. However, I don't know how I would get contacts in the medical industry to test drive the technology. Who should I try and talk to and does anybody know of any existing studies into this? Anybody work in the field of transporting fragile goods and know what certifications are required?"

This is much more in the field of 'asking for feedback' and is very unlikely to be brushed aside.

Many of my friends have a very interesting trait: pose any sort of problem in front of them, and they'll practically drop everything to work out a solution.

Trebuchets using only materials in the office. Conquering South Africa. Lesbian strip clubs. Realistic world domination. What the world would look like if D&D rules actually applied. AI for storytelling. Etc.

Regardless of how ridiculous the premise, they'll work on it until they have at least a plausible solution. Ignoring impossibilities and slim chances and jumping straight into brainstorming and crazy ideas is a lot of fun. I'd rather have more of those conversations.

The problem with this is that for questions like "how could I build a cable company?", answers don't help".

If you want to build a cable company, a payments system, a new government, or colonize mars there is no high level answer. There are hundreds, thousands, tens of thousands of sub-questions that need to be answered. Eventually so many that you need to hire people to ask and answer them, and you won't even know what the question was.

The truth is that if you really want to go to space, no one will help you. Not because we don't want to help you, but because we can't. We can speculate, and dig up research or law or economics---but at the end of the day, all you're doing is inviting us to help design your bikeshed. If you can break down the problem, figure out which questions to ask, what information you don't have, and who does, that person might be able to help you. By the time you've gotten that far, you're already on the verge of sinking your life into this---and if you want to build a cable company, that's what it's going to take.

I missed that question and would be happy to help with the technical aspects of starting a cable company (though I'd like to see content delivery decoupled from the infrastructure). I have 25 years of experience in the industry if he gets that far.

The hard part of the process (and I believe the limiting factor in a new company's success), is arranging the contracts with the content syndicators. Find my e-mail in my profile if you're serious.

>"I'm really disappointed in the universally pessimistic and generally unhelpful answers this question received. Some people pitched some interesting ideas and helpful analysis, but most of the replies reinforced the notion that Hacker News readers are predominantly male know-it-alls and on the average, a bunch of snarky dicks."

This would have been both more convincing and shorter without the anti-male bit. I realize that western media is pretty insensitive to this variety of sexism, but a quick test is to try substituting "black", "female", or "Jewish" where you've written "male". If it sounds offensive, then it probably is.

Criticizing what someone has made with middlebrow dismissiveness is one thing but pointing out that taking on a legally and commercially entrenched industry will be hard and here are some of the specific challenges is entirely valid.

I think the original question is worth picking apart if the answers are being criticized.

>Ask HN: I want to build a cable company. How would I get started?

> I want to build a cable company that centers around viewer types. Basically, it is my understanding that the majority of my cable costs centers around channels (like fox) that I just dont watch, if I wanted to build a system that let customers limit this, where would I get started?

The original question is worse (by a long way) than most of the answers. Most of the answers pointed out the major challenges or suggested better ways to achieve the vaguely stated aims although there were some useless dismissals.

The question indicates (possibly wrongly) a massive lack of understanding of the business "Basically, my understanding...[something basically right but oversimplified]". It is badly punctuated "fox" rather than "Fox", 'dont' rather than "don't". The question also fails to really be clear about what he wants to do. Does he really want to run a cable company maintaining wires in the ground and boxes in homes? Or is this mostly about securing more favorable and flexible content deals? Or is there an implicit assumption that he needs to own the cables to get the deals? What scale does he want to start on? A small town/city or national?

I didn't see the original question while it was active but if I did I would have pointed out how hard such a business is to break into especially if top grade content is required due to the maze of exclusivity contracts and the value of them which means massive amounts of money are needed to make the sellers choose non-exclusive options. Even companies like Microsoft and Sony with massive deployed platforms (Xbox and PS3 in particular) need prolonged negotiations to get any content and aren't anywhere near being able to offer full cable replacements and even with their own content Sony can offer only what is not exclusively licensed elsewhere.

However that doesn't mean that the video space can't be attacked as Netflix is showing but it will in my view need to be an indirect attack that builds audience until it can compete head to head for the major deals and content as a viable distribution platform and pricing model. Note that when the competition really starts costs may rise not fall as the platforms will compete for the most important content potentially driving up content prices for all the distribution platforms.

Everybody already knows it would be great if we could get cable speed and reliability without actually laying cable. Building a business around that is easy. The hard part is actually achieving that.

Asking people for the miracle upon which you'd like to build a business, that anyone could build given that miracle, isn't a request for constructive feedback. At its absolute best it is a request for an IP handout.

An interesting rant. I've observed that problem solvers also follow something like the inverse power law, which is to say it takes a certain kind of person to attack large problems and as the problems get larger the set of people willing to consider them seriously decreases logarithmically until it asymptotically approaches zero.

In an analagous fashion the "quality" (using karma as a metric of quality) of comments on HN are also quite high and drop off steeply approaching zero. This is reflected both in the comments, and in the karma distribution [1]. The trick is to keep the noise floor high enough for quality reading and low enough that you don't miss out on new people making great comments.

So if someone says they want to jump off a cliff naked, should we all applaud and encourage them? I think it's better to tell them it isn't going to work out well and why.

There was some interesting analysis and procedures in that thread about how to go about it, but the best post for the OP was really about how, as a company, he would be paying to license certain channels, and if users only subscribed to one or two instead of all of them, he wouldn't make enough money to keep in business.

As a user he can whine and complain all he wants about a la carte being rare, but as a business, he has to make enough money to keep running and he didn't understand that. Pointing out big problems in his plan is doing him a favor because he can then change the plan or address them.

There is nothing "rude" about asking HN to chime in on your exploratory-stage ideas, especially when they are uncommonly ambitious and not your average badgeville startup ideas.Face it, Foursquare is not going to fetch you a mercenary if you ever needed one.

Why can group-mulling of ideas, be a terrific endeavor?

1) It can stress-test your idea and expose the glaringly gaping holes you might have overlooked, in the very fashioning of the prospect itself.

2)Some ideas may not lend to "socialization" because of their inherent nature. Helps to have them flagged -- if not decimated -- before you even begin. Eg: A startup seeking to disrupt the litigation law market isn't very readily socialize-able as one targeting the fitness trainer market.

3) The group-mulling process need not be just one-way beneficial. The "mullers" stand to benefit from having their horizons broadened too. They might for once realize, that there are entire industries -- non-glamorous but nonetheless high gravity, real-impact ones -- waiting to be disrupted beyond the Mayorvilles and Filtergrams of the valley.

I didn't participate in the discussion because the question was so vague. "I want to colonize Mars. How would I get started?" The same way NASA got to the Moon -- they didn't log onto ARPANET and ask around, they got to work.

I didn't read any of the comments in the original thread, but I can imagine why many of them were negative. Here is the question:

' I want to build a cable company that centers around viewer types. Basically, it is my understanding that the majority of my cable costs centers around channels (like fox) that I just dont watch, if I wanted to build a system that let customers limit this, where would I get started?'

This, to me, reeks of laziness. The OP would have done better to say 'this is what I've done, research, and/or these are the ideas I have. Am I right, wrong, or close?'

So problem #1 is that, from the looks of it, the OP didn't do any research before asking the question.

Problem #2 is that the question is a lazy one.

I can guarantee you that Elon didn't ask these type of questions, but instead presented lists of assertions and hypothesis which he was prepared to test and/or validate.

This reminded me of Richard Branson (the Virgin guy) and a post by James Altucher. I think James' point is that to do something huge, like start a new airline, you always have to find the next step. Pretty basic really, break a big task into smaller pieces. But worth a read nonetheless, it really goes into OP's question.

"Q: How do you know when you're thinking too big or aiming too high (if that's even possible)?"

"A: In the mid-90s I had an idea that lasted about the amount of time it takes to drink two beers. I say this because I had the idea at a bar and it was quickly squashed by the two friends I was with

(...)

My real problem was: I didn't have confidence. And I didn't know what the next step was. In retrospect, I should've written down my idea, written down ten ideas for possible shows to launch with, and started pitching TV companies to get someone to partner with me on it. That would've been simple and not taken too much time before there was some payoff."

> However, why does everyone assume that the inquiring mind is an idiot?

Because the inquiring mind did zero research and asked a question so broad and so vague that it was impossible for anyone to answer it without months of work and very specific knowledge.

To quote Carl Sagan: "If you wish to make an apple pie from scratch, you must first invent the universe." I don't think anyone was wanting to sit and spend hours explaining to the author how to do something incredibly difficult and complex without the author putting anything forward. Why not just take that knowledge and do it myself?

I definitely agree with this article. We shouldn't be pessimistic about someone's grand vision. However, if you really wanted to start a cable company you probably wouldn't be asking hacker news on how to start a cable company. You would already have other resources that you can tap into and get started on it.

Elon Musk started from very low hanging fruits for him, whose success enabled him to move on with more ambitious goals. This model is easily replicable, so if you have ambitious goals, first build capital with something easier. Which, I think, most of people here are doing.

But I do always find myself needing to point out: Hacker News is the most constructive and positive community on the internet. Yes there are lots of dicks and know-it-alls with "why didn't they just do this?" syndrome, but you won't find another anonymous discussion based community of this quality anywhere.

While I do find it interesting to read about the work involved to start such a company - I believe most here would be even more interested to read what steps it would take to start a disruptive company in such a market.A lot of these markets are either ready for disruption or have recently been disrupted. I would love to read an in depth-article about the work involved.

I think the issue isn't about how feasible the start up idea is in general, but how feasible it is for YOU. The comments may have been directing you to another idea for this reason.

Using your example of Elon Musk, you can see even his start ups have a progression of ambition. That is, space travel and electric car companies are much less feasible than a payment system or news site.

What is the difference between Musk and you today? Simple. Investors trust him with large sums of money since he has led successful companies before.

There is a reason YC companies generally have low capital requirements. That is the lack of track record of the founders (and that business experience does matter).

For your cable idea, the path towards that goal may very well be starting other companies first. Or be an industry insider who has the experience and connections to convince the incumbents to invest in you (like Pandora).

This is exactly the mentality we need to see more of. Dismissing good ideas is not just unproductive but actually destructive to the overall progression of thought - which is why we're here in the first place.

I didn't see the thread either, but I like this post. I think the best way to go about starting something large and ridiculous is to tell all your friends and family you're going to do it, to the point where it would become embarrassing if you didn't even try to make it a success. Friends and family are the best at calling you out on not living your dreams.

"Hey Gaius, weren't you going to build a Cylon detector? You talked about it for months"

"Yeah, but it was too hard and I didn't know how to start"

"You suck"

A more practical benefit of telling all the people you meet is that one of them might know or be associated with someone who is connected with the industry.

Sure, the internet can be a cynical place, remote communication seems to wake the worth in a lot of people. At the same time it is full of wonders. Maybe I can invite you to join a discussion about how it could impact in political decision making? Click along: http://news.ycombinator.com/item?id=4891571

LOVE this! I'm new to programming and it's the toughest thing to answer. How or where do I get started? I have some great ideas but not much in the way of skills to get it done. I know we all have different levels of "figure it out" skills but it's always helpful if you find someone who can at least show you how to open the door.

You really shouldn't declare a variable without using var - can lead to all sorts of scoping problems.

Oh, good catch! You should submit a pull request to fix this

+<mass of span elements>

There's a couple of redundant span elements here, when you get time, you could optimise this

+ <p align="center" dir="rtl">&nbsp;</p>

It's great that you've made sure that non-breaking space is read right to left, your readers would have been screwed otherwise.

+ $bind_port_p="IyEvdXN...<base64 encoded string>";

It's going to be hard to comment on these Base64 encoded C and Perl programs in their current form. That daemon() function call is going to cause problems on a whole bunch of non-Linux architectures. Solaris doesn't have it for example. You should definitely consider supporting IPv6, you can't just assume IPv4 connectivity... You're also missing a return at the end of main().

+$auth_pass = "fe3f6d96a1ee06bc5415a5c05540c7a8";

1911990 is not a good password. Your birthday?

Let's hope you didn't use that for your email account, lovestory8976@yahoo.com

This attempt is blatant and obvious, but what about a more serious attempt where you first establish some credibility with a couple of "good" PR that fix major problems and then add a tiny little backdoor that loads code from somewhere else. Distribute the relevant code over a couple of commits and you might just slip it in.

With Githubs ease of merging and automatted testing by Travis, it's easy to forget that changes may be actively malicious and not just buggy.

Edit 2: So if I understand correctly, OP tried to hack into a website... by submitting code to github. I was confused at first because that would have been (very) wrong way to "hack", but as it turns out, that is indeed true. And rest is about the code he/she used. It seems to be auto generated in some wysiwyg html editor that uses old html.

In case anyone's curious and got down to the embedded YouTube video in the code, it's an Arabic-titled video of a screen recording of a Facebook video (further evidence of the author's technical prowess) of two Israelis in a place undergoing rocket fire from Gaza... definitely recent as well as they say "Where's the iron dome!?" in Hebrew. It's about a minute and a half of the rocket sirens blaring and them hearing rockets landing in the distance, screaming out of fright/being startled when they do.

Pretty disturbing stuff, to say the least. Combined with the english text about the Zionists leaving Palestine, I just wanted to shed some light on the intention of the defacement.

But if I ever need a jury of my peers to audit my coding style to see how good it is, now I know what to do - a pretend-attempted-defacement is bound to be more effective than finding some place on the net to ask 'Is this proper idiomatic javascript?'.

I can't understand how did someone capable of understanding and doing a git pull request can produce this kind of "code".

Or how did he think that he can pull this thing off? is there a "10 ways to hack a website" where a git pull is one of them?

The fact that there is a code snipped a tutorial on "How to Create a Website With Notepad" and the whole thing seems like it came out from an old WYSIWYG editor, the thing obviously was made by a script kiddie.script kiddies uses git now? wow

What if they are palestinians "being bombed back into middle ages"[1] or from other arab country that provides 1/100th of the opportunities we have to know `how to be a pro hacker`, a foreign language or, say, an expert modern coder that know all the little beautiful standards?

The idea being that manufacturing was pushed abroad due to the appeal of cheaper labor, without a deep consideration of hidden costs or the overall context of such a transition.

GE, which is featured extensively in the article, actually managed to reduce manufacturing costs by bringing the fabrication of a water heater and other appliances back to American shores â€" largely due to the faster loop and better communication between designers, engineers, and laborers who all speak the same language and are in the same factory.

The only way I can see this making sense is if it is an assembly plant rather than a real chips-to-finished-product factory.

Why?

The advantage you have in these cities (almost literally) in China dedicated to manufacturing is that almost the entire supply chain is local and very finely tuned. This is particularly true for operations that might do work for companies like Apple.

The PCB manufacturers, assemblers, chip makers, connector manufacturers, LED manufacturers, display manufactures, plastics and sheet-metal manufacturers and more, are all centrally located. If not, they are within the proximal geographic regions.

The same is true of qualified workers. Need 100,000 assemblers in a hurry? No problem. Technicians, engineers, managers, etc. Lots of them and easy to hire within days of your requirement.

In sharp contrast to this, the supply chain anywhere in the US is most-definitely not localized and highly fragmented. Virtually nothing you are going to use in electronics manufacturing is made in the US. That means that rather than your LEDs being a few hours away by truck they are three weeks away by boat --from China.

In terms of mechanical components, such as screws, well, yes, they are available in the US, of course. The problem is that they will cost more. No question about it. Because our industry, due to the need to survive, has had to focus on market segments that can pay a premium (military, medical, etc.) you can pay through your teeth to get anything made here. That's just the truth.

In terms of machining and bending metal or injecting plastics, well, it depends. If you are dealing with a unionized operation, forget it. Costs will be ridiculous. Plastics, in very large quantities, can be reasonable here. Punching and bending metal or machining metal could be plausible at a very large scale and with a very finely tuned factory.

Let's not add regulatory and tax issues to the pile.

Because of all of this and a few more data points from first-hand experience manufacturing in the US, my guess is that Apple is going to simply import pre-fabricated modules assemblies and parts. They'll have US workers bolt them together and test the finished product. You can slap a "Assembled in the US" (and maybe even "Made in the US" sticker on it and feel good about it.

Remember what Steve Jobs told Obama about manufacturing jobs coming back to the US. I don't think anything significant enough has changed since then to invalidate his statement.

I suspect Apple will replicate what they are already doing in Brazil, where Foxconn locally built a factory to manufacture specific Apple products to the local market.

Note two things: 1. I suspect this would be a Foxconn factory, not Apple (note how Tim Cook says "we'll be working with people"); 2. The total investment of the Foxconn factory in Brazil was 5 times bigger, so I suspect that Apple's $100m would cover only a fraction of the total investment required to build a factory in the US.

In terms of investment in manufacturing, $100 million is chump change. Fabs are an order of magnitude or more greater. The amount is approximately that required for a middling "power center" shopping development. Or constructing a handful of Apple stores.

Not to be cynical, but I suspect that local, state and federal tax subsidies will yield a positive ROI on the $100 million. This looks like pure PR.

Ironically the rest of his statements and the article as a whole are extremely vague. $100MM at Apple scale doesn't seem like very much and since (from the article) they won't be doing it themselves I'd love to hear more about where that money is going.

I wonder if Apple could build a factory somewhere touristy in the US (California, or maybe near Disney World, or something), designed specifically for tours (like the BMW factory and European Delivery center), and make enough from tourism (either cash or "brand value") to overcome the costs of production in the US.

I'd pay $20 to see the modern equivalent of the NeXT factory for 30 minutes, from an overhead viewing gallery.

Obviously Apple wouldn't want to reveal trade secrets, but I don't think there's much secret about how Apple produces the desktop and laptop products.

I wonder how much this move was precipitated by Google's purchase of Motorola. Google produced each generation of nexus phone with a different manufacturer. This allows Google to learn the best practices of each of them. But eventually Google will produce the nexus devices themselves through their Motorola facilities, allowing them to iterate fast and produce cheaply. Apple sees this coming and figures out they also need to produce their devices in-house.

I'm sure there are multiple reasons for this move and labor costs are almost certainly the biggest factor. But one aspect that I haven't seen discussed but which may be relevant is that companies increasingly turn to the ITC as a patent enforcement lever. The ITC is a US agency which can effect import bans of products found to infringe. Obviously a great workaround to an otherwise crippling import ban decision would be having a US based assembly network.

Now the big arena is mobile and this announcement is about Mac lines but it's not hard to believe that this is a tip of the iceberg investment and that Apple might increasingly move to a system of using Foxconn's non-China factories around the world. Apple is partly financing the Brazil plant for Foxconn IIRC.

Another aspect that hasn't been discussed is the opening around the world of rare earth mines (reopening in the case of the California mine). Part of the reason "those jobs [weren't] coming back" was because China's rare earth's monopoly increasingly was reserved for Chinese made products.

Well, once robotics takes over a large portion of Foxconn, it won't make as much sense to hire low wage employees overseas, thus why not have the robots building/assembling the products as close to the consumer as possible?

Apple spends a lot of shipping too, so I'm sure it's a balancing act between cost of labor/automation/shipping.

> We decided being more transparent about some things is greatâ€"not that we were not transparent at all before, but we've stepped it up in places where we think we can make a bigger difference, where we want people to copy us

I'm willing to bet that the reason they're doing this is exactly the opposite. Integrating their last-stage manufacturing would be a great way of squashing those pesky product leaks from contractors.

Wonder if this is the first step towards a move across all products. It'd make sense to start with a relatively low-volume, high-margin product like the Mac.

What extent of the production will be here though? Are they just assembling motherboards, chassis, and displays all together that have came from overseas? I don't think Apple would be so willing to give up the advantage of having all of the tightly knit infrastructure available in China just for political reasons.

Possible side effect: Change in business privacy? Say Foxconn are the contractor, a product leak in the factory's first year might be a lot more chase-able than one in China, but also a lot more inevitable. Will there be robot-only zones for the private parts, or more lawsuits?

I think they will be making product chasis and frames in the USA. Think about it.. If suddenly there was another uprising at some contractor in China they would not be making Apple products, they would be making commodity parts like motherboards and memory that are simply components. People would identify their Mac mini or iMac as being made in the USA and some parts from China in it.

Also there probably is not much savings going on by making these parts in China. The expensive part would be if you were making your own motherboards and chips in the USA. Apple does not need to do it, and gets good PR in the process.

It's happening. Wages are rising elsewhere, while US wages remain stagnant, thus making domestic manufacturing for high-end goods more profitable than manufacturing elsewhere again. I've been predicting an eventual return of manufacturing to the USA for years now. It'll be a slow process, but it will inevitably happen - the only questions are when and how long.

It's interesting how a cultural artifact belonging to a museum is better preserved in software. It is unlikely anyone will put the hardware in a museum, let alone getting it operational and allowing someone to play with the exhibit.

Many of the better museum exhibits tend to tie in things to people's daily life in the present. I searched an anachronistic term, "game of thrones."

How nostalgic for me, reminds me of my days as a junior Data General field engineer - Nova 3, Eclipse S/130, S/140, S/200'S + Phoenix and Gemini 10+10 and 5+5 toploaders.

Admittedly this was their Dasher D200 (current loop) and LP2 era, but we did sometimes bootstrap DTOS (Diagnostic Tape Operating System) from paper tape if all else failed. We even had a couple of ancient punched card readers in stock for certain oddball customers, just in case.

Makes you realize in 1000 years there will be museums where people will go to see the early ipad and android devices and wonder how anyone got anything done with them, and stare at 3.5" and 2.5" hard drives with their ridiculously tiny 1TB capacities. Hmm, maybe even in just 100 years.

Just having the most fun playing a round of "thermonuclear war" (well, actually, it's mostly playing itself), better than any (AAA) title I have been playing lately.This one is going to take forever. And I am glad that we moved from printed output to graphical displays, the amount of forests we would have had to cut down...

Watching the tape animation makes me think that these guys have never actually seen a magnetic tape unit before. At several points, their animation shows the reels spinning in opposite directions. An actual tape would snap if you could somehow get the reels to do that.

(Disclosure: I work for Google, but not in anything to do with maps. These opinions are my own.)

I've been to this national park before. It's a beautiful place, but not one you want to get stuck in unprepared. The temperature estimates given by the police are accurate. It gets damn hot out there. The roads are gravel at best, and often just packed dirt or sand. You can very easily get bogged, if you don't know what you are doing. I got to a point where the road became unsuitable for my car, and I had to drive backwards for nearly 100 metres to get out as there was nowhere to turn. I would be concerned for inexperienced drivers with two-wheel-drive vehicles.

Map data is serious business, and this particular case is an egregious error.

Recently drove a 4,500 km trip in Australia, including near Mildura. We relied on two iPhones, one with Google Maps and one with Apple's newer version. For the most part, we relied on Google's maps and they never failed. Occasionally, we switched to iOS 6 and had a few different problems including roads simply not existing. After a while, we'd watch the iOS 6 directions just for a laugh.

Some of the stretches further East of Mildura run 100-200km without petrol stations and supplies. A number of the national parks in those areas are unmanned and irregularly visited, and there can be long periods even on sealed roads where you might not sight another car in an hour of driving. For one stretch (in and out of Mungo National Park), we stocked up on a lot of extra water as a precaution and notified family of our plans. Having an infant with us, I was very conscious of how quickly the temperature can rise once the air-con is off.

Can easily see how people might blindly follow maps/GPS and end up caught out. Especially when you're relying on cached phone maps data if your connection drops, as it does frequently out there.

I think I can explain this one - if I'm right, Apple already has the correct data, but is using it incorrectly.

One of the Apple Maps data sources is GeoNames (geonames.org), a free data source available under a Creative Commons Attribution license and also used with the search on OpenStreetMap. If you search for Mildura at OpenStreetMap (openstreetmap.org) you'll get the correct location of the town, but if you scroll down the GeoNames list you'll also be offered "Mildura Shire" as a search option. Click on that, and the pointer moves to a location similar to the Apple Maps screenshot, about 70km away from the actual town. It seems that's the location that Apple's search has chosen to prioritize, over the actual town.

Mildura Shire is listed in GeoNames as a "second-order administrative division", so it's probably similar to the middle of a county or council area.

This isn't a problem with datasets or incorrect data, it's a problem with how Apple is searching & prioritizing the correct data they already have. Choosing the "town" instead of the "second-order administrative division" would be appropriate here.

For what it's worth, GeoNames already prioritizes the town of Mildura correctly in their search engine.

Interestingly enough, today I was having a conversation about whether or not Maps had made any real improvements. You know, the kind that might compel one to finally upgrade to iOS6 or even consider upgrading to an iPhone 5. I asked a few people and nearly all of them expressed concern about being able to trust Maps. News like this doesn't make one feel better at all.

Frankly, I don't understand Apple's decision in the context of the idea of being customer-focused. In other words, if you, as an organization, make decisions for the benefit of your customers --or, at the very least not to their detriment-- how can you justify pushing out Maps and not keeping Google Maps on there?

OK, I get it. It would have cost more. A lot more. Fine. That's your problem. Pay Google for another five years exactly because you care about your customers. At the same time, put out your own Maps app and --funny enough-- compete on the merits of the app, not the hype.

If in five years you can't turn Maps into an app that people will choose over Google Maps, then, well, why are you in the mapping business in the first place?

Especially in remote areas/national parks/widerness areas. Hate to say this is "common sense", but really it should be. It really has nothing to do with Apple, the iPhone, or iOS. Its quite the opposite, there are errors on all sorts of digitized maps. Most people are not aware or the vintage of the underlying mapping data (pre-digitization) and the variances of map-set data even amonst variant databases of "real maps".

I had to go to the main emergency room in Oslo last week. iOS6 couldn't find it, or even the correct street. Looking at the map in detail later, it didn't even have a building at the correct location. I was furious.

So, no, it's not good that apple's map data is inaccurate. But the title might be misleading.

The bottom line is if you're driving into a wilderness area like a national park, you should not be depending on your cell phone for your own safety. One of the first things they tell you if you read the pamphlet outside of a hiking area is: do not depend on you cell phone.

So sure, the map data is inaccurate. The more dangerous thing about this is that your battery has a finite lifespan. Also your signal is not guaranteed.

You need to bring water, you need to bring supplies. You need to bring warm clothing.

I heard similar stories about Google Maps a few years ago, people getting stuck in US desert/parks because the GPS is incorrect (and also reception wasn't good back then, but that might have improved by now maybe).

At least this story is hitting the HN front page. Because the park rangers were hitting Google's traditionally deaf customer service ears (which must have been super-frustrating because people had already died and/or gotten into life-threatening situations).

I lost the link to that article, sorry. If I happen upon it, I'll post it here.

On the one hand I suppose it's people's own fault for venturing into dangerous terrain unprepared. But on the other hand, what's the use of having GPS Maps when you don't know when it's trustworthy or not? Great effort by OpenStreetMap (read below) of cross-checking their own maps for consistency with competitor maps, is at least a step in the right direction.

I'm intrigued that the police let this happen five times, and then put out a press release. After the second time in two weeks, I would have put up a sign: "This is not the road to Mildura. Go back and turn left at the highway."

It is quite common for tourists to get into trouble here in Australia due to the differences in climate etc too.

FYI, when you travel make sure to bring extra clothes, water and food. Australia is a massive land mass and it is hard to fathom for people form a lot of countries that it can be a very harsh place for the unprepared.

The map failure is just another example of where apple's design principle cannot be blindly applied to every product. Apple's top-down approach on software design is expected to fail on Maps. Maps put hard requirement on data, bottom data, nothing to do with your leader's vision. Apple's way out of this is not to engage user input to add missing data or correct data errors --OSM tried that for years, the most accurate data still comes from semi-professional survey-er.

Look at other companies that does map, google map started out using Tele Atlas, NavTeq serves yahoo, bing and mapquest. Let's face it, spatial data cost money to collect and even more costly to update/maintain. Nevteq and Tele Atlas are gigantic companies for serving basic spatial data for a reason.

I guess apple didn't do sufficient data QA before saying, "hell yeah we are going with OSM where every big player is going with commercial data."

Without a solid baseline data, any fancy pants software development would just evaporate in air.

I have to say though, the GUI for apple map and functionality has very high usability. Apple just need to adapt a different mindset when dealing with data-dependent applications.

(disclaimer: I am a PhD student in Geography with CS background, did my share of processing spatial data for the last 8 years)

I live in Australia, and while I am not a fan of Apple or their products,I have to say that I'm surprised anyone in their right mind would go bush with just one map. Electronic or paper, maps have errors. Don't bet your life otherwise.

Even before the iOS 6 maps debacle, relying on GPS for guidance in places like National Parks was pretty dumb. There are multiple stories of people getting stranded (and even dying) when relying on maps for places like Death Valley. Often times, even if one is in a suitable vehicle, knowledge of the environment, what to do in case of a problem, etc. are essential necessities in the case of anything going wrong.

A buddy and I travel the back country of Death Valley regularly and the stories we have heard, the vehicles we have seen trying to go places they shouldn't, etc. are just amazing. At a certain point, however, common sense and personal responsibility need to be considered.

Out of couriosity, I navigated the Kalahari with both a map and a TomTom. I was quite surprised to find that the paths were quite accurate - but shifted by roughly 5 kms. So I could use the maps, but not the navigation capabilities of the system at all.

Oddly enough Google Maps got me lost before, a couple of times. Lets stop assuming Google Maps was all that and a bag of chips, it did have issues at times.

I think it's because I've done long journeys powered by digital maps but I always made sure that there was an agreement level between two, or that it looked sane on a tried and tested road map. As good as they've gotten for the most part they're still prone to bugs, or to lose signal. Paper maps are a car essential still.

I love my iPhone, MBP, iPad, and how generally apple has raised the bar on software quality. But the maps app is truly intolerable - it's the most unusable thing apple released since ping, maybe even more so.

It looks like this issue was caused by Apple mistakenly marking the centre of Local Government Areas as 'cities', in Australia at least. See example at http://imgur.com/qlciM for an example from Perth; Cambridge and Vincent aren't suburbs, and the others are in the wrong spot (Joondalup CBD is on the wrong side of the freeway)

You should always have a road atlas in your car. Batteries run out and network connections may not be reliable, you will always have access to a paper map if you have one. If you have an atlas you can always navigate yourself out of any problem (diversions, heavy traffic, accidents closing roads etc.) and they are a good sanity check of whatever your GPS or map app is telling you. Better yet, check your route in the atlas before you set out and you will know your route better, and be able to react quicker to any unforeseen problems.

I think this is an example of seeing the past through rose-coloured glasses.

Yes there was Flickr but you could discover photos. Thing is, Flickr is still there and you can still use it. What's clear from this is that Flickr didn't (and doesn't) cover what is the use case for most people: sharing photos with a limited group of friends and family.

Technorati? Honestly, I think this is an example of living inside a very small bubble. I'd honestly never heard of Technorati until long after it had waned.

I don't agree that the monetization of the Web has degraded the value (to the user) of links on sites other than links on sites aren't the primary discovery mechanism like they used to be, which is actually a good thing (IMHO).

> In the early part of this century, if you made a service that let users create or share content, the expectation was that they could easily download a full-fidelity copy of their data, or import that data into other competitive services, with no restrictions

This is only true to a limited extent IMHO. The primary services for creating information 10+ years ago were email providers. Because Web-based mail was a latecomer, services like Yahoo Mail and Hotmail grew up in an era where many people used Outlook, Thunderbird and other desktop email clients so they had to support POP3 (and later IMAP) and you could use those services to export your mail.

But that isn't the same as designing your services for interoperability. That was an unintended consequence.

As the idea of "your mail, everywhere (you have an Internet connection)" became dominant, so did Webmail. POP3/IMAP became less important.

Again, I consider this a net positive change.

> In the early days of the social web, there was a broad expectation that regular people might own their own identities by having their own websites

This I disagree with. Having your own domain and Website 10+ years ago was pretty unusual. Administering your own site is not easy, particularly as malware became more prevalent. This has declined because no one wants to run their own Website (or email server for that matter) because it's a crazy amount of effort for very little real gain.

The only real problem I see with the present state of the Web is that Facebook wants to own all your data. It wants to be your identity. It wants to be your Internet. That's bad. It's bad for the Web and bad for consumers. But honestly, I don't see it coming to pass. Facebook is just as susceptible to disruption as so many behemoths that have come (and gone) before it.

10+ years ago Microsoft dominated your computing environment. Many couldn't envision a future that would break free of this grasp. In a few short years Microsoft has diminished their control of your computing experience in ways few could've predicted. I'll just leave this as an example of the danger of extrapolation:

I agree with Anil 110% that the Web he's talking about was, in many, many ways, a Better Web than the one we have today.

The problem is that it's worse than the one we have today in the only way that most people care about: it's harder. To participate, it expected you to know how to do a bunch of things that seem trivial to tech folks but frighteningly complicated to everybody else. You had to buy a domain. You had to choose a Web host. You had to know how to connect the domain to the Web host. You had to choose the right software to do what you wanted to do. You had to install that software, and configure it properly.

The reason hosted services became popular is because they let you skip all that stuff. You fill out a form and you're up and running. Someone else worries about all that other stuff for you. This makes those services accessible in a way that the Web of 2000 was not.

Of course, to get that accessibility, the hosted services make you give up a lot of things. You lose access to your raw data. You lose your privacy. You lose the ability to change vendors if the one you're on turns evil.

But to non-technical people, those losses aren't obvious. They don't understand what they've lost until losing those things turns around and bites them. It's like DRM: people don't understand why DRM-encumbered music downloads are bad until their iPod dies and they want to move their iTunes-bought music to an Android phone. "What do you mean I can't do that?" is what you hear the moment the penny drops. But before then, they don't understand the risk.

This is what will need to be overcome to make tomorrow's Web like yesterday's was: it'll need to be as easy for people to use as today's is, or you'll need to educate the entire world about why they should put up with it not being that easy. Otherwise people will keep on blindly stumbling into the heavily-advertised walled gardens, not realizing that's what they're doing until the day they decide they want to leave, and can't.

Maybe I have changed, or maybe the Internet has changed, but I used to meet people on the internet. I used to make friends online, and some of these friendships gradually mutated into "offline" friendships. There used to be message boards, IRC and web chats where people would talk, form groups, become friends or enemies.

People used to have blogs on livejournal or other services, some were trying to create content, write interesting posts. I met a lot of new people through that medium too.

But now everybody is locked inside the narrow bubble of their own social network. People don't become friends on facebook - they usually "friend" their IRL friends. You can't fit a good meaningful post into a tweet. And you can't have a normal discussion without sane comment threads like on livejournal - and I haven't seen that on any of the popular social sites.

There is an example in there of how creating a single sign-on service in 2005 being "described as introducing a tracking system worthy of the PATRIOT act". That was years after this kind of thing was considered a problem, however, and it was somewhat rightfully so, and I believe the real story is that things actually got "better" as we came to understand these services more. I am not certain things actually got worse over the last ten years: in some ways they really got better.

Going back to 2002, Microsoft had been working on "Hailstorm", which was a very poorly chosen name for something that people rapidly became afraid of ;P. It was later renamed to "My Services", but it included Microsoft Passport (yes, this is mentioned in the article, but I don't think it is given enough weight), a single sign-on service provider that Microsoft was encouraging other websites to use. It would provide details about you, including your e-mail address, to the sites you connected with.

I had remembered a bunch of people being angry about it, so I did a Google search for "Microsoft Password mark of the beast", and came across an article written at the time in some random magazine called "Microsoft's Passport to Controversy -- Depending on whom you ask, Passport is either a useful consumer convenience or the mark of the beast".

However, it should be noted that one of the fears at the time was not "man, vague centralization is bad", it was "omg, Microsoft doesn't just want this service to take over the web... they want this service to take over the world". Now, of course, you read me saying that, and think "ugh, stop with the rhetoric: that's just an example of people freaking out about something we find common-place; that's what the article is about: did you read it? ;P".

But... it was actually for real. Microsoft was lobbying to make Microsoft Passport be the new US National ID system, and it wasn't just a pie-in-the-sky goal... they were lobbying to make it happen, had the ears of the right people, and were making serious progress on it. For reference, there was an article written about the situation in the Seattle Times with the title "Feds might use Microsoft product for online ID".

> Forget about a national ID card. Instead, the federal government might use Microsoft's Passport technology to verify the online identity of America's citizens, federal employees and businesses, according to the White House technology czar.

> On Sept. 30, the government plans to begin testing Web sites where businesses can pay taxes and citizens can learn about benefits and social services. It's also exploring how to verify the identity of users so the sites can share private information.

I thereby feel the need to note that, even as late as 2005, if you were going to start talking about building the world's next best "single sign-on" provider, this is what you were being mentally compared with: yes, the one service mentioned (TypeKey) ended up having "much more restrictive terms of service about sharing data", but it is looking at the past through rose-colored glasses to think that things have gone downhill.

Let's put it this way: can you seriously imagine Facebook or Twitter ever being considered as the official login system for the IRS? I can't in 2012, but that was the honest-to-goodness reality of "the web we lost" from 10 years ago. At some point, in the last 10 years, it became more, not less, clear to everyone that this kind of service needed limits. There was backlash in 2002; but I believe it was much more fringe-concern than it would be now in 2012.

> Yesterday, appearing at the conference, Gates reiterated the goal, saying he expects governments in many countries will find it difficult getting to "critical mass" with authentication systems they develop on their own. He said some governments may opt to use companies such as Microsoft or America Online as "the bank" that registers people for online usage.

I think it's rather funny when people talk about the, "social web." Before the social graph, technorati, and flickr there were newsgroups, email lists, HTTP, IRC, etc. The Internet itself is a social tool. Perhaps the term refers to some epoch of which I am not aware but it seems to me from a big-picture perspective that we've only narrowly improved the experience since Eternal September.

The "walled garden" networks will always strive to find their value in lowering the barrier to entry for new participants on the web. Facebook makes it super easy to share your photos with your family and friends and passively update them on the minutiae of your life. Twitter does the same thing to large degree in a more public fashion. Tumblr, Instagram, Pinterest... all of the same zeitgeist: user experience.

But the cognoscenti are certainly aware that the web is the sum of its parts and walled gardens are antithesis to participation within its ecosystem. However the problem is and has always been participation: there is no single sign-in, no simple user experience, no common parlance for the mainstream to absorb. We got about as far as blogs and stopped there once MySpace, Facebook, et al took over.

I'd prefer a return to the roots but I think we'll need software and services that provide a better user experience and product-based focus rather than the service-oriented approach that has become popular.

Funny that he says all this then has a Facebook comments box at the bottom of the page.

Don't get me wrong, there's nothing wrong with that IMO- people are far more likely to have their real names on Facebook, and thus leave sensible comments rather than total drivel. But it makes a point that he doesn't include in the article- sometimes these centralised information stores can be useful.

I think Anil missed something in his allusion to AOL. There was an Internet before AOL that a few of us were actively using. There was a thing called â€śthe webâ€ť that some folks were toying with while the masses toiled in Prodigy and AOL. Were those services bad or evil? No, but they accelerated the onramping of the next generation of Internet adopters that then quickly moved to the wild and free Web. With the development of the web came all sorts innovation and novel services that brought order to the often chaotic web.

We are on the same onramp now as we were in the late 90's. Facebook, Twitter, et. al. are just another stopping point to whatever comes next. We lost some things along the way, we abandoned some of our anonymity, and in some ways our freedom and experience suffered. But we have also gained tremendously in the decade since. We have smartphones with apps that guide us to cool places and discovery new experiences. We have apps that make our shopping experiences easier and cheaper. We have apps that let us express ourselves in sounds, pictures, videos, text, and to share those expressions of ourselves to the world in a few clicks. We can find any number of experts and sites that offer assistance without flipping open phonebooks or blindly Googling the world.

Yes, we lost something. I also agree that we have forgotten some of the earlier values that made the web such a joy. We got enticed by free apps and gaudy user experiences. However, there will be a backlash someday and the next generation of Internet users will jump outside of these walled gardens to take control of their own online identity.

This. This is the web I care about. The principles that keep me doing what I do for a living. I love this web, and how it works.

But the thing is, I love the web we have now, too. I love the interconnectedness and the fact that you don't need to be technical to find, share and create amazing stuff. You just have to have imagination and humanity.

So, let's go back. Let's take the web we've got today, and let's consciously retrofit it with the plumbing we had back then. Let's take the services we all work on and stick in those APIs. Let's make it all work better together, so that the sum of all the web applications is far more than all the web applications separately.

Think about the back-end services we all value: Stripe. Twilio. AWS. What unites all of them is that they're incredibly simple to develop with, and to connect into other applications. That's why Twitter succeeded in the beginning, too: because its API was simple enough that people could build apps for the nascent mobile app ecosystem. This is good for all of our products, as well as for the web's health as a platform.

It's not hard. That's the beauty of it: all these APIs and standards are simple to build and simple to use. That's why they survived. All that has to happen is an understanding that being closed is not a better way to serve your users or run a tech business.

I think people are missing the point. Yes, Flickr is still here and you could use it. But Flickr never really got to mobile (a major strategy failure). Do you know since when Flickr has similar functionality to Instagram? Today - they just released a new version with filters.

The point is, you can't build much on top of instagram, twitter, facebook, whatever. APIs are encumbered by pricy licenses, nobody wants to collaborate. Open standards for sharing data are dying. RSS is dead. Mash-ups are dead. Everything is behind private APIs and walled gardens, the web doesn't connect everything anymore.

Meh, I disagree with a lot of that. You speak as if the internet ONLY consists of social now. Your points are nostalgic and looking at the past through rose tinted glasses IMO.

Five years ago, most social photos were uploaded to Flickr

You can still do this. People choose not to. I don't want strangers viewing my social pictures, esp if I had kids. These are private moments to be shared with my friends.

Ten years ago, you could allow people to post links on your site

You still can, it's your site. If you decide to monetize your site and display AdWords then that's your call. You don't have to be a sheep and follow what everyone else is doing.

In 2003, if you introduced a single-sign-in service that was run by a company...

Don't use them and create an account. No one is forcing you to use them, but for some of us (me) it's just easier to link several sign-ins together with my Google account. These are generally sites I trust. If I don't trust them then I'll use a disposable email account anyway to register. If the "average man" on the street doesn't know better then that's his/her problem, it's the same basic principle as identity theft and people guard against that. It's time they did the same online.

In the early days of the social web, there was a broad expectation that regular people might own their own identities by having their own websites

Really? A few people maybe, but most non-tech people I know really couldn't give 2 hoots. Wordpress and all the blogging sites have made a lot more people I know open their "own" sites than would have been owning a domain name and all the other hosting and "headache" that goes with it.

Five years ago, if you wanted to show content from one site or app on your own site or app...

Yes, agree it is bad, but that's business. The same thing happens in the real world, just because it is online the principles of business do not disappear and unfortunately not everyone is that tech-savvy and some of those people who pumped millions into a business may not "get" the web like you.

I don't think we have "lost" any of these. People have just decided to move on as the technology has advanced. The internet is a lot more open and a lot more accessible to many more people than it has ever been. As a developer I may care about the above (I don't) but as a regular joe, I don't think I would waste 2 seconds, no matter how long I have been using the web.

Just a few reasons today's Web trumps anything from the 'Technorati' (seriously?) era: Open API's that reply in JSON, Cloud VPS's at $0.02 per hour, 10 Gb ethernet, 54 Mb fiber in my house, multicore computers in everyones pocket, GPS at everyones fingertips, web frameworks like Sinatra (yes, it took more than three lines of code and two bash commands to publish 'Hello World!' to the web back then), caching solutions like Redis, data crunching pipelines like hadoop, payment processing like Dwolla...need I go on? There will always be folks hankering for the glory days of alt.religion.kibology and compuserve. Ignore them. Create something game-changing instead.

The web we lost is still there, it's just that's it's just as accessible as it was 10 years. We post photos to Facebook not because of the technical superiority but because our friends and family can see them.

You can still put your photos on flickr where no one you know will ever see them.

I think most of the frameworks, libraries, and tools we use to build these new services can do an awful lot more to make this kind of thing easier. I actually suspect we'll enter a new age of programming soon, where a lot of the cruft and boilerplate of managing filesystems and metadata around your data (from databases) will be handled automatically, making this kind of thing much, much easier.

This is an excellent writeup. I think it misses an important trend in the Web's population: fewer nerds. It used to be a lot of work to get a PC with a broadband connection. Now every cell phone has cheap broadband and a suite of apps built in. The Web today is mostly populated by users who are not enthralled with the technological underpinnings that make it possible. And that is natural. The the lamentable effect is that now there is a market for accessible communication and media. And this is overwhelming the traditions of sharing and valuing anonymity on the Web.

I imagine the nerd population has grown, and accelerated over time. It is just that the non-nerds are getting on-line much faster.

I find myself disagreeing with my cofounder on things like giving the user more and more control. It adds too much complexity. Providing the basic, minimum requirements will be sufficient for 99% of our customers. I'd rather focus on them than the small sliver of folks who'd want that extra control.

I'm ashamed to say I didn't know until TFA that Guido is the BDFL...I actually thought Linus had inspired it, though that's because he seems to be the one who most frequently causes others to invoke it.

This ultimately comes down to the fact that newspapers are going out of business because not enough people want to buy them any more, and they're lashing out in desperation.

I think there's a simple argument with content and markets that all of these media companies disingenuously ignore. If no-one is prepared to pay for your content then the market rate for your content is, by definition, zero. We pay in two ways - direct purchasing, and delivering surplus attention beyond that required to consume the media, which can be redirected towards various forms of advertising. When you cannot run your business on advertising revenue alone it means that not enough people are giving enough surplus attention of enough value to cover.

This argument about the "inherent value" of one form of content or another - of the need to pay artists or creators, or in this case journalists - doesn't extend to other realms. If I decide I want to be a carpenter, I cannot build a table and demand someone buy that table for $1,000. If the market won't bear that price for the table then the government isn't going to force others to pay me $1,000 because I feel that's what it's "worth".

The market rates for all forms of media have plummeted due to there being more supply of attention-draining media than there is demand either in terms of hard-dollars in direct payment, or surplus attention to be redirected towards paid advertising (and its ilk).

When old media companies had a stranglehold on distribution because paper production and distribution was expensive, or video production costs were prohibitive, only a small elite were able to publish their opinions, and the monopolistic distributors were able to charge a premium for the limited access to information or entertainment they provided. They weren't paid well because they provided an outstanding product (though it often was), they were paid well because they limited supply. Those limits are gone. Many many people produce entertainment and informational content. Many do it just for fun and are happy not to get paid. Many more do it with the hope of getting paid anything without the expectation of the lavish salaries and expense accounts of journalists of old.

This undercuts their economics and doesn't even touch on the fact that the newer voices often offer media that is more attractive to younger audiences. Not to mention declining quality of the product in many cases. Many media companies as they've become bloated monstrosities have undermined their own product quality with short-term-profit-focused decision making which had long term harm.

Is it really the case that piracy accounts for all problems in a record industry where the giants spent the pre-Internet-boom '90s crushing independent labels, monopolizing market channels and creating a modern-day payola system on radio where programming was rigidly sliced and diced to the lowest common denominator? Is none of the loss of popularity of the New York Times down to their abandoning their predominantly liberal subscriber audience during the Bush years and being guilty of mis-leading story after mis-leading story in the build up to the Iraq War, destroying people's confidence in their role as a reliable neutral arbiter?

tl;dr The publishers referenced by the OP aren't happy at the market rate for their product and services and want government to rig the market to pay them a cushy wage. Such subsidies rarely save industries in the long term, and the public should be outraged - because legislation of this sort is a public subsidy on a privileged elite in no uncertain terms.

I'm very disappointed with Europe. There is a trend of idiotic legislation drafted, & voted on by people who don't really understand the implications. They just don't understand the internet & technology enough and/or aren't smart enough. It' embarrassing.

The other recent example was/is the brain-dead UK/EU cookie law. Is our privacy any more protected? No. All we got was some dumb generic "cookie policy" popups that we have to "agree" to and further balkanization of the web. Additional costs to having a website (are there any scaremongering companies offering auditing to make sure your website does not expose you to legal risk?). Disempowering people by raising the barrier to running a site.

All downside. No upside from any perspective. We can't even blame lobbies or interested parties because literally nobody got anything positive from this. Just pure stupidity.

To be fair, there were a few positive/neutral articles, for example by Frank Rieger, a popular member of the Chaos Computer Club, in the FAZ (Frankfurter Allgemeine Zeitung) but they are in the minority.

Of course, there is the obvious ignorance regarding technical facts like: "robots.txt is from the stone age. On or off for everyone is the only possibility" [1] which reminds me a lot of the discussion we had some time back regarding internet filters. But the one thing I find really dangerous is that they (meaning major newspapers, politicians, etc.) managed to spin the story so that the narrative is now "greedy google" vs hard working journalists. I applaud google for their efforts (and I am fully aware of their commercial interests in this matter) but I slowly begin to think they did their cause a disservice. If a discussion takes place its always about google and their lobbying. The extend of this law which could lead to bloggers being sued (btw: a side effect of the very fuzzy written law which leaves a lot open and almost certainly will need a court to decide on the details) when they link to news articles is almost never mentioned.

One last thing: Recently, two big news newspapers had to shut down and that print sales are declining is nothing new. I cant remember the last time I bought a newspaper and I am also pretty sure that although blogs/twitter/whatever are a good addition they cant replace classical media. There is definately a need for the discussion for new sources of incomes for classical paper based medias as ad sales from their online publication wont cut it. Perhaps something like a "culture/media tax/flatrate" as we currently have with the GEZ (for the financing of the public tv stations)? I dont know, but the #lsr is certainly not the way to go.

As this site is nearly down, I add my thoughts as a comment here as well:

Well, some media (FAZ) did a pretty good job, letting Frank Rieger explain the "Leistungsschutzrecht". OK, even there, it was one article of many. And only one.

And what is new, when it comes to the press not publishing anything, that goes against their own agenda. Even across a lot of publishing-houses. Well nothing new under the sun.

What is really, really bad, is the fact, that the law is so fuzzy, that everyone quoting from another source might be potentially liable. This law is so bad, because it just might kill the independent voices. And I think there might be a reason for this.

Because the press oftentimes has no incentive to dig deeper, to ask critical questions, when it comes to the really important questions, this job is left for the independent voices, that do this out of a feeling of necessity. But if these voices are silenced through fear...

... well, I think you get the drift. And I know, this sounds a lot like conspiracy - and I am not saying, my thoughts come anywhere near the truth, but I just wanted to share the thought.

Just one example: The so called "Netzsperren" (blocking sites, because of content with filter-lists - aka censorship) were reported by the big media as being bad, after a lot of independent bloggers had written about it and the discussion just could not be "ignored" any longer.

Interesting and sad. I don't know if it is possible but I am wondering if such a law could include an 'opt out' policy. Something which said to search engines and the like that they are allowed to snippet your articles.

Then have Google turn off indexing of everything that doesn't have opt-out enabled.

My expectation would be that the opt-out publications would flourish and the ones who had opted "in" would quickly die or decide to join the "opt out" group. I can't imagine anyone looking at their referrer links would think this scheme was a "good" idea.

FWIW, I've written to my member of parliament [0], expressed my dissatisfaction and asked him to oppose the law. He wrote back saying hat his parliamentary party is already opposing this law but for slightly different reasons: It won't increase the quality of journalism and will just create a flood of lawsuits. Lastly it is far too vaguely phrased as to not have grave side effects.

The changes of the opposition are slim though as there is a conservative majority the parliament.

Interestingly, he has a personal axe to grind with Axel Springer AG as he was a big part of the student movement of '68 which was so intensely vilified by said company ("Youth in the street - Germany going down the drain ...").

Frankly, I can understand the publisher's point, but I'm surprised that they do not seem to see how this could backfire if Google simply stopped crawling their sites.

But what's more: I think if this law became a reality it wouldn't affect my personal web usage at all. News sites are about the only type of website left where I still type in the URL and go to the page directly instead of doing a search.

1) The notion that the systematic clipping of one or two sentences from every news story you can find on the Internet for the purpose of selling ads is "fair use" is totally absurd.

2) The "robots.txt" argument misses the point because the publishers never gave permission to use their content, so the presence of an "opt-out" mechanism is irrelevant. The mechanism must be "opt-in", i.e., Google must ask to use their content.

3)If we believe that intellectual property has become an obsolete concept altogether, we should be prepared to accept all of the consequences of this, e.g., a site might pop up using a new TLD that accepts a search query from a client, passes it on to Google using a spoofed IPv6 address, and passes the result back to the client. We also shouldn't care about plagiarism, trademark infringement, etc., because the Internet has clearly made these concepts obsolete and laws related to them unenforceable.

IANAL but trying to read the law, it seems to be a rather blatant lex google.[1] They specifically state "search engine or similar services" should be prohibited from (in an extreme interpretation) linking to newspapers, if they do not pay. ( The rest of the law is probably just redundant, since it seems to reimplement copyright for a small subset of already copyrighted material.)

The bigger trend to this story is that information changed from something that was scarce to something that is now abundant.

What the German publishers don't seem to understand is that their once so valuable and scarce goods (information and news) are becoming less and less valuable. Looks like they are trying to defend a dying business model with legislation.

Herbert Simon said in the 70s: â€śA wealth of information creates a poverty of attention.â€ť Google is a major dispenser of attention, and I think the German publishers are doing themselves a huge disservice by making it harder for Google to send attention to the publishers.

The big unresolved issue is: how can we finance good journalism in an era where the value of static information approximates zero very quickly? Maybe the answer is in moving away from static information to an information process as the product... Something that can't be copied easily.

"What's wrong with newspapers being paid for the content that they produce? No one has to use their headlines if they don't want to pay for it."

What about fair use? Does that exist in Germany? I'm a quote geek for example. I love collecting "favourite quotes", giving a source (link if possible). I started out with the general quotes everybody knows, but of course I also copy and paste from the web in general, and sometimes I actually type what I read in a book, and translate it to English. Man, I even love talking about it. I love quotes.

Now, I consider that "fair use", and since I do it mostly in English the noobs left me alone so far. But I don't even know if there is such a thing as fair use in Germany... any ideas?

Why is this topic so complicated? A Google news snippet very rarely would suffice for a user, if they are interested they would click through, if they are not, they would not visiting the newspaper anyway. Google are not showing ads in Google News, although they might show them in search results. But still, no profit that newspaper could address is being taken by Google. In fact, Google is providing traffic.

So taking all the philosophical statements out of the way, what is the econimical case that the newspapers are making? If Google actually showed a bigger part or the whole articles, they'd obviously be taking revenue from newspapers, but it's not the case.

I think the intuition behind these proposed laws is that Google has become so dominant in providing the information substrate of society that it should be subject to regulation as a monopoly. The argument would be something along the lines that search works best when provided as one comprehensive resource. Therefore, as a natural monopoly it should be regulated for similar reasons that other utilities are (electricity, water). The publishers are trying to make a case that their product has value as a "public good." Classical economic theory says market forces alone will produce a sub-optimal supply of a public good with out regulation.

Something like that. It's not a totally implausible argument. It does seem notable to me that Google is able to extract value from content that the creators can't.

Isn't this a case for the internet bat signal? I must admit I still haven't included it on my sites, mostly because last time I looked they only provided JavaScript hooks making my site vulnerable to their site being hacked. Perhaps they have an API by now?

Technology generally finds a way around poorly written laws but I think there is a real underlying problem. Google News has become its own thing now. The headlines, snippets and icons I get from a variety of newspapers is often enough to skim over the day's news and I only click on maybe one or two stories for more in-depth reading. I think a way for Google to charge news sources and for news sources to charge Google both make sense but they have to be developed by willing participants, not by legislators.

what is really scary is not the proposed law but the public relation campaign google is mounting against it.It is a (very likely) unprecedented internet lobbying effort involving not only googles advertising network, but also a link on google.de (no longer visible).To a casual observer, watching the short youtube clip or reading the campaign slogan "Willst Du auch in Zukunft finden, was Du suchst?"("Do you want to continue to find what you're searching for ?") it could seem like the government was trying to shut down google search.

It seems like a scary prospect to think of the ways google, a private company, could start to use its unparralled reach to lobby for legislation in its favor around the entire globe.

Nothing scary, they just are aiming carefully to shoot their own buttocks with some good German schrapnel. If there will be no readership for them, they will not be able to sell ads and die in 3-5 years, depending on long term contracts. And this is a good thing. Since you can use free Russian and Urdu headlines for free! Oh, I forgot to mention Chinese.

This is rather interesting that it would be coming from Axel Springer. As someone involved in the Berlin tech scene I've seen them really investing a lot of money in innovative events and companies. My guess is there is some typical corporate disconnect there. I doubt this makes it through the legislation.

IPython notebook is actually one of the most interesting developments not just in the python world, but in computing generally. It's the first step towards the fully graphical shell on the internet that many of us have been looking for, whether or not we realize it.

Being able to weave together text|markdown, tabular data (in almost any format you want) and images ( whether from matplotlib, raw captures, or synthetic assemblies ) in one environment is very powerful.

I suspect that with some additional tools IPython notebook will become the integrators workbench par excellence, useful in many contexts.

The better feature, for me anyway, is the notebook interface (it starts a local web server and gives you a rich web guy, somewhat like a Mathematica idea of interactive notebook editing with inline charting etc.)

But, that's a whole other bunch of dependencies that aren't in scope when installing ipython.

Not sure of the best fix, offer ipython and ipython-full-stack on PyPI?

The notebook interface is the natural generalisation of the REPL, and has the potential of being reused and copied by other languages (clojure, JavaScript) just as, in offline form, it has been used in Mathematica and Racket for years.

You can try IPython (shell) and IPython Notebook (web interface) online with Wakari at https://www.wakari.io - platform (with all popular Python scientific libraries installed) that lets you run multiple Python sessions and access them online anywhere. It also allows IPython Notebook sharing - you can share your workflow.

This is the awesomest news for the day. It is so much more than a tool purely for the scientific community. I use it for quick fire data analytics codes all the time. Especially when coupled with pandas and matplotlib. pandas can output data in a tabular format directly onto the HTML notebook.

If creating forms on the same was a bit more simpler, then it would be biggest boon to a data analyst since excel sheets...dare i say.It would lead to quick fix applications, rapid prototyping for reporting applications. And it can be hosted and shared directly: http://nbviewer.ipython.org/

It was great to click through and learn that the funding was a grant. With that kind of a number, I thought they had raised a VC round - not that companies working on open source projects is bad, but it is always nice that they don't have to worry about a business model.

If you haven't seen it, I highly recommend Judge's 2009 movie 'Extract' with Jason Bateman of Arrested Development.

It sort of inverts the Office Space formula and shows the sometimes taxing, even soul-crushing nature of entrepreneurship.

It seems to follow the pattern of Office Space and Idiocracy in being panned by critics and ignored by audiences at first, and then quickly developing a cult following and being recognized as pretty brilliant.

Mike Judge and the other writers stopped by our offices at Tout.com (startup in SF) and were asking us about startups and being a developer. We told them about meetups and open source stuff. Probably won't affect the show, but it was fun talking to them and seeing they were really into it.

He made the Goode Family, which was short-lived but from most accounts, terrible. Hopefully he'd be able to grasp SV culture as much as he did in Office Space, and not miss it by going after typical caricatures of San Francisco.

Using this felt similar to using the Github Launch Bar[1] for the first time.

Something about having a command line interface really makes it feel like you are communicating deeply with the software, as opposed to poking around its surface in arranged flows. With a command line the interaction is different - it is exploratory with freestyle flow composition, and that can be really fun.

Modern graphical interfaces combined with a means to freely manipulate data and compose commands seem to be quite rare. I have always wished to have a full command line accessible in RPGs inside the game's menu system, so I could script some tedious things.

If the world were full of people who were interested in optimizing their computer interfaces this way, all websites would support a mode of operation like this and there would be a framework for the end user to pipe them to each other like the unix pipeline.

Google Search (with Instant Search enabled) has had keyboard navigation for a long time. After you hit enter on a search, hit tab and you can navigate the results with your arrow keys. And when you go to Google's homepage your cursor is automatically placed in the Search box, so you can search Google with only a keyboard.

I really like this, but there needs to be a better way to navigate to a page. The pop-up scheme is blocked by my browser. Perhaps, a iframe with the terminal becoming a header similar to Google image search? Maybe using Links?

In addition to the mixed content issue on Chrome, I've pushed out a fix for the tab completion component. It too inherently suffers from mixed content because our autocomplete server only does HTTP at the moment.

This is a great example of Bratus et al refer to as a "weird machine" [1], in which an attacker uses crafted input to create a new and unexpected computational environment. That is, by using special input, the attacker manipulates pieces of the target program to act like a CPU that processes "weird commands" (ie more special inputs are used as assembly instructions that run on the weird machine).

Anyway, there is a rising field ("language-theoretic security") that studies this phenomenon. If this Pokemon example interests you, then you should give it a look.

Note that a key part of the hack requires the hardware to reset while a save game write is in progress. This causes the file to have invalid data -- an inventory list count is set to an "impossible" value.

Then, within the game, the invalid-length-list is used to overwrite other arbitrary locations, including a function pointer to an update procedure. Once that's overwritten he can jump to his own code and it's "game over" as in, he completely controls the hardware.

But from what I can see, it wouldn't be possible without the initial hardware resetting during a write. Not that it diminishes the awesomeness, it'd just be a bit purer if it was a software-only hack.

Fascinating. I always wondered if this sort of thing was possible when I noticed as a child that pressing multiple buttons at the same time on my parents calculator made the screen show stuff that wasn't actually real numbers (things like a 2 with a part missing, etc.). Likewise seeing which buttons on the VCR have precedence (e.g. if I hold down "play" and press "stop" then what happens? And vice versa?). I always assumed the device wouldn't be designed comprehensively to handle all possible inputs like that, so there was a chance some of them would allow you to do funky unintended stuff.

Given the complexity and freedom of access that a videogame has, I'm not surprised that this hack is technically possible, but it is very impressive that someone's managed to do it!

"This script walks from the Viridian City pokemon store to Oak's Lab in the most efficient way possible. The walk-thru-grass function guarantees that no wild battles will happen by manipulating the game's random number generator."

The guys that do these tool-assisted speed runs are incredible. One fellow plays 4 Mega Mans all at once with a single controller doing input for all 4 games at the same time.

This Pokemon hack is insane, but was inspired by a guy who uses it to beat the game in around 2 minutes. That particular speed run abuses the fact that everything in the game has a simple identifier. So, what he does is inserts a warp point into his inventory, drops it in front of himself, and walks through it to the end of the game.

So am I correct in my understanding that this sort of hack is made possible only because the Gameboy uses an 8080 derived chip with Von Neumann architecture? ie. if it used a Z8 (or any other Harvard Arch chip) it wouldn't be possible to "bootstrap" like this?

For those that are wondering if something like this is possible outside of computers/video games, I would recommend a study of Lucid Dreaming. If brain hacking is possible, this has to be the best method of entry into the system.

I think you might be making the assumption that events consist of an event name plus arbitrary JSON envelope. This is a very MixPanelish view of the world - it doesn't really translate to Google Analytics, Piwik, SnowPlow, Omniture...

1. It includes code to load up various analytics tools even if you never use them. For example, if I only use GA and Mixpanel, do I really want to serve the bytes for all the other plugins to my users? Maybe it would be useful to only package the JavaScript that a certain site needs.

Happy to be supporting this project at http://customer.io . A lot of folks who use other tools want it to be easier to send the same information to everyone. Thanks guys for helping us all play nicer together.

This looks awesome! One thing would be nice on the demo/landing site is to maybe add screenshots of how your examples translate in Google Analytics or the others. Like karolisd mentioned in his comment, I would like to see how that stuff translates or what it looks like in the respective apps. Again, great work! :)

Broadly speaking: you are not "investing" in any market if you only own the amount you intend to use or consume for yourself. You are only "investing" if you own (or have contracts on) more than you intend to consume.

For example, having a few bananas in your fruit bowl does not give you a position in the banana market. Owning thousands of bananas you intend to resell, or shorting a banana company, gives you a position in that market.

Likewise with housing. Owning enough housing to live in (at whatever standard of living you intend) is a market-neutral position, so you should not be particularly excited by price changes in the market. If you own too little housing, you're "short" and price raises are a negative, and if you own excess housing (extra properties, or a larger-than-desired property) you're "long" and price raises are a positive.

I'm surprised we don't see more discussion of house prices on sites such as this. In my opinion it is THE issue affecting our generation.

Specifically for HN, the high cost of housing severely restricts entrepreneurship. If you are paying 50% of your income on a rent or mortgage, you are of course less likely to start a business, and the cost of doing that business in terms of employees salaries and commercial rents make it less viable.

Also consider that your suppliers, and their suppliers, and their suppliers ad infinitum, all have the same high costs ultimately derived from high housing costs for their employees.

This all adds up across the country and the whole economy is made less competitive and agile as a result. With lower housing costs there would certainly be less unemployment.

The sad thing is that the even intelligent people are duped into believing the myth that high house prices are good for them. Clearly lower prices mean more disposable income, which is usually a good thing? Clearly high house prices barely benefit anyone as we all need a house, and the next house you buy would likely have gone up by a greater price than the one you have now? And yet we persist - 'we MUST get on the housing ladder' and we blame those nasty banks because 'they aren't lending any more' when all they are doing is scaling back lending multiples from the absurd back to the ridiculous.

All that high house prices achieve is to keep young people into debt serfdom, keeping us on a treadmill servicing massive mortgages for the same bricks our parents bought for 20% of the price. We really could be out doing something much more worthwhile.

I know next to nothing about trading. But it's become apparent to me that some of the core concepts/terms in trading (long, short, hedge, liquidity, futures, cover, etc) are very useful for modeling things in other walks of life. So, trading-savvy HNers: Are there any books, websites, or habits that you'd highly recommend to help a newbie become familiar with basic trading concepts?

"If house prices rise, the value of your house (the hedge) increases but so does the cost of shelter."

What about the ability to rent? As demand for purchasing housing goes up, demand for renting typically goes down (ie. either levels off or regresses). Sure over time both go up, but at a particular slice of time, usually on a cost of living basis one is advantageous over the other. And while you're renting at a cheaper cost than it would be to essentially rent money (let's not kid ourselves, you don't own a house if you have a mortgage), you have liquidity to place in other investment vehicles.

Small observation... maybe there's something to it, maybe not, but it seems I'm 2/2 on calling when market swings happen here in downtown San Diego over the past 10 years. Take a look at the delta in rental prices in properties in various classes and compare verses actual selling prices. Just like a P/E for a company, if you can't at least turn a profit on owning a piece of property as a rental, it is likely overpriced. As much as people love to talk about various market factors much of this is emotionally driven.

For instance, last year I observed an entry level condo complex in a marginal area rent 1 beds in the $1800-1900 range. The latest comps for that building were $150k for like properties. About 5 blocks away 1 beds in a luxury high-rise were renting in the $2200-2300 area. Comps? High $300k. At the height of the market comps for the luxury high-rise were in the mid $500k area and the marginal property were in the upper $300k area. Back when that was happening, the luxury high-rise unit was renting just shy of $2k. The marginal unit? About $1400. I took another 4-5 other buildings in the area of varying quality and this trend was intact. To be fair this is a small, insular market and these swings are largely driven by speculation from the bubble-tastic situation many overheated markets found themselves in (ie. people owning and flipping multiple properties found the bottom of the market being rented cheaply).

Wtf?! Yes you are short housing...then u buy one...prices go up... you sell the damn thing...at which point you become short again...but with that extra money you can cover your short and come out ahead in a different market.It isn't like gold in that the prices are fairly constant across markets. If my house in the bay area goes up 30% i sell that and relocate to illinois you can bet i will cover the short and come out ahead since home prices in illinois lag ca by a good 150-200k.

This is true, you can't sell your house and then NOT have shelter. What older people can do is sell their house and then move into an apartment and gain financially from this transaction, assuming they don't live for many many more years.

However, if they do they "lose" on the transaction: and this proves the "covering a short" nature of the transaction.

What about if you get to 60, sell your house and use the money to rent for the rest of your life. In that case you might have higher short term costs as well but to you can stay in short term as you will likely not need a house for long enough to make long term more economical.

Also housing markets in different places don't move in unison so there is some room to trade markets and turn a profit.

My parents bought their house 25 years ago 250k and sold it for 1.5m. Now they are renting a 2 br apt for 1500/month. Their housing cost will be roughly 25k/yr and they are millionaires. The idea that the article talks about how you can't be long the housing market is just dumb. His analogy doesn't work and to say that you can't benefit from house prices going up is dumb.

Depending where you live there are alternative shelters. The alternatives are not for everyone, and they are usually not "easy", but there are advantages and disadvantages to not owning (a house) and to not renting.

- Author fails to make the distinction between two aspect of owning a house: for shelter and as an investment. These are two, although related, distinct aspects.

- Author fails to acknowledge that the point of making money an a house ('using it as investment') hinges on not rising prices, but prices that rise faster than inflation / COL, and/or inflation being above the mortgage rate the house was financed with. So no you're not 'just covering a short position', you're covering a short position and going long at the same time, with the same vehicle. But this is of course where the analogy falls apart. I posit that his analogy is unhelpful in understanding the role of a house in personal wealth management.

I love google products (and I'm not worried about my email for a second), but their explanation is really bad:

"The problem with Google Mail should be resolved. We apologize for the inconvenience and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better."

It sounds like something an airline might say after bumping you to another flight!

What is with the URL on this submission pointing to the URL any user of Gmail would use to access their account? Could the curators please change the submission URL to a source that shows reliable updated information?

My Chrome did not crash, yet I am logged in with Sync. Instead of keeping Gmail in a Chrome tab, I use Fluid app to make Gmail into a "desktop" app. Seems to still be up. Others can use the same workaround by opening Google apps in Firefox and Safari.

Given the massive scale of Gmail and the infrastructure of Google, it's highly unlikely that the entire audience of Hacker News sitting and pressing F5 every second would hurt Google's ability to bring it back up.

In other news, 23andMe introduced relative finder not long ago. I hadn't logged on in over 2 years after I did it with a groupon promo, but I had a public profile with my name and city. A half-sister i didn't know i had found me with matching DNA, owns a business 5 blocks away and walked over to meet me (what's the probability of that!?). Found my dad was not my biological dad and i was from the same sperm donor as my new sister. That was a weird week...

Is the most obvious question being adressed in any jurisdiction yet? - Do you legally have to disclose your results to any insurance company you already have an existing contract with or prior to any new insurance contract?

By chance, I just read a typical life insurance contract and it already stated that if you have undergone a genetic testing, you do have to disclose your results if you enter into a life insurance contract >300.000â‚¬. This was stipulated in the contract and not under general German contract law. If you do not disclose your results, the company can void the contract any time and/or terminate it any time in the future.

This is going to be THE most important issue with genetic testing - the implications for your insurance contracts. Obviously, your test results can have massively positive or negative results.

I did a news search to see what researchers are saying recently about the data analyzed by 23andMe. One news article led me to a company blog post by a bioinformatics researcher, Gabe Rudy, "GATK is a Research Tool. Clinics Beware"

in which he applied his own industry knowledge to his updated 23andMe report. His conclusions suggest that the product needs much more work:

"I promptly sent an email off to 23andMe's exome team letting them know about what is clearly a bug in the GATK variant caller. They confirmed it was a bug that went away after updating to a newer release. I talked to 23andMe's bioinformatician behind the report face-to-face a bit at this year's ASHG conference, and it sounds like it was most likely a bug in the tool's multi-sample variant calling mode as this phantom insertion was a real insertion in one of the other samples."

. . . .

"But because GATK has been used so prolifically in publications and is backed by the Broad Institute, it can be viewed as a 'safe' choice. As small labs and clinical centers around the world are starting to set up their DNA-seq pipelines for gene panel and exome sequencing, they may choose GATK with the assumption that the output doesn't need to be validated.

"And that would be a mistake.

"GATK is as susceptible to bugs as much as any complex software. Their new mixed licensing model (free for academic, fee for commercial) is intended to add more dedicated support resources to the team. I suggest they think about adding dedicated testers as well."

So for those of us following along at home, the crucial idea is that most of the "information" that 23andMe provides paying clients has not been validated. Not only has it not been validated as to correctness of the genome analysis software (the industry scientist's observation), it has even less been validated as a clue to clinically significant disease risk for the majority of diseases that afflict people in developed countries. Pay your money for the service at the new lower price if you like, but prepared to see your personal genome results repackaged and reinterpreted for years to come before you learn anything from them that will help you improve your health.

Just a note, especially for non-Europeans who are thinking of doing this:

I just used 23andme from Canada during the $49 A/B testing. Delivery was next-day, and other than 15ml of saliva being a lot to collect, the process was pretty smooth. They say it'll take 2-3 weeks, but realistically it took them about 4 business days to process my sample.

Unfortunately, almost all the results are for Europeans. There may not be that much of a difference between Europeans and Asians (I'm not sure), but it's worth mentioning. They also don't test for repetition induced diseases, so things like Huntington's won't show up.

If their results are accurate to a degree, why are their tests not being subsidized by insurance companies. How much would an insurance co save by alerting a patient to her chances of getting diabetes, not to mention the money the patient would itself save.

I paid in at their first $99 discounted rate (2010) and was very disappointed when they tried to demand a non-discounted new sample for newer tests about a year later. If I had paid the full $500 I would have been downright peeved.

Still, I'm glad I did it, 23andMe is a very enlightening experience. I even found some distant relatives I never knew about.

Minor pet peeve: When I'm paying $100 for something, I prefer that shipping be included in the price. It comes across as less of a premium service when the advertised price doesn't include the full standard service.

Will they give my genotype information to FBI or CIA? Will they pay damages if information gets stolen? This is not some passwords or account numbers, this is data about real world which can't be changed.

I signed up for their service few years ago, honestly didn't get any useful info. My wife has genetic mutation that seriously affects her health, nothing in their profile is even remotely indicating (that we could see).

Overall I am pretty disappointed with data I received from their service. For example I supposedly have genes that make me risk for obesity, yet I was most of my life very skinny and in recent years gain some weight because I thought it wasn't healthy to be so skinny. So, I wonder what kind of useful into others can get, when we can't even confirm basic info. I also believe (it was a while ago) that color of our eyes was 'guessed' wrong'.

Whole slew of startups should look into genetics. DNA sequencing is basically the final frontier of digitalization of the world. Look what happened with text and images, and it's now happening to everything living. Medicine is going to change.

Interesting! In the past if you made an account on their site and signed up for the new letter (but never actually bought a test), after a while they would send discount offers. The last one I received was $50 off, but the tota l cost was still over $200. No I have no excuse (cost wise) not to try it.

This is a great attitude because it moves the locus of control from outside of yourself ("I'm a helpless victim of everyone else") to inside ("this is my fault"), which is generally a more productive mindset since it puts you in control of your life. Obviously if you then translate that control into guilt, it breaks down badly, but I don't think that's what he's advocating.

This is not a good thing. In the context of the article it seems more like a defense mechanism than anything else. Yes, it's good to take responsibility and people don't do it as often as they should but to take on the mentality that everything is your fault is unhealthy. In this context it looks like it stems from a need to be powerful and in control all the time.

The truth is, we can only control efforts, not outcomes. Sometimes things aren't your fault, you are the victim and while believing its your fault may shield you from feeling bad about it, it ultimately sets you up for failure. When you believe everything is within your control you're in for a rude awakening because a lot of bad random shit will end up happening. Once enough bad things pile up and you're still thinking its your fault, those feelings you're trying avoid by pretending you're all powerful will begin to bubble up and then you think "not only is everything my fault, but no matter how much I learn I just can't get most things right. So now I'm in control and suck at it".

This is classic avoidance. A defense mechanism. A more healthy attitude would be to identify what is in your control as well as realize and admit the things that are out of your control and cope with them as they come. By all means, take responsibility and he in control of your life because lord know most people aren't but also remember that there's no shame in admitting some bullshit happened that was out of your control and subsequently feeling bad about it.

What happens if there's a natural disaster? It's your fault you didn't predict it and lived or worked in a certain place? What about a key figure in your business dying suddenly? Are you at fault for not keeping good enough watch? The example of someone being rude right in the article is a great one! Of course it isn't your fault if someone is rude (at least not always)! Sometimes people have shitty days or attitudes and it has nothing to do with you! Besides being a defense mechanism you could argue that this attitude may stem from egomania.

So do you want to know the real reason I cut those chapters?...I this...I that...I something else...(I could list another 20 of these, but you get the idea.)

Lots of good stuff here, but I never found the reason I would most hope for: No one gives a shit.

In business, we all see it every day: who did this, who did that, who said what, who was right, who was wrong, who is hurt, who is angry, blah, blah, blah. The gossip part of our brain wants to hear more, but the business side doesn't; it just sounds like a whole bunch of played out drama.

Better to just shut and move on. Glad you did. Just not sure I'm glad why you did it.

Something like this appeared in Good to Great. In a study of CEOs who lead their companies to significant improvement that lasted after the CEO stepped down, they found a common personality characteristic. The best CEOs constantly were watching for problems, took responsibility (aka fault) for everything that went wrong, credited everything good to others, and then anything that could not be credited to someone they said was luck.

This is a useful attitude. As news goes up the corporate ladder, it inevitably is colored in the most positive possible light. (http://www.davar.net/HUMOR/JOKES/SHIT.HTM is a funny, but true, take on this phenomena.) You need a constant vigilance for problems at the top to counteract it. And the common willingness to rest on your laurels and deflect blame elsewhere is not going to lead to that willingness.

Step back from your solipsistic convictions for five minutes and realize that your rationalizations here only serve the purpose of avoiding forgiveness. You have decided that not only are you not a victim but in fact you and you alone are responsible for everything and therefore you don't ever need to forgive anyone. Moreover you have now implicitly made the rest of the world victims. Hm...

I've found this attitude very useful, especially in social and personal situations. My reaction to everything that doesn't go my way is to examine myself first - what did I do wrong, or what didn't I do, or what could I have done different.[1]

At it's core, this is about accepting and embracing agency - you have the power to effect changes. I think that so many people in the modern world suffer from learned helplessness[1], where they think that the situation leaves them with no ability to impact events. This is false. Something went wrong? Try something else!

[1] Important Note: This is about things that I do, not things that I am. If you blame yourself innately, that's bad and leads to low self-confidence. It's not yourself, it's your actions - and actions are changeable.[2] http://en.wikipedia.org/wiki/Learned_helplessness

While it is generally a good idea to be open to the idea of taking responsibility rather than fobbing it off on someone else... I can't say I agree in this case.

You had terrible, awful expectations. You expected your employees to be dedicated to the benefit of your clients rather than themselves? This is capitalism. People have jobs because those jobs provide THEM a benefit. Especially today, where you as an employer make ABSURD margins of profit off of everything your employees do (as worker productivity is astronomically high thanks to computers and such - just ask yourself how productive your company would be if you banned any technology produced after 1980. Couldn't survive, could you? Yet, companies did. And they paid their employees the same average wages you are paying now. You're just getting far more benefit.)

Pick up a book on capitalism. It's likely not what you think it is. It is not workers making as little as possible sacrificing everything for corporate praying that one day they'll win the lottery and become CEO. That's not capitalism. Capitalism is based on free exchange to mutual benefit of both parties. Your clients use your business because it benefits them. You tend to your clients because it benefits you. Win-win. Your employees work for you exclusively and only because it provides them substantial benefit, and you provide them that benefit because you make profit off of them. Win-win.

Expecting a worker to be dedicated to your company above their own interests is disgusting. You would not in a thousand years consider cutting your projected profits in order to improve their salaries, so there is no reason whatever for them to display any loyalty to you. Workers do not exist to work. They are human beings who exist to seek their own enjoyment, and if you as an employer fail to provide adequate pay/benefits/etc they will leave you to fail on your own. That is how it should be.

There is an escape. If you really do want to build an organization with invested, loyal coworkers, you can. The first step is to read a book on capitalism. Understand what a persons compensation is supposed to be based upon. It is not supposed to be based upon market average rates. It is supposed to be based upon the value of the work being done. If one employee earns a million dollars for the organization, that employee deserves most of it. If you want people to be invested, you have to actually make it in their best interests to do so. Right now, it is explicitly AGAINST their interests to be productive for you. It can't help them, and can only cost them. You'll cap them out at what they can make based on 'market rates'. And the chance of them rising to senior management is an invitation to self destruction with no payoff.

Employees are not going to be dedicated to the poisonous cesspit modern business has become. Just not going to happen. It's not your fault, and it's not theirs. And you cannot fix it while maintaining the kind of absurd growth figures and profit margins that modern businesses are taught to expect.

"Everything is my fault" is entrepreneurs' wishful thinking so that they can believe they can fix whatever situation they're in. However, not everything is under their control, no matter how hard they try to believe.

You don't control the weather, the people in the streets, the companies sending you checks, the currency exchange rates, etc.

This is way better than forgiving. When you forgive, you're still playing the victim, and they're still wrong, but you're charitably pardoning their horrible deeds.

That's NOT forgiving. That's playing the victim, them being wrong and charitably pardoning their horrible deeds mumbojumbo. "And then they bitterly stabbed each other again..."

Forgiving means deciding to let go of what has happened; letting go as in not letting what has happened influence your future life and decisions, and forgiving starts from yourself. The act of forgiveness is to free yourself from the past events. It goes like first realizing that past is past, you can't change it and nobody else can change it either, and that ultimately the suffering seeps not from what had happened but from the way how you still take it, and eventually realizing that generally you're just so much better off not thinking about it anymore, regardless of if you win or lose the corresponding power struggle inside of your head.

When you've already removed yourself from any emotional entanglements it's basically a no-op to forgive someone else because at that point you no longer care about what s/he has done. It's good to forgive others to encourage them to let it go but it's not necessarily for your own recovery.

Having said that, and realizing this is a motivational piece "It's my fault" is a complex statement (combining loaded notions of Self & Fault), and some interpretations allows (you or others) to sweep real issues under the carpet.

When sh*t happens the real thing to do is to admit that there was a failure instead of denying it and have a frank conversation about it with the parties involved instead of blaming others. (Note: sometimes the same person may take both parts of this conversation.) Sometimes it helps to separate Responsibility from Accountability, and assessing "fault" usually is counter-productive in this regard. In contrast, the Truth and Reconciliation movement in South Africa is a prominent example of coming to peace without placing blame. There, they separate concerns which gives room to people to more easily admit their mistakes.

This seems like something directly out of Dale Carnegie's "How to Win Friends and Influence People," and it plays into what Dr. John Dewey referred to as human's basic desire to be important. Any time a person takes responsibility for someone else's actions, they feel a sense of power over that person. Whether this is good or bad depends on how they intend to use that power. In this case, it's being used in a way that affects a positive outcome for both the author and the people around him. In other cases, this increased sense of power can lead people to believe they have power over others which can lead to corruption (you know, 'absolute power corrupts absolutely').

In general, this seems like something that should be done in moderation. A willingness to accept responsibility for some things and be proud that you aren't blaming someone for an outcome you created is healthy. At the same time, feeling as if you have the power to affect everything is not healthy. Some things are just outside of your control.

I personally have a lot of guilt and beat myself quite a bit, but I also realize constantly being the martyr and assuming I have control means my ego is as big as it is when I always blame others.

While blame can be placed and responsibility can be taken, that is a trap. It happens to be a trap that most of the country is in. It is Bush's fault. It is Obama's fault. Sound familiar? Instead we should be saying "How can we work together to make things better?" Sometimes feelings run too deep to switch into a Kanban philosophy where life is a constant process of potential and realized amelioration. We don't have to be liberals, conservatives, or martyrs to do that.

That said, Derek, I think realizing the leader should take responsibility is a great lesson for all. It is a step towards enlightenment.

I have been doing roughly this for a few years now. It isn't always possible to achieve, but it is incredibly effective if you can get beyond thinking about who is at fault, who is guilty, who has wronged you.

Looking at my own behavior, it is almost always possible to find something I could have done differently that would have avoided an undesirable outcome. Over time this has improved my intuition of whether certain actions are better avoided, even if doing them is theoretically something that should be just fine.

Concrete example: you could complain about getting hit by a car yesterday, or you might re-examine what was so important that you decided that driving two hours a day was a good idea.

Nothing is really anybody's fault. People only do abusive things because somebody else previously did something abusive to them. This is not likely provable, it's just a belief I have. Of course victims still need to express their anger and abusers need to be held accountable, but it's also important to ask, "How can I not be an abuser or a victim in the future?"

Of course a complete apology feels good, it provides absolution. It will feel even better if forgiveness is tendered by the employees. But to reiterate the real challenge is not getting into situations like this in the future.

This touches on an important way of looking at life but the wording (and some of the examples) are throwing people off the scent of the good advice.

It's not that everything is my fault, it's that how my life goes is my responsibility. Born poor? That sucks, but what are you going to do about it? Even if you could find someone to blame for it, what good would it do? What possible effect could it have to find someone to point a finger at? If the answer is "no effect what so ever" then you're better off just taking responsibility for your situation and trying to do something about it.

In contrast, I have friends/family members who, no matter what experience based advice I might offer, have someone or something to blame for why they won't be able to take it and why they'll have to just remain miserable. And their situation doesn't change (maybe changing it really is outside of their power, but if so they certainly don't articulate this well). So I just don't see an advantage of their strategy.

No, it's not my fault if e.g. my country suddenly slaps a ridiculous tax on new businesses making my venture a negative value proposition over night. But I can still have some response other than just pointing my finger.

I respect Derek Sivers a lot, for a lot of things. But I honestly think, this one lacks the punch of some of his other (earlier) writings. For example, he was at his insightful best in 'ideas are just a multiplier of execution'. When I read that, I was blown away, "what clarity!" I thought. And it seemed like a genuine outpouring of a Doer.

But this one is ambivalence at its peak. Sort of Chicken-soupish. It lacks the balance, and is a bit extremist. Any very unlike the writings of a stoical mind (And please-please, I am not pulling any punches at all. I hold Derek Sivers, in high esteem. But even gurus, need some candid feedback at times), which I am sure he is, most of the times.

This reminds me of the idea that leaders share the credit when things go right & take the blame when things go wrong. I've personally found it to be a far more productive way to live. Instead of looking to blame others, I try to look at myself and ask what I could have done better. It focuses my attention on the one person I have the greatest power to change... me.

The only drawback i personally see is in fact that, when you realize that "everything is your fault" ie, that you can influence things, people around you to obtain the desired result, is in fact manipulation.

And I don't like manipulating people. Yes, it's a lot easier. yes, leaders do that non-stop. But personally I can't get around to doing that.

Also, if everyone was doing it, we'd probably play a much higher level of "manipulation game" and feel fake about everything we do.

This attitude works well in social situations where you have some advantage (parent, boss) or are roughly on equal footing. It works less well in non-social settings (the natural disaster example other people are giving) or if you are at a serious disadvantage. For the example he starts with, where he was an embittered boss who felt screwed over, he is absolutely right: "Forgiving" others requires you to first blame them. Blaming them when you had the power over them is petty and asinine. Embracing your own power is far healthier, more effective and feels tons better. And it leads to future outcomes where you stop and wonder what you could do differently to not set your people up for failure.

Though I think it is a tad unfortunate that some remarks here are taking it too literally when it was clearly intended as lighthearted and tongue-in-cheek.

The power in this article is not in accepting responsibility in general, but in accepting that a degree of self-interested rationality governs the actions of all people, including those on your team whose efforts run counter to yours, undermine you, or betray you, and that as a person with authority, you are responsible for the environment in which this rationality will play out.

A lot of times you and your behavior are the only variables you have to work with. Something might actually be mostly due to someone else, but you can't control that someone else very well, so changing yourself is better to work on even if it would have less result.

I share this point of view. If I've learned something after watching Mortal Kombat movie a dozen times is that every man is responsible for his own destiny. We usually blame other people for bad things that happens to us because is more confortable than assuming that even after many years on this planet we still make a lot of mistakes.

To accept the obvious truth is really liberating. It takes away all bad feelings you had for people involved on situations where you've felt the victim. And it also makes you see that if you're responsible for your mistakes, you're also responsible for your success. There's no luck, it's your responsability to make things right.

The article only recommends accepting blame for things that you are involved in. I would go a bit further than the article and accept the blame for things that had nothing to do with me.

Choose the moments to accept the blame carefully, like when no one else will accept it. In my experience, the consequences have never been bad and most people kind-of know you're taking one for the team and respect it. It defuses issues quickly and everyone can get on with fixing the problem.

I think this is what some Americans might refer to as being a stand-up guy. The only thing that surprises me is that this is news to anyone.

I wouldn't call it "fault" as so much as it is "responsibility". Of course, he's looking at it from the negative, but otherwise the initial success and growth of his company was to his own credit. Owning up to what you have control over is freeing, yet burdensome. One also has to realize that one does not have control over everything, and when others take away from what one has done, forgiveness is still a necessary process do undergo.

It's impossible to live like that! Taking responsibility of ones actions - sure, but taking the blame even for random events is useless. Why does it matter whose fault it was, as long as you can learn from it? You don't need to feel guilty in order to spot the errors. It looks like making me feel worse about myself without any good reason. Guilt is a powerful mechanism and abusing it like that might lead bad consequences.

I'm not an expert on psychology, but just skimming Kahneman's book will tell you that focusing on negatives will be detrimental to your mental health and long-term performance. Flogging yourself may ease your pain but is not that different from teenagers cutting their body to occupy themselves with pain instead of facing the problems.

A wonderful book that is similar in concept to this post is "Leadership and Self-Deception". I think the first part of the book: "Self-deception and the Box" is what Derek is describing here and he's working on getting out of the box.

Content is a $100Bn industry. Technology is 5-6x bigger, but the tech companies trying to disrupt content are an insignificant fraction of that industry.

Meanwhile, set aside whether you believe in the reforms proposed in the Khanna memo --- reducing statutory damages, increasing cost of enforcement, reducing copyright terms. Ask instead, "was this a politically effective policy memo?" Were its goals in the 113th congress realistic? Were its arguments persuasive? Something like 35% of all congresspeople are lawyers, and this memo starts out with a highly dubious argument about the meaning of the copyright clause.

It seems to me (and I am prepared to hear smart people tell me how wrong I am about this) that a reasonable short-term goal would have been to reduce the term of copyright, ratcheting it back to where it was, say, before Sonny Bono. Instead, this "RSC" memo proposed beyond that a gift basket of what seem like mostly not-useful policy trinkets for Redditors: expanded fair use for DJ culture (really? spend political capital to modify regulations on a $100bn industry for... DJs?), lower statutory caps for damages (the MPAA and RIAA already sue for a tiny fraction of the likely liability for many infringers), and punishing false copyright claims (the claims studios take to court are overwhelmingly not false; penalizing bogus DMCA takedowns wouldn't move the dials at all).

The real copyright reform is probably something like reduced term and compulsory licensing. What was the value to the RSC of trolling the Content industry for reforms that had no chance of happening, that wouldn't have actually kept people from being bankrupted by lawsuits, that wouldn't make it easier to launch tech companies, and that at the same time manage to almost uniformly enrage rightsholders?

Was this memo really "shockingly sensible"? A lot of smart people say it was. But I wonder whether they're more shocked that any conversation could have happened at all, and not really looking closely at the content of the memo itself.

The Internet alliance or whatever it's called formed by Reddit etc. after the SOPA fiasco needs to show their support for this person right away or there would be absolutely no point to it. Putting black badges on sites for SOPA was good but this needs to be taken up by sites too. Perhaps someone(Google?) can hire him to further write more papers analyzing copyright?

The memo was widely hailed by tech policy scholars and public interests advocates.

And also the internet.

At least one thing is certain - there is vehement opposition to any sort of copyright reform, and it is embedded deep into the political system. I don't think I've ever heard of a policy proposal being met with this kind of reaction. Really pushed some buttons.

Wow, this is on top of the GOP silently kicking younger members out of budget committees earlier this week for not bowing enough to the party line[1]. The GOP has had opportunities lately to shed some of their hypocritical big-government stances; their growing younger libertarian-ish/anti-special-interests wing (Rand Paul, Justin Amash, etc) is in my view their best hope for long-term survival, but the establishment seems to be denying that as much as possible.

I'm okay with this. I'm not sure I want a party of bigots to lead the charge on copyright reform. It would put me in an awkward position. Firing the only intelligent person in the party seems to solidify this.

This probably violates HN guidelines, but I'd like to suggest a modification re: keeping the original article titles if they contain names of political parties. Remove them. They invite political arguments.

The GOP seems more and more against things going the way of the "public Interest" in favor of the few connected, this makes it even harder to build a future with the more and more progressive majority.

It would be nice if it was possible to supply a seed for the randomization algorithm. Then the image wouldn't change at every reload and it's still possible to get another one if it's desired.

I imagine it would be a bit annoying if the image changed every time I reloaded the page which I do quite a lot while developing a web page. If I were to show a page I've developed to a customer I would not like to have to explain why the images are changing all the time.

I've done a ton of scraping (mostly legal: on behalf of end users of an app on sites they have legit access to). This article misses something that affects several sites: JavaScript driven content. Faking headers and even setting cookies doesn't get around this. This is of course is easy to get around, using something like phantom.js or Selenium. Selenium is great because unlike all the whiz bang scraping techniques, you're driving a real browser and your requests look real (if you make 10000 requests to index.php and never pull down a single image, you might look a bit suspicious). There's a bit more overhead, but micro instances on EC2 can easily run 2 or 3 Selenium sessions at the same time, and at 0.3 cents per hour for spot instances, you can have 200-300 browsers going for 30-50 cents/hour.

(shameless plug) I can scrape asynchronously, anonymously, with JS wizardry, and feed it into your defined models in your MVC (e.g. Django). But! I need to get to a hacker conference on the other side of the world (29c3). Any other time of year, I'd just drop a tutorial. See profile if you'd like to help me with a consulting gig.

EDIT: Knowledge isn't zero-sum. Here's an overview of a kick-ass way to spider/scrape:

I use Scrapy to spider asynchronously. When I define the crawler bot as an object, if the site contains complicated stuff (stateful forms or javascript) I usually create methods that involve importing either Mechanize or QtWebKit. Xpath selectors are also useful for the ability to not have to specify the entire XML tree from trunk to leaf. I then import pre-existing Django models from a site I want the data to go into and write to the DB. At this point you usually have to convert some types.

I find Scrapy cleaner and more like a pipeline so it seems to produce less 'side effect kludge' than other scraping methods (if anybody has seen a complex Beautiful Soup + Mechanize scraper you know what I mean by 'side effect kludge'). It can also act as a server to return json.

Being asynchronous, you can do crazy req/s.

I will leave out how to do all this through Tor because I don't want the Tor network being abused but am happy to talk about it one on one if your interest is beyond spamming the web.

Through this + a couple of unmentioned tricks, it's possible to get insane data, so much so it crosses over into security research & could be used for pen-testing.

Web scraping, as fun as it is (and btw, this title again abuses "Fun and Profit"), is not a practice we should encourage. Yes, it's the sort of dirty practice many people do, at one point or another, but it shouldn't be glorified.

Since the third party service conducted rate-limiting based on IP address (stated in their docs), my solution was to put the code that hit their service into some client-side Javascript, and then send the results back to my server from each of the clients.

This way, the requests would appear to come from thousands of different places, since each client would presumably have their own unique IP address, and none of them would individually be going over the rate limit.

Pretty sure the browser Same Original Policy forbids this. Think about it- if this worked, you'd be able to scrape inside corporate firewalls simply by having users visit your website from behind the firewall.

In the past, I have successfully used HtmlUnit to fulfill my admittedly limited scraping needs.

It runs headless, but it has a virtual head designed to pretend it's a user visting a web application to be be tested for QA purposes. You just program it to go through the motions of a human visting a site to be tested (or scraped). E.g., click here, get some response. For each whatever in the response, click and aggregate the results in your output (to whatever granularity).

Alas, it's in Java. But, if you use JRuby, you can avoid most of the nastiness that implies. (You do need to know Java, but at least you don't have to write Java.)

Hartley, what is your recommended toolkit?

I note you mentioned the problem of dynamically generated content. You develop your plan of attack using the browser plus Chrome Inspector or Firebug. So far, so good. But what if you want to be headless? Then you need something that will generate a DOM as if presenting a real user interface but instead simply returns a reference to the DOM tree that you are free to scan and react to.

Another issue not covered: file downloads. Let's say you have a process that creates a dynamic image, or logs in and downloads dynamic PDFs. Even Selenium can't handle this (the download dialog is an OS-level feature). At one point I was able to get Chrome to auto-download in Selenium, but had zero control over filename and where it was saving. I ended up using iMacros (the pay version) to drive this (using Windows instances: their Linux version is very immature comparably).

I love HTML scraping.But Javascript???...The juiciest data sets these days are increasingly in JS. For the love of me i can't get around scraping JS :(

I do know that Selenium can be used for this...but am yet to see a decent example for the same. Does anyone have any good resources/examples on JS scraping that they could share??I would be eternally grateful.

What I wish I could do is capture Flash audio (or any audio) streams with my Mac. All I want is to listen to the audio-only content with an audio player when I'm out driving or jogging, etc. Audio-only content that has to be played off a web page usually runs into the contradiction that if I'm in a position to click buttons on my web browser (not driving, for example), I'm in a position to do my REAL work and have no time to listen to the audio. I'll go to the web page, see whatever ads they may have, but then I'd like to be able to "scrape" the audio stream into a file so I don't have to sit there staring at a static web page the whole time I'm listening.

It's pointless to think of it as "wrong" for third-parties to web-scrape. Entities will do as they must to survive. The onus of mitigating web scraping, if in the interests of the publisher, is on the publisher.

As a startup developer, third-party scraping is something I need to be aware of, that I need to defend against if doing so suits my interests. A little bit of research shows that this is not impractical. Dynamic IP restrictions (or slowbanning), rudimentary data watermarking, caching of anonymous request output all mitigate this. Spot-checking popular content by running it through Google Search requires all of five minutes per week. At that point, the specific situation can be addressed holistically (a simple attribution license might make everyone happy). With enough research, one might consider hellbanning the offender (serving bogus content to requests satisfying some certain heuristic) as a deterrent. A legal pursuit with its cost would likely be a last resort.

This illustrates the significant difference between the use-cases of "web APIs" and conventional APIs, that the former are more like a database CRUD (including REST), rather than a request for computation. They (usually) are an alternative interface to a website (a GUI), and that's how most websites are used. e.g. an API for HN would allow story/comment retrieval, voting, submission, commenting.

When scraping HTML where data gets populated with js/ajax, you can get a web inspector to look at where that data is coming from and manually GET it and it will likely be in some nice JSON.

Scraping used to be the way to get data back in the days, but websites also didn't change their layout/structure on a weekly basis too back then and were much more static when it came to the structure.

Having recently written a small app that was forced to scrape HTML and having to update it every month to make it keep working, I can't imagine doing this for a larger project and maintaining it.

From a site owner's perspective: if you have a LOT of data then scraping can be very disruptive. I've had someone scraping my site for literally months, using hundreds of different open proxies, plus multiple faked user-agents, in order to defeat scraping detection. At one point they were accessing my site over 300,000 times per day (3.5/sec), which exceeded the level of the next busiest (and welcome) agent... Googlebot. In total I estimate this person has made more than 30 million fetch attempts over the past few months. I eventually figured out a unique signature for their bot and blocked 95%+ of their attempts, but they still kept trying. I managed to find a contact for their network administrator and the constant door-knocking finally stopped today.

Be careful. I got banned from Google for scraping. I did a few hundred thousand searches one day, and that night, they banned my office IP address for a week. This was in 2001, so I estimate I cost them a few hundred dollars, which is now impossible to repay. :(

I had to do some scraping of a rather Javascript-heavy site last year - I found the entire process was made almost trivial using Ruby and Nokugiri. Particularly relevant for a non-uber-programmer like me, it's simple to use, as well as powerful.

Not every site. There is data I would really love to access on Facebook without having to gain specific authorization from the user. It's odd that for most user profiles the most you can extract via the graph API (with no access token) is their name and sex. Whereas I can visit their profile page in the browser, see all sorts of info and latest updates (and not even be friends with them)

The problem with scraping instead of using the API is that when the website makes even a slight change to their markup it breaks your code. I have had that experience and it's a living hell. I can say it's not worth it to scrap when there is an API available.

when i need to scrap a webpage, i use phpQuery (http://code.google.com/p/phpquery/), it's dead simple if you have experience with jQuery and i get all the benefits of a server-side programming language.

This is a shameless plug but I've created a service that aims to help with a lot of the issues that OP describes such as rate limiting, JS and scaling. It's a bit like Heroku for web scraping and automation. It's still in beta but if anyone is interested then check out http://tubes.io.

There is just one major trouble with not needing stinking API. You can not POST as a possible client without requiring them to give their password to you, which actually would give you full access to their account instead of limited access with API.

So bloody true. A web page is a resource just like an xml doc, there's no reason public facing urls and web content can't be treated as such and I regularly take advantage of that fact aswell. great post

I think the author just completly missed the point with API vs Screen scraping.The API allows for accessing structured data. Even if the website changes once, the datas would be accessible the same way through the API.Whereas, the author, would have to rewrite his code each time an update his made to the front-office code of the website.

A simple API providing simple json response with http basic auth is far more efficient than a screen scraping program where you have to parse the response using HTML / XML parsers.

My favorite new feature: you can finally search through your own gists!

I use gists all day every day: it's where I dump and share pieces of reusable code. It's always been a huge pain to search for something you know you gisted, but can no longer find without browsing through 20 pages of 3-line excerpts. Search makes this a million times better.

In my opinion: In a couple of years we will not use Emacs or Visual Studio or anything like that: Most of programming will be done socially in Github via small changes directly from the website, from hundreds of people... it'd simplify the process of collaborative programming a lot.

I love it -- I've been looking for a place to put trivial programs that is lighter-weight than creating your own repository.

I just wish that the advertising when you embed into another website was a bit more subtle. Right now it's a full line of text "This Gist brought to you by GitHub" that is tagged onto the bottom of every embed. If your blog post has a lot of these, especially short ones, it can quickly become intrusive.

I'd love to see the ability to print (with the wonderful formating in-tact, and none of the interface) directly from my Gists. There are various bookmarklets out in the wild that attempt to do this, all of which are of course broken after this update.

I just had a need to do this with a bit of documentation I was writing. Ended up modifying the DOM to show/hide what I wanted and allow the screen stylesheet to be used in printing (to PDF).

When I'm learning a new framework / lib / language I love to look at other people's gists for good (and bad!) working examples. That said, I would LOVE the ability to sort search results by date / number of comments / forks etc.

It's about time. I've been waiting for Github to overhaul the Gists functionality, I've been ever-increasingly relying on it to store my snippets as well as posting code in blog posts, and being able to search through your own Gists is a MASSIVE addition.

I've really been loving all of these Github updates the past year, they've been killing it and it's good to see they've finally given Gist some love.

GitHub sees the value in the snippet concept behind Gist and has been giving it serious limelight lately.

I want to see Gist evolve into a Dribbble/Behance gallery but for showing off code snippets of value instead of images. We can already convey advanced ideas simply in few lines. To provide a little more story behind our contribution and share it in a small network could really build a nice broad community whose infrastructure has been laid already.

As many have already said, Github just keeps delivering quality, well-conceived and well-integrated updates. I hope they keep coming.

Now for my shameless plug:

If you use Gist as a means to share web-related snippets, you might be amazed by the Plunker editor (http://plnkr.co/edit/). It was inspired by Gist, but will let you actually 'run' your code (and will do some other cool stuff like real-time collaboration, previewing and linting). This tool has become quite popular in the AngularJS community and I hope to see more groups get on board in the future!

Heads up, opening a Gist to edit/view from my list in a new tab (CTRL + clicking) doesn't work in Chrome 21.0.1180.75. The Stylesheet doesn't seem to load, not sure if this is an individual issue on my system.