MetaFilter posts tagged with information and Statisticshttp://www.metafilter.com/tags/information+Statistics
Posts tagged with 'information' and 'Statistics' at MetaFilter.Tue, 06 Sep 2016 00:24:25 -0800Tue, 06 Sep 2016 00:24:25 -0800en-ushttp://blogs.law.harvard.edu/tech/rss60Auditing Algorithms and Algorithmic Auditinghttp://www.metafilter.com/162085/Auditing%2DAlgorithms%2Dand%2DAlgorithmic%2DAuditing
<a href="http://www.breakingviews.com/features/review-big-datas-all-too-human-failings/">How big data increases inequality and threatens democracy</a> - "A former academic mathematician and ex-hedge fund quant exposes flaws in how information is used to assess everything from creditworthiness to policing tactics, with results that cause damage both financially and to the fabric of society. Programmed biases and a lack of feedback are among the concerns behind the clever and apt title of <a href="https://mathbabe.org/">Cathy O'Neil</a>'s book: <i><a href="https://weaponsofmathdestructionbook.com/">Weapons of Math Destruction</a></i>." <i>"Cathy O'Neil has seen Big Data from the inside, and the picture isn't pretty.</i> Weapons of Math Destruction <i>opens the curtain on algorithms that exploit people and distort the truth while posing as neutral mathematical tools. This book is wise, fierce, and desperately necessary."</i> —[<a href="http://www.metafilter.com/user/21049">mefi's own</a>] Jordan Ellenberg, University of Wisconsin-Madison, author of <i><a href="http://www.metafilter.com/activity/21049/posts/projects/">How Not To Be Wrong</a></i>
<a href="https://mathbabe.org/2016/09/01/excerpt-of-my-book-in-the-guardian/">Excerpt</a>: <a href="https://www.theguardian.com/science/2016/sep/01/how-algorithms-rule-our-working-lives">How algorithms rule our working lives</a> - "Employers are turning to mathematically modelled ways of sifting through job applications. Even when wrong, their verdicts seem beyond dispute – and they tend to punish the poor."
<a href="http://www.bloomberg.com/news/articles/2016-08-24/a-math-nerd-wants-to-stop-the-big-data-monster">The 'Rithm is Gonna Get You</a> - "MathBabe Cathy O'Neil is out to stop the Big Data monster."
<blockquote>The decision to leave her job as a tenure-track math professor at Barnard College and join hedge fund D.E. Shaw in 2007 seemed like a no-brainer. Cathy O'Neil would apply her math skills to the financial markets and make three times the pay. What could go wrong?
Less than a year later, subprime mortgages imploded, the financial crisis set in, and so-called math wizards were targets for blame. "The housing crisis, the collapse of major financial institutions, the rise of unemployment—all that had been aided and abetted by mathematicians wielding magic formulas," she writes...
The book chronicles O'Neil's odyssey from math-loving nerd clutching a Rubik's Cube to <a href="http://altbanking.net/">Occupy Wall Streeter</a> pushing for banking reform; along the way, she learns how algorithms—models used by governments, schools, and companies to find patterns in data—can produce nasty, or at least unintended, consequences (the WMDs of her title).</blockquote>
<a href="http://blogs.scientificamerican.com/roots-of-unity/review-weapons-of-math-destruction/">If we develop the will, we can use big data to advance equality and justice</a> - "Weapons of math destruction, which O'Neil refers to throughout the book as WMDs, are mathematical models or algorithms that claim to quantify important traits: teacher quality, recidivism risk, creditworthiness but have harmful outcomes and often reinforce inequality, keeping the poor poor and the rich rich. They have three things in common: opacity, scale, and damage. They are often proprietary or otherwise shielded from prying eyes, so they have the effect of being a black box. They affect large numbers of people, increasing the chances that they get it wrong for some of them. And they have a negative effect on people, perhaps by encoding racism or other biases into an algorithm or enabling predatory companies to advertise selectively to vulnerable people, or even by causing a global financial crisis."
<a href="http://time.com/4471451/cathy-oneil-math-destruction/">This Mathematician Says Big Data Is Causing a 'Silent Financial Crisis'</a> - "Like the dark financial arts employed in the run up to the 2008 financial crisis, the Big Data algorithms that sort us into piles of 'worthy' and 'unworthy' are mostly opaque and unregulated, not to mention generated (and used) by large multinational firms with huge lobbying power to keep it that way."
more from mathbabe...
<ul><li><a href="https://mathbabe.org/2016/07/27/reform-the-cfaa/">Reform the CFAA</a> - "The Computer Fraud and Abuse Act is <a href="http://continuations.com/post/149178213205/the-terrible-no-good-cfaa-giving-more-power-to">badly in need of reform</a>... Specifically, the CFAA keeps researchers from understanding how algorithms work."</li>
<li><a href="https://mathbabe.org/2016/08/11/donald-trump-is-like-a-biased-machine-learning-algorithm/">Donald Trump is like a biased machine learning algorithm</a> - "The reason I bring this up: first of all, it's a great way of understanding how machine learning algorithms can give us stuff we absolutely don't want, even though they fundamentally lack prior agendas."</li>
<li><a href="https://mathbabe.org/2016/07/22/auditing-algorithms/">Auditing Algorithms</a> - "I've started a company called ORCAA, which stands for O'Neil Risk Consulting and Algorithmic Auditing and is pronounced 'orcaaaaaa'. ORCAA will audit algorithms and conduct risk assessments for algorithms, first as a consulting entity and eventually, if all goes well, as a more formal auditing firm, with open methodologies and toolkits."</li>
<li><a href="https://mathbabe.org/2016/07/11/when-is-ai-appropriate/">When is AI appropriate?</a> - "The short version of my answer is, AI can be made appropriate if it's thoughtfully done, but most AI shops are <a href="https://twitter.com/FrankPasquale/status/771322585158127617">not set up to be at all thoughtful</a> about how it's done."</li>
<li><a href="https://mathbabe.org/2016/07/25/horrifying-new-credit-scoring-in-china/">Horrifying New Credit Scoring in China</a> - "ZestFinance is the American company, led by ex-Googler Douglas Merrill who likes to say 'all data is credit data'... <a href="http://www.theverge.com/2016/9/1/12725804/baidu-machine-learning-open-source-paddle">Baidu is the Google of China</a>. So they have a shit ton of browsing history on people. Things like, 'symptoms for Hepatitis' or 'how do I get a job'. In other words, the <a href="http://a16z.com/2016/07/24/money-as-message/">company collects information</a> on a person's most vulnerable hopes and fears. Now put these two together, which they already did thankyouverymuch, and you've got a toxic cocktail of personal information, on the one hand, and absolutely no hesitation in using information against people, on the other."</li>
<li><a href="https://mathbabe.org/2016/05/17/white-house-report-on-big-data-and-civil-rights/">White House report on big data and civil rights</a> - "Last week the White House issued a report entitled <a href="https://www.whitehouse.gov/blog/2016/05/04/big-risks-big-opportunities-intersection-big-data-and-civil-rights">Big Risks, Big Opportunities: the Intersection of Big Data and Civil Rights</a>. Specifically, the authors were United States C.T.O. <a href="https://www.whitehouse.gov/blog/author/megan-smith">Megan Smith</a>, Chief Data Scientist <a href="https://www.whitehouse.gov/blog/author/dj-patil">DJ Patil</a>, and <a href="https://www.whitehouse.gov/blog/author/cecilia-mu%C3%B1oz">Cecilia Munoz</a>, who is Assistant to the President and Director of the Domestic Policy Council. It is a remarkable report, and covered a lot in 24 readable pages. I was especially excited to see the following paragraph in <a href="https://www.whitehouse.gov/blog/2016/05/04/big-risks-big-opportunities-intersection-big-data-and-civil-rights">the summary of the report</a>: 'Using case studies on credit lending, employment, higher education, and criminal justice, the <a href="https://www.whitehouse.gov/sites/default/files/microsites/ostp/2016_0504_data_discrimination.pdf">report we are releasing today</a> illustrates how big data techniques can be used to detect bias and prevent discrimination. It also demonstrates the risks involved, particularly how technologies can deliberately or inadvertently perpetuate, exacerbate, or mask discrimination'. The <a href="https://www.whitehouse.gov/sites/default/files/microsites/ostp/2016_0504_data_discrimination.pdf">report itself</a> is broken up into an abstract discussion of algorithms, which for example debunks the widely held assumption that algorithms are objective, discusses problems of biased training data, and discusses problems of opacity, unfairness, and disparate impact."</li>
<li><a href="https://mathbabe.org/2016/06/03/three-ideas-for-defusing-weapons-of-math-destruction/">Three Ideas for defusing Weapons of Math Destruction</a> - "1) Use open datasets; 2) Take manual curation seriously; 3) Demand causal models."</li></ul>
also btw...
<ul><li><a href="https://freedom-to-tinker.com/blog/randomwalker/the-workshop-on-data-and-algorithmic-transparency/">The workshop on Data and Algorithmic Transparency</a> - "From online advertising to Uber to predictive policing, <a href="https://twitter.com/FrankPasquale/status/763390691884040194">algorithmic</a> <a href="https://twitter.com/FrankPasquale/status/763061056189104129">systems</a> <a href="https://twitter.com/FrankPasquale/status/770249954447286272">powered</a> by <a href="https://twitter.com/nicolasterry/status/766964589716090880">personal data</a> affect <a href="https://twitter.com/FrankPasquale/status/763105465580589056">more and more</a> of our lives. As our <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2750148">society begins to grapple</a> with the consequences of this shift, empirical investigation of these systems has proved vital to understand the potential for <a href="http://www.nytimes.com/2016/05/19/opinion/the-real-bias-built-in-at-facebook.html">discrimination</a>, <a href="http://elaineou.com/2016/08/17/linkedin-vs-the-bots/">privacy breaches</a>, and <a href="http://www.nytimes.com/2016/08/14/opinion/campaign-stops/the-election-wont-be-rigged-but-it-could-be-hacked.html">vulnerability to manipulation</a>."</li>
<li><a href="http://www.information-age.com/it-management/risk-and-compliance/123461954/why-gdpr-catalyst-global-digital-transformation">Why GDPR is the catalyst for a global digital transformation</a> - "The European Union's General Data Protection Regulation, which <a href="https://twitter.com/jathansadowski/status/760954763726901248">all businesses must comply</a> with by 2018, will trigger the next wave of global digital transformation... Recently, Facebook had to ensure that the data it had collected on EU citizens wasn't being misused in the US. The ruling resulted in US firms scrambling to get their <a href="https://twitter.com/FrankPasquale/status/763880621064261632">data agreements</a> in order and ensure the safe sharing of data."</li>
<li><a href="https://www.bloomberg.com/news/articles/2016-08-05/this-company-has-built-a-profile-on-every-american-adult">This Company Has Built a Profile on Every American Adult</a> - "For more than a decade, professional snoops have been able to search troves of public and nonpublic records—known addresses, DMV records, photographs of a person's car—and condense them into comprehensive reports costing as little as $10. Now they can combine that information with the kinds of things marketers know about you, such as which politicians you donate to, what you spend on groceries, and whether it's weird that you ate in last night, to create a portrait of your life and predict your behavior. IDI, a year-old company in the so-called data-fusion business, is the first to centralize and weaponize all that information for its customers. The Boca Raton, Fla., company's database service, idiCORE, combines public records with purchasing, demographic, and behavioral data. Chief Executive Officer Derek Dubner says the system isn't waiting for requests from clients—it's already built a profile on every American adult, including young people who wouldn't be swept up in conventional databases, which only index transactions."</li>
<li><a href="http://www.full-stop.net/2016/08/10/features/essays/jacobsilverman/what-machines-know-surveillance-anxiety-and-digitizing-the-world/">What Machines Know: Surveillance Anxiety and Digitizing the World</a> - "Who I think I am doesn't matter — what matters is what the algorithms of Google, my potential employer, my health insurer, and the Department of Homeland Security say I am."</li>
<li><a href="http://www.nytimes.com/2016/08/21/upshot/ban-the-box-an-effort-to-stop-discrimination-may-actually-increase-it.html">Ban the Box? An Effort to Stop Discrimination May Actually Increase It</a> - "<a href="https://twitter.com/jenniferdoleac/status/766848697204301824">Research suggests</a> that unintended consequences may foil well-intentioned policies."</li>
<li><a href="http://smerity.com/articles/2016/algorithms_can_be_prejudiced.html">It's ML, not magic: machine learning can be prejudiced</a> - "<a href="https://twitter.com/FrankPasquale/status/763038091753947136">If we're not careful</a>, optimizing life for some will be equivalent to handicapping life for others."</li>
<li><a href="https://www.entrepreneur.com/article/279927">Machine Learning Needs Bias Training to Overcome Stereotypes</a> - "It's time for the risks of social bias to be <a href="http://arxiv.org/abs/1608.08196">embedded deeply</a> in data science codes of ethics and education."</li>
<li><a href="http://blogs.lse.ac.uk/mediapolicyproject/2016/02/05/bittersweet-mysteries-of-machine-learning-a-provocation/">The complexity of machine learning does not make it unregulable</a> - "For the most <i>laissez-faire</i> commentators in the debate on algorithmic accountability, each step in the process of algorithmic ordering is immune from legal contestation or inspection: 1) the data gathered for processing are protected as trade secrets, 2) the processing itself is too complex for any human to understand, and 3) its outputs are 'free expression', exempt from ordinary legal restrictions. I have disputed 1) and 3) in other work, and in this provocation I deem 2) the '<a href="https://pressron.wordpress.com/2016/08/30/what-we-talk-about-when-we-talk-about-computation/">sweet mystery of machine learning</a>' approach to deflecting regulation... Even if algorithms at the heart of these processes '<a href="http://nautil.us/issue/40/learning/is-artificial-intelligence-permanently-inscrutable">transcend all understanding</a>', we can inspect the inputs (data) that go into them, restrict the contexts in which they are used, and demand outputs that avoid disparate impacts."</li>
<li><a href="https://www.theguardian.com/science/political-science/2016/aug/11/numbers-dont-need-to-be-trusted-to-shape-our-lives-they-just-need-our-attention-bbc">Numbers don't need to be trusted to shape our lives: they just need our attention</a> - "With the escalating use of metrics across all areas of our lives, we have seen the power of numbers shifting from being about faith to being about persuasion. It is no longer what numerical measures tell us, but what metrics <i>tell us to do</i>. When we look at the way that numbers are used, we see <a href="http://www.tandfonline.com/doi/full/10.1080/1369118X.2016.1216147">increasingly calculative modes of reasoning</a> deployed in our workplaces, in our consumer behaviour, and in determining whether schools, hospitals, universities, countries are failing or succeeding."</li>
<li><a href="https://theintercept.com/2016/07/29/a-famed-hacker-is-grading-thousands-of-programs-and-may-revolutionize-software-in-the-process/">A Famed Hacker Is Grading Thousands of Programs — and May Revolutionize Software in the Process</a> - "Mudge and his wife, Sarah, a former NSA mathematician, have developed a first-of-its-kind method for testing and scoring the security of software — a method inspired partly by Underwriters Laboratories, that century-old entity responsible for the familiar circled UL seal that tells you your toaster and hair dryer have been tested for safety and won't burst into flames. Called the Cyber Independent Testing Lab, the Zatkos' operation won't tell you if your software is literally incendiary, but it will give you a way to comparison-shop browsers, applications, and antivirus products according to <a href="https://medium.com/@avsa/the-truth-about-the-fork-fd040c7ca955">how hardened they are</a> against attack. It may also push software makers to improve their code to avoid a low score and remain competitive."</li>
<li><a href="http://www.nytimes.com/2016/08/01/opinion/make-algorithms-accountable.html">Make Algorithms Accountable</a> - "We need more due process protections to assure the accuracy of the algorithms that have become ubiquitous in our lives."</li></ul> tag:metafilter.com,2016:site.162085Tue, 06 Sep 2016 00:24:25 -0800kliuless"Our survey data pixelates—it's a big blur."http://www.metafilter.com/153141/Our%2Dsurvey%2Ddata%2Dpixelatesits%2Da%2Dbig%2Dblur
<a href="http://www.macleans.ca/news/canada/vanishing-canada-why-were-all-losers-in-ottawas-war-on-data/">Vanishing Canada: Why we're all losers in Ottawa's war on data. <small>[Maclean's Magazine]</small></a> <blockquote>Stories about government data and historical records being deleted, burned—even tossed into Dumpsters—have become so common in recent years that many Canadians may feel inured to them. But such accounts are only the tip of a rapidly melting iceberg. A months-long Maclean's investigation, which includes interviews with dozens of academics, scientists, statisticians, economists and librarians, has found that the federal government's "austerity" program, which resulted in staff cuts and library closures (16 libraries since 2012)—as well as arbitrary changes to policy, when it comes to data—has led to a systematic erosion of government records far deeper than most realize, with the data and data-gathering capability we do have severely compromised as a result.</blockquote> tag:metafilter.com,2015:site.153141Sat, 19 Sep 2015 09:19:20 -0800FizzChina announces it is scoring its citizens using big datahttp://www.metafilter.com/149392/China%2Dannounces%2Dit%2Dis%2Dscoring%2Dits%2Dcitizens%2Dusing%2Dbig%2Ddata
<a href="http://www.volkskrant.nl/buitenland/china-rates-its-own-citizens-including-online-behaviour~a3979668/">China rates its own citizens - including online behaviour</a>: "The Chinese government is currently implementing a nationwide electronic system, called the <a href="http://www.volkskrant.nl/buitenland/china-rates-its-own-citizens-including-online-behaviour~a3979668/">Social Credit System</a>, attributing to each of its 1,3 billion citizens a score for his or her behavior. The system will be based on various criteria, ranging from financial credibility and criminal record to social media behavior. From 2020 onwards each adult citizen should, besides his identity card, have such a credit code." <a href="http://www.metafilter.com/148966/Opting-out#6034952">see also</a>...
<a href="https://digital.law.washington.edu/dspace-law/bitstream/handle/1773.1/1318/89WLR0001.pdf">THE SCORED SOCIETY: DUE PROCESS FOR AUTOMATED PREDICTIONS</a> (<a href="https://twitter.com/interfluidity/status/595192140273029121">via</a>)
<blockquote>Big Data is increasingly mined to rank and rate individuals. Predictive algorithms assess whether we are good credit risks, desirable employees, reliable tenants, valuable customers—or deadbeats, shirkers, menaces, and "wastes of time." Crucial opportunities are on the line, including the ability to obtain loans, work, housing, and insurance. Though automated scoring is pervasive and consequential, it is also opaque and lacking oversight. In one area where regulation does prevail—credit—the law focuses on credit history, not the derivation of scores from data.
Procedural regularity is essential for those stigmatized by "artificially intelligent" scoring systems. The American due process tradition should inform basic safeguards. Regulators should be able to test scoring systems to ensure their fairness and accuracy. Individuals should be granted meaningful opportunities to challenge adverse decisions based on scores miscategorizing them. Without such protections in place, systems could launder biased and arbitrary data into powerfully stigmatizing scores.</blockquote>
<a href="http://mathbabe.org/2015/05/04/china-announces-it-is-scoring-its-citizens-using-big-data/">Cathy O'Neil writes</a>: "Given <a href="http://mathbabe.org/2015/04/29/looking-for-big-data-reading-suggestions/">my research</a> over the past couple of years, I see this kind of '<a href="https://www.youtube.com/watch?v=vnHkr84as-4">social credit scoring</a>' being <a href="http://mathbabe.org/2013/08/29/summers-lending-club-makes-money-by-bypassing-the-equal-credit-opportunity-act/">widely implemented here</a> in the United States." tag:metafilter.com,2015:site.149392Tue, 05 May 2015 14:34:45 -0800kliulessIntelligence Testshttp://www.metafilter.com/126930/Intelligence%2DTests
<a href="http://humanvarieties.org/2013/04/03/is-psychometric-g-a-myth/">Is Psychometric <i>g</i> a Myth?</a> - "As an online discussion about IQ or general intelligence grows longer, the probability of someone linking to statistician Cosma Shalizi's essay <i><a href="http://bactra.org/weblog/523.html">g, a Statistical Myth</a></i> approaches 1. Usually the link is accompanied by an assertion to the effect that Shalizi offers a definitive refutation of the concept of general mental ability, or psychometric <em>g</em>." <a href="http://infoproc.blogspot.com/2013/04/myths-sisyphus-and-g.html">Myths, Sisyphus and g</a> - "Over the years I have not encountered a single endorser of Shalizi's article who actually understands the relevant subject matter. His article is loved for its reassuring conclusions, not the strength of its arguments. I am sure many 'thinkers' resisted Darwinism, the abandonment of geocentrism, and even the notion that the Earth is a sphere, for similar psychological reasons."
<a href="http://noahpinionblog.blogspot.com/2013/04/nuthin-but-g-thang.html">Nuthin' but a 'g' thang</a>
<blockquote>So I've always had the intuitive hypothesis that there are different types of intelligence; that different people tend to process information in different ways, whether due to habit or nature.
But then there are all those people who say that intelligence can be boiled down to a single factor, the mysterious "g" (which I assume stands for either "general intelligence" or "gangsta"). Since this went against years of casual observation, I was somewhat pleased to see the eminent Cosma Shalizi write an essay debunking the notion of "g". But then I saw this blog post defending the notion of "g", and claiming that Shalizi makes a bunch of errors. Basically, the disagreement revolves around the question of why most or all psychometric tests and tasks seem positively correlated with each other. Shalizi points out that this correlation structure will naturally lead to the emergence of a "g"-like factor, even if one doesn't really exist; his opponent points out that if no "g" exists, it should be possible to design uncorrelated psychometric tests, which so far has proven extremely difficult to do.
The latter post, by a pseudonymous blogger calling himself "Dalliard", contains a bunch of references to psychometric research that I don't know about and have neither the time nor the will to evaluate, so I'm a bit stumped. Normally I'd leave the matter at that, shrug, and go read something else, but I realized that my intuitive hypothesis about intelligence didn't really seem to be explicitly stated in either of the posts. So I thought I'd explain my conjecture about how intelligence works.
In a nutshell, it's this: What if there are multiple "g's"? ...just imagine several dozen hyperplanes, and project them all onto one hyperplane... Remember that psychometric tests are <em>simple</em> mental tasks, but most of the mental tasks we do are <em>complex</em>, like computer programming or chess or writing. And for those tasks, learning and practice matter as much as innate skill, or more (for example, see <a href="http://www.scientificamerican.com/article.cfm?id=brain-study-shows-grandma">this study about the neurology of chess players</a>). Therefore, everyone can be "smart" in some way, if "smart" means "good at some complex mental task".</blockquote>
also btw <a href="http://www.npr.org/blogs/health/2013/03/07/173531832/Human-Cells-Invade-Mice-Brains-And-Make-Them-Smarter">To Make Mice Smarter, Add A Few Human Brain Cells</a> (<a href="http://www.metafilter.com/126538/Human-astrocytes-injected-into-mice-improve-learning">previously</a>) tag:metafilter.com,2013:site.126930Thu, 11 Apr 2013 10:40:00 -0800kliulessF**k Statisticshttp://www.metafilter.com/107011/Fk%2DStatistics
<a href="http://blog.okcupid.com/index.php/10-charts-about-sex/">Statistical analysis of OKCupid profiles</a> exposes some sexually fascinating revelations:<br><br>
- <a href="http://www.treehugger.com/files/2011/08/vegetarians-enjoy-oral-sex-more-says-okcupid.php">Herbivores</a> like giving oral more than omnivores<br>
- Twitter users are more likely to masturbate today<br>
- Christians and Atheists are just as likely to claim they have <em>never</em> masturbated<br>
- The correlation between men who prefer gentle sex &amp; use of the word 'boating'<br><br>
I f**king love statistics [<a href="http://www.metafilter.com/94604/More-hot-stats-from-OKCupid">previous</a> OKCupid <a href="http://www.metafilter.com/106963/Hes-a-Magic-Man-Mama">magic</a>] tag:metafilter.com,2011:site.107011Wed, 31 Aug 2011 05:05:11 -08000bviousExperimental type of typehttp://www.metafilter.com/106930/Experimental%2Dtype%2Dof%2Dtype
<a href="http://generative-typografie.de/"><em>Generative Typografie</em></a> - experimental programmatic type and infographics (demos and text <em>auf Deutsch</em>) tag:metafilter.com,2011:site.106930Sun, 28 Aug 2011 18:20:10 -0800Blazecock PileonWranglerhttp://www.metafilter.com/100225/Wrangler
<a href="http://vis.stanford.edu/">Stanford's Visualization Group</a> has produced a data cleanup web app called <a href="http://vis.stanford.edu/wrangler/">Wrangler</a> that works like <a href="http://vimeo.com/19185801">straight up magic</a>. tag:metafilter.com,2011:site.100225Fri, 04 Feb 2011 08:18:53 -0800chunking expressData Tools of the Fuuuuture ... fuuture ... future ... uture... ture ... re ...http://www.metafilter.com/99412/Data%2DTools%2Dof%2Dthe%2DFuuuuture%2Dfuuture%2Dfuture%2Duture%2Dture%2Dre
<a href="http://www.dataists.com/2011/01/our-predictions-and-hopes-for-data-science-in-2011/">Dataists</a> give their hopes and dreams for data, data tools and <a href="http://radar.oreilly.com/2010/06/what-is-data-science.html">data science</a> in 2011.
Already, Google has provided <a href="http://code.google.com/p/google-refine/">Google Refine</a> (<a href="http://www.metafilter.com/99219/Google-Refine-Data-Cleaning-Software">previously</a>) to help clean your datasets. While great <a href="http://www.r-bloggers.com/tag/ggplot2/">visualizations</a> can be created with online <a href="http://www.tableausoftware.com">tools</a> or by combining R (great <a href="http://www.metafilter.com/mefi/91495">posts</a> <a href="http://www.metafilter.com/89236/The-R-Project-for-Statistical-Computing">previously</a>), with <a href="http://had.co.nz/ggplot2/">ggplot2</a>, <a href="http://www.ggobi.org">GGobi</a>, and even <a href="http://code.google.com/p/google-motion-charts-with-r/">Google Motion Charts With R</a> (already built into Google <a href="https://spreadsheets.google.com/ccc?key=0Aq3K-CZwPWxOdEx2SXh4VDFDYzA5UWhYczlCdWZ2UGc&amp;hl=en#gid=10">Spreadsheets</a>).
Need data? <a href="http://www.needlebase.com/">Needlebase</a>, helps non-programmers scrape, harvest, merge, and data from the web. Or if you're introspective, <a href="http://your.flowingdata.com">Your Flowing Data</a> and <a href="http://daytum.com/">Daytum</a> provide tools to measure and chart details of your own life. tag:metafilter.com,2011:site.99412Tue, 11 Jan 2011 13:23:32 -0800stratastarLies, damn lies and just plain making things upñhttp://www.metafilter.com/57350/Lies%2Ddamn%2Dlies%2Dand%2Djust%2Dplain%2Dmaking%2Dthings%2Dup0241
The meeting's in 5 minutes, and your boss asked you to find a statistic online to prove a point. Like that the tobacco consumption in Brazil is decreasing, or that most seniors prefer cats to dogs. Whatever it is, we're now here to help you create <a href="http://www.esolutionsdata.com/">valid-looking statistics in an instant</a>! <small><a href="http://www.egmstrategy.com/ice/direct_link.cfm?bid=70D8E06B-97B8-84FE-43237B7FCAE1021F">via</a></small> tag:metafilter.com,2006:site.57350Fri, 29 Dec 2006 13:56:29 -0800signalInformation is not knowledge.http://www.metafilter.com/48815/Information%2Dis%2Dnot%2Dknowledge
<a href="http://www.bwired.nl/">Privacy?</a> No thanks. tag:metafilter.com,2006:site.48815Thu, 02 Feb 2006 11:57:47 -0800I Love TacosThe Complexity of a Controversial Concepthttp://www.metafilter.com/42922/The%2DComplexity%2Dof%2Da%2DControversial%2DConcept
<a href="http://bactra.org/bulletin/logic-of-diversity.html">The Logic of Diversity</a> "A new book, <i>The Wisdom of Crowds</i> [<a href="http://www.metafilter.com/mefi/33307">..:</a>] by <a href="http://greg.org/archive/new_yorker_magazine_database.php">The New Yorker</a> columnist James Surowiecki, has recently popularized the idea that groups can, in some ways, be smarter than their members, which is superficially similar to <a href="http://www.cscs.umich.edu/~spage/">Page's results</a>. While Surowiecki gives many examples of what one might call collective cognition, where groups out-perform isolated individuals, he really has only one explanation for this phenomenon, based on one of his examples: jelly beans [<a href="http://www.randomhouse.com/features/wisdomofcrowds/contest.html">...</a>] averaging together many independent, unbiased guesses gives a result that is probably closer to the truth than any one guess. While true — it's the <a href="http://en.wikipedia.org/wiki/Central_limit_theorem">central limit theorem</a> of statistics — it's far from being the only way in which <a href="http://www.cscs.umich.edu/diversity/">diversity</a> can be beneficial in problem solving." <a href="http://bactra.org/weblog/362.html">(Three-Toed Sloth)</a> tag:metafilter.com,2005:site.42922Mon, 20 Jun 2005 18:03:37 -0800kliulessToo Much Information?http://www.metafilter.com/3861/Too%2DMuch%2DInformation
<a href="http://www.sims.berkeley.edu/how-much-info/">Too Much Information?</a> Heavy information overload: the world's total yearly production of print, film, optical, and magnetic content would require roughly 1.5 billion gigabytes of storage. This is the equivalent of 250 megabytes per person for each man, woman, and child on earth.
tag:metafilter.com,2000:site.3861Tue, 24 Oct 2000 23:59:23 -0800faithnomore