{"id":5530,"date":"2013-02-08T12:53:32","date_gmt":"2013-02-08T20:53:32","guid":{"rendered":"http:\/\/www.blackswanreport.com\/blog\/?p=5530"},"modified":"2013-02-08T12:53:32","modified_gmt":"2013-02-08T20:53:32","slug":"beware-the-big-errors-of-big-data-wired-opinion-wired-com","status":"publish","type":"post","link":"https:\/\/www.blackswanreport.com\/blog\/2013\/02\/beware-the-big-errors-of-big-data-wired-opinion-wired-com\/","title":{"rendered":"Beware the Big Errors of &#8216;Big Data&#8217; | Wired Opinion | Wired.com"},"content":{"rendered":"<blockquote>\n<p>And speaking of genetics, why haven\u2019t we found much of significance in the dozen or so years since we\u2019ve decoded the human genome?<\/p>\n<p>Well, if I generate by simulation a set of 200 variables \u2014 completely random and totally unrelated to each other \u2014 with about 1,000 data points for each, then it would be near impossible not to find in it a certain number of \u201csignificant\u201d correlations of sorts. But these correlations would be entirely spurious. And while there are techniques to control the cherry-picking such as the Bonferroni adjustment, they don\u2019t catch the culprits \u2014 much as regulation didn\u2019t stop insiders from gaming the system. You can\u2019t really police researchers, particularly when they are free agents toying with the large data available on the web.<\/p>\n<p>I am not saying here that there is no information in big data. There is plenty of information. The problem \u2014 the central issue \u2014 is that the needle comes in an increasingly larger haystack.<\/p>\n<\/blockquote>\n<p>via <a href=\"http:\/\/www.wired.com\/opinion\/2013\/02\/big-data-means-big-errors-people\/\">Beware the Big Errors of &#8216;Big Data&#8217; | Wired Opinion | Wired.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>And speaking of genetics, why haven\u2019t we found much of significance in the dozen or so years since we\u2019ve decoded the human genome? Well, if I generate by simulation a set of 200 variables \u2014 completely random and totally unrelated to each other \u2014 with about 1,000 data points for each, then it would be [&hellip;]<\/p>\n","protected":false},"author":8,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[680,681,630],"class_list":["post-5530","post","type-post","status-publish","format-standard","hentry","category-contributors","tag-big-data","tag-cherry-picking","tag-wired"],"_links":{"self":[{"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/posts\/5530","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/comments?post=5530"}],"version-history":[{"count":1,"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/posts\/5530\/revisions"}],"predecessor-version":[{"id":5531,"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/posts\/5530\/revisions\/5531"}],"wp:attachment":[{"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/media?parent=5530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/categories?post=5530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.blackswanreport.com\/blog\/wp-json\/wp\/v2\/tags?post=5530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}