Home » Posts tagged "Big Data"

Data Is A People Problem

I've found that "having the data" is about half the effort. The other half is not how to make it compelling. The data was always compelling in my experience because it generally showed that we were making poor decisions. That rarely went over well.
Continue reading »

It’s Not All Big Data

The funny thing is many problems aren’t big enough to use the fanciest big data solutions. Sure, companies like Google or Yahoo track all of our web browsing; they have data files measured in petabytes or yottabytes. But most companies have data sets that can...
Continue reading »

Dealing with the Uncertainty in Scientific Results

MOST EVIDENCE IN the empirical sciences is statistical in nature, and scientists rely on a variety of statistical tests to distinguish valid scientific discoveries from spurious ones. Unfortunately, there is a growing recognition that many important research findings based on statistical evidence are not reproducible,...
Continue reading »

Doing The Impossible By Using Simple Math

This is not to say Musk will realize his most ambitious goals: landing on Mars, electrifying much of the world’s transportation fleet, and accelerating the transition from a fossil-fuel to a carbon-free economy. But each is closer than most thought possible in such a short...
Continue reading »

Lies, Damn Lies, and Project Statistics

Harvey’s term for torturing the data until it confesses is “p-hacking,” a reference to the p-value, a measure of statistical significance. P-hacking is also known as overfitting, data-mining—or data-snooping, the coinage of Andrew Lo, director of MIT’s Laboratory of Financial Engineering. Says Lo: “The more...
Continue reading »

Lessons Learned From Finally Using Medical Big Data

Lessons Learned From Finally Using Medical Big Data
Several Boston hospitals began building that platform in 2004 to enable researchers to query vast databases of patients’ electronic health records, letting them study treatment outcomes …. “We could say, ‘I’m looking for age-50 Asian females who were treated with stage III breast cancer,’ …...
Continue reading »