Home » Posts tagged "Big Data"

Data is a people problem

Troy Magennis, an Agile consultant and founder of Focused Objectives who made a keynote presentation at the 2018 Agile conference in San Diego this week. According to Magennis, you can collect all the data in the world and make charts and dashboards, but if people...
Continue reading »

It’s not all Big Data

The funny thing is many problems aren’t big enough to use the fanciest big data solutions. Sure, companies like Google or Yahoo track all of our web browsing; they have data files measured in petabytes or yottabytes. But most companies have data sets that can...
Continue reading »

Yes, we should keep our data for a long time

  Modern ethos is that all data is valuable, should be stored forever, and that machine learning will one day magically find the value of it. You’ve probably seen that EMC picture about how there will be 44 zettabytes of data by 2020? Remember how...
Continue reading »

Dealing with the Uncertainty in Scientific Results

MOST EVIDENCE IN the empirical sciences is statistical in nature, and scientists rely on a variety of statistical tests to distinguish valid scientific discoveries from spurious ones. Unfortunately, there is a growing recognition that many important research findings based on statistical evidence are not reproducible,...
Continue reading »

Doing The Impossible By Using Simple Math

This is not to say Musk will realize his most ambitious goals: landing on Mars, electrifying much of the world’s transportation fleet, and accelerating the transition from a fossil-fuel to a carbon-free economy. But each is closer than most thought possible in such a short...
Continue reading »

Keeping it Simple for a Reason

However, data will also exhibit idiosyncrasies that result from the randomness in the process of data sampling and do not say anything about the process that generated them—these idiosyncrasies will disappear if we resample new data from the process. Teasing out the true properties of...
Continue reading »

Lies, Damn Lies, and Project Statistics

Harvey’s term for torturing the data until it confesses is “p-hacking,” a reference to the p-value, a measure of statistical significance. P-hacking is also known as overfitting, data-mining—or data-snooping, the coinage of Andrew Lo, director of MIT’s Laboratory of Financial Engineering. Says Lo: “The more...
Continue reading »

Your Big Data Is At Your Fingertips

So Bull Moose was able to anticipate demand for then-unknown bands such as Pearl Jam and Limp Bizkit before competitors. “We ordered stuff far better than others,” Wickard says, “but we also returned far less merchandise.” Over the past two decades, Wickard and a handful...
Continue reading »

Lessons Learned From Finally Using Medical Big Data

Lessons Learned From Finally Using Medical Big Data
Several Boston hospitals began building that platform in 2004 to enable researchers to query vast databases of patients’ electronic health records, letting them study treatment outcomes …. “We could say, ‘I’m looking for age-50 Asian females who were treated with stage III breast cancer,’ …...
Continue reading »

Nobody Is Healthy, You Just Need To Be Tested More

Nobody Is Healthy You Just Need To Be Tested More
“As my internist friend likes to say, ‘Nobody’s healthy — they just haven’t been tested enough,’” says Nigel Paneth, a professor of epidemiology and biostatistics at Michigan State University. Hacking the Human OS, IEEE Spectrum, June 2015. I love data.  Once I started to collect...
Continue reading »