Even Doctors Will Be Data Scientists
We all know how it works. You walk into a doctor’s office complaining about some pain in your leg or otherwise. They take your temperature, get you on the scale, check your blood pressure, and perhaps even get out the rubber hammer. These measurements are simply snapshots at one particular instant in time and may be subject to error. This limited dataset fails to capture temporal variations or the many other important factors that are required to assess the patient’s health status. After reviewing the few measurements collected, the consultation between the patient and doctor begins. Baased on the rudimentary physical analysis, along with the discussion with the patient, the physician will assert the condition that they believe is present, followed by a recommended treatment.
This approach, which is common throughout the world, is much more based on instinct and gut feeling than a scientific approach to analyzing data. Accordingly, it seems that most decisions are made based on the opinion of the physician instead of a data-proven truth. This type of opinion-based medicine is a problem in both doctor-patient care and in medical research. This is a symptom of a lack of data, as well as years of training physicians to perform without complete data.
The data collected in a typical office visit is only a fraction of the data that could be collected if health were viewed as a data problem. And, if health were redefined as a data problem, physicians would likely need different skills to process and analyze the data.
***
Vinod Khosla is one of the most successful venture capitalists in the history of Silicon Valley. He was an original founder of Sun Microsystems, and has since gone on to finance a variety of start-up companies as a venture capitalist. While he is not a medical expert, he is a data expert. In his speech at Stanford Medicine X, Khosla highlights three major issues in medicine today:
1) Doctors are human: Doctors, like everyone else, have cognitive limitations. Some are naturally smarter than others or have deeper knowledge about a particular topic. The latter leads to biases in how they think, act, and prescribe. Most shockingly, Khosla cites that doctors often decide on a patient diagnosis in the first 30 seconds of the observation. Said another way, they base their diagnosis on a gut reaction to the symptoms that they can see or are described to them.
2) Opinions dominate medicine: Khosla asserts that medicine is much more based on opinion than data. He cites the Cleveland Clinic Doctors’ Review of Initial Diagnosis study, asserting that Cleveland Clinic doctors disagree with initial diagnoses 11 percent of the time. In 22 percent of cases, minor changes to treatment are recommended. And in a startling 18 percent of cases, major changes to treatment are recommended. As Khosla states, “This means it’s not medical science.”
3) Disagreement is common among physicians: Doctors disagree a lot. It’s so dramatic, that, Khosla states, “whether or not you have surgery is a function of whom you ask.”
Medicine is currently a process of trial and error, coupled with professional opinion.
***
The Data era in medicine will be defined by a shift from intuition and opinion to data. We can collect more data in a day now than we could in a year not too long ago. Collecting data and applying it to solve healthcare problems will transform the cost and effectiveness of medicine. The question is how quickly we can get there.
Medical schools must evolve as technology advances. Most advancement in medical schools, based on technology, have been focused on utilizing advanced tools and equipment, as opposed to addressing the core knowledge needed by a physician in the data era.
The curriculum for the first two years of medical school varies by school, but it is heavy on the sciences, the human body, and the human condition. This has been typical since the first medical schools in the 1200s. All this time, investment and history, yet the newly minted physician is unprepared for practicing in the data era.
The data era requires an augmentation in curriculum to include key skills required for data-based analysis:
*Mathematics
*Statistics
*Probability
*Data Analysis and Tools
The skills of physicians will necessarily evolve in the data era, and that has to begin in medical schools. This focus will expedite the move away from opinion-based medicine to a future that the ill prefer: prescriptions based on hardened data analysis.
***
This week, IBM is announcing a set of tools, technology, and processes to bring data science to the masses. Said another way, armed with IBM technology, everyone is a data scientist. We are democratizing the access to data in your organization.
Every organization sees Hadoop as providing an open-source, rapidly evolving platform that is capable of collecting and economically storing a large corpus of data, waiting to be tapped. Yet, most organizations are not yet fully realizing the value of Hadoop due to the lack of skilled data scientists and developers to extract valuable insight. IBM will make everyone a data scientist. We take the first steps this week by:
1) Introducing new modules for In-Hadoop analytics including SQL, Machine Learning, and R.
2) Confirming our commitment to open source with IBM BigInsights Open Platform with Apache Hadoop, to include new innovations like Apache Spark. We are excited to be a founding member of the Open Data Platform.
3) Rolling out expanded data science training for Machine Learning and Apache Spark via BigDataUniversity. Today, over 230,000 professionals and students are being trained at BigDataUniversity and we are on our way to 1 million trained.
***
We all look forward to how things will be in 15 years. You walk into a doctor’s office, and the physician immediately knows why you are there. In fact, she had discussed some data irregularities that she had spotted at your annual physical exam, six months prior. She doesn’t need to take your temperature, as she receives that data direct from your home every day. You also take your own blood pressure monthly and that is transmitted directly to your physician. Instead, the discussion immediately turns to the possible treatments, along with the probability of success with each one. Recent data from other patients with a similar history and physiology indicate that regular medication will solve the issue 95 percent of the time. With this quick diagnosis, involving no opinions, you are on your way after ten minutes, confident that the problem has been solved. This is medicine in the data era, administered by a physician steeped in mathematics and statistics. In the data era, even doctors become data scientists.
This post is adapted from my book, Big Data Revolution: What farmers, doctors, and insurance agents teach us about discovering big data patterns, Wiley, 2015. Find more on the web at http://www.bigdatarevolutionbook.com
This approach, which is common throughout the world, is much more based on instinct and gut feeling than a scientific approach to analyzing data. Accordingly, it seems that most decisions are made based on the opinion of the physician instead of a data-proven truth. This type of opinion-based medicine is a problem in both doctor-patient care and in medical research. This is a symptom of a lack of data, as well as years of training physicians to perform without complete data.
The data collected in a typical office visit is only a fraction of the data that could be collected if health were viewed as a data problem. And, if health were redefined as a data problem, physicians would likely need different skills to process and analyze the data.
***
Vinod Khosla is one of the most successful venture capitalists in the history of Silicon Valley. He was an original founder of Sun Microsystems, and has since gone on to finance a variety of start-up companies as a venture capitalist. While he is not a medical expert, he is a data expert. In his speech at Stanford Medicine X, Khosla highlights three major issues in medicine today:
1) Doctors are human: Doctors, like everyone else, have cognitive limitations. Some are naturally smarter than others or have deeper knowledge about a particular topic. The latter leads to biases in how they think, act, and prescribe. Most shockingly, Khosla cites that doctors often decide on a patient diagnosis in the first 30 seconds of the observation. Said another way, they base their diagnosis on a gut reaction to the symptoms that they can see or are described to them.
2) Opinions dominate medicine: Khosla asserts that medicine is much more based on opinion than data. He cites the Cleveland Clinic Doctors’ Review of Initial Diagnosis study, asserting that Cleveland Clinic doctors disagree with initial diagnoses 11 percent of the time. In 22 percent of cases, minor changes to treatment are recommended. And in a startling 18 percent of cases, major changes to treatment are recommended. As Khosla states, “This means it’s not medical science.”
3) Disagreement is common among physicians: Doctors disagree a lot. It’s so dramatic, that, Khosla states, “whether or not you have surgery is a function of whom you ask.”
Medicine is currently a process of trial and error, coupled with professional opinion.
***
The Data era in medicine will be defined by a shift from intuition and opinion to data. We can collect more data in a day now than we could in a year not too long ago. Collecting data and applying it to solve healthcare problems will transform the cost and effectiveness of medicine. The question is how quickly we can get there.
Medical schools must evolve as technology advances. Most advancement in medical schools, based on technology, have been focused on utilizing advanced tools and equipment, as opposed to addressing the core knowledge needed by a physician in the data era.
The curriculum for the first two years of medical school varies by school, but it is heavy on the sciences, the human body, and the human condition. This has been typical since the first medical schools in the 1200s. All this time, investment and history, yet the newly minted physician is unprepared for practicing in the data era.
The data era requires an augmentation in curriculum to include key skills required for data-based analysis:
*Mathematics
*Statistics
*Probability
*Data Analysis and Tools
The skills of physicians will necessarily evolve in the data era, and that has to begin in medical schools. This focus will expedite the move away from opinion-based medicine to a future that the ill prefer: prescriptions based on hardened data analysis.
***
This week, IBM is announcing a set of tools, technology, and processes to bring data science to the masses. Said another way, armed with IBM technology, everyone is a data scientist. We are democratizing the access to data in your organization.
Every organization sees Hadoop as providing an open-source, rapidly evolving platform that is capable of collecting and economically storing a large corpus of data, waiting to be tapped. Yet, most organizations are not yet fully realizing the value of Hadoop due to the lack of skilled data scientists and developers to extract valuable insight. IBM will make everyone a data scientist. We take the first steps this week by:
1) Introducing new modules for In-Hadoop analytics including SQL, Machine Learning, and R.
2) Confirming our commitment to open source with IBM BigInsights Open Platform with Apache Hadoop, to include new innovations like Apache Spark. We are excited to be a founding member of the Open Data Platform.
3) Rolling out expanded data science training for Machine Learning and Apache Spark via BigDataUniversity. Today, over 230,000 professionals and students are being trained at BigDataUniversity and we are on our way to 1 million trained.
***
We all look forward to how things will be in 15 years. You walk into a doctor’s office, and the physician immediately knows why you are there. In fact, she had discussed some data irregularities that she had spotted at your annual physical exam, six months prior. She doesn’t need to take your temperature, as she receives that data direct from your home every day. You also take your own blood pressure monthly and that is transmitted directly to your physician. Instead, the discussion immediately turns to the possible treatments, along with the probability of success with each one. Recent data from other patients with a similar history and physiology indicate that regular medication will solve the issue 95 percent of the time. With this quick diagnosis, involving no opinions, you are on your way after ten minutes, confident that the problem has been solved. This is medicine in the data era, administered by a physician steeped in mathematics and statistics. In the data era, even doctors become data scientists.
This post is adapted from my book, Big Data Revolution: What farmers, doctors, and insurance agents teach us about discovering big data patterns, Wiley, 2015. Find more on the web at http://www.bigdatarevolutionbook.com
Committing to One Percent
Dave Brailsford basked in the glow of the Olympics. The Great Britain cycling team just completed their participation at the 2012 Olympics in London, England, winning 70 percent of the medals in men’s cycling. Reporters probed with aggressive questioning, wanting to understand the silver bullet that led to this success. The irony: There was no silver bullet. In fact, it was the opposite of a silver bullet.
When the Great Britain cycling team, Team Sky, hired coach Dave Brailsford in 2010, the country had never won a Tour De France. In fact, the history of the sport in the country was filled with errors, mishaps, and minimal success. Historically, the team chased fads of success: new equipment, new uniforms, new techniques. But nothing changed the trajectory. Then, Dave Brailsford arrived.
Dave Brailsford fanatically talks about the aggregation of marginal gains. This concept means that by marginally improving each and every aspect of a process, the aggregation of those small gains will lead to large improvements. Brailsford’s goal was simple: one percent. He sought a one-percent improve- ment in every aspect of the cycling team.
Setting out to improve all aspects of a cycling team, the obvious places to start are in areas like nutrition, bike performance, and physical conditioning. After all, improving every meal by one percent promised a path to continued improvement. However, for Brailsford, those enhancements merely scratched the surface. He set out to improve every aspect by one percent. Not only sports massage, but the gels used for sports massage. Not only the bikes, but the grips on the bikes and, more specifically, the tackiness of the grips. He studied not only the physical conditioning, but also the sleep habits and, more specifically, the pillows used. He focused on every aspect: one-percent improvement. It’s that simple.
In 2012, a short two years after Brailsford joined the team, Great Britain won its first Tour De France. Shortly thereafter, the triumph at the Olympics in London occurred. The aggregation of the one-percent gains created superior outcomes.
***
Coincidentally, there is recent proof of the power of One Percent in the 'Deflategate' saga involving the New England Patriots. In this case, the One Percent improvement in the football led to exponentially differentiated results. See here for the details. For the record, I don't condone exploiting the One Percent outside of the rules or law.
***
You'll find this story and more like it in Big Data Revolution: What Farmers, Doctors, and Insurance Agents Teach Us About Discovering Big Data Patterns.
When the Great Britain cycling team, Team Sky, hired coach Dave Brailsford in 2010, the country had never won a Tour De France. In fact, the history of the sport in the country was filled with errors, mishaps, and minimal success. Historically, the team chased fads of success: new equipment, new uniforms, new techniques. But nothing changed the trajectory. Then, Dave Brailsford arrived.
Dave Brailsford fanatically talks about the aggregation of marginal gains. This concept means that by marginally improving each and every aspect of a process, the aggregation of those small gains will lead to large improvements. Brailsford’s goal was simple: one percent. He sought a one-percent improve- ment in every aspect of the cycling team.
Setting out to improve all aspects of a cycling team, the obvious places to start are in areas like nutrition, bike performance, and physical conditioning. After all, improving every meal by one percent promised a path to continued improvement. However, for Brailsford, those enhancements merely scratched the surface. He set out to improve every aspect by one percent. Not only sports massage, but the gels used for sports massage. Not only the bikes, but the grips on the bikes and, more specifically, the tackiness of the grips. He studied not only the physical conditioning, but also the sleep habits and, more specifically, the pillows used. He focused on every aspect: one-percent improvement. It’s that simple.
In 2012, a short two years after Brailsford joined the team, Great Britain won its first Tour De France. Shortly thereafter, the triumph at the Olympics in London occurred. The aggregation of the one-percent gains created superior outcomes.
***
Coincidentally, there is recent proof of the power of One Percent in the 'Deflategate' saga involving the New England Patriots. In this case, the One Percent improvement in the football led to exponentially differentiated results. See here for the details. For the record, I don't condone exploiting the One Percent outside of the rules or law.
***
You'll find this story and more like it in Big Data Revolution: What Farmers, Doctors, and Insurance Agents Teach Us About Discovering Big Data Patterns.
Thoughts for the New Year
While there is nothing magical about a single day in the year, the New Year does trigger a moment of reflection on personal continuous improvement. As the saying goes,
I read this post recently and thought it had a lot of good ideas. Three of the things that stuck with me are a) Being busy is a form of laziness, and b) Focus on effective instead of efficient (The difference is being outcome focused) and c) Planning what needs to be done (and setting aside time to do so).
With that context, and borrowing some ideas from the post, here are 5 goals for the New Year:
1) Each Friday, I'll write down the 3 outcomes that I must achieve for the upcoming week. Not tasks, but outcomes.
2) I'm going to set up an email rule to move all emails that I am cc'd on to a dedicated folder. I'll check that only once a day.
3) I'll reduce my number of information sources, but read more. I probably need to find some more challenging books.
4) For thoughts/direction that apply to many people, I'll reply via blog, instead of email.
5) I'll commit to three Pomodoro's a day.
"Each day you can either get better or get worse. There's no staying the same. What will it be today?"
I read this post recently and thought it had a lot of good ideas. Three of the things that stuck with me are a) Being busy is a form of laziness, and b) Focus on effective instead of efficient (The difference is being outcome focused) and c) Planning what needs to be done (and setting aside time to do so).
With that context, and borrowing some ideas from the post, here are 5 goals for the New Year:
1) Each Friday, I'll write down the 3 outcomes that I must achieve for the upcoming week. Not tasks, but outcomes.
2) I'm going to set up an email rule to move all emails that I am cc'd on to a dedicated folder. I'll check that only once a day.
3) I'll reduce my number of information sources, but read more. I probably need to find some more challenging books.
4) For thoughts/direction that apply to many people, I'll reply via blog, instead of email.
5) I'll commit to three Pomodoro's a day.
Big Data Revolution- Read the Introduction
Big Data Revolution, the book, is almost a reality.
It's available for pre-order here and the website has gone live at bigdatarevolutionbook.com.
You can read the introduction on the website and more content (podcasts, etc) will be coming soon!
Thanks for all the support I've heard from many of you.
It's available for pre-order here and the website has gone live at bigdatarevolutionbook.com.
You can read the introduction on the website and more content (podcasts, etc) will be coming soon!
Thanks for all the support I've heard from many of you.