SlideShare une entreprise Scribd logo
1  sur  35
Télécharger pour lire hors ligne
The Big Data
Economy
Why and how our
future with data is cleaner,
leaner, and smarter.
Table of Contents
3...
6...
9...
10...
12...
16...

Part I
Introduction to the Big Data Economy
Power of Information
The Challenge of Understanding Big Data
Using Data to Create Value
Different Types of Data
The Complexity of Big Data

17...
19...

Part II
Catching Up to Data - A Brief History
Rational Thinking Over Time

30...
31...
32...
33...
34...

Part III
The Future is Clear
We Have to be Bold Like Galileo
Learn to Think at the Scale of Data
The Data Will Self Identify
We Will See Interdisciplinary Data Layering Creating Value

The Big Data Economy: Our Future With Data

2
Welcome to the Big Data Economy
So much of our American identity is tied up in the iconic image: the ability to move freely,
to experience a wider array of goods than when we were regionally locked by limited
infrastructure, and the opportunity to pursue wildly different dreams than what were possible
before these paths to prosperity were laid.
For example, there’s a 76.6% chance that you or a loved one uses a car as your primary
source of transportation to and from work. Considering that nearly 130 million people were
employed in 2010, that’s a lot of cars buzzing around every day. Just a century ago this
number was impossible. Cars were expensive, roads were spotty, and the morning commutes
were much shorter than those seen in modern American cities today.
The 20th century witnessed the shift from a rail-centric transportation system to one
dominated by the automotive centric industry—one that opened up the path for people and
goods to travel wider, more distant paths on extensive public roads projects.

The Big Data Economy: Our Future With Data

3
The advent of the Internet in the latter half of the 20th Century as an accessible, user-friendly apparatus
created a new highway: the Digital Highway. Whereas the Interstate Highway System exists as an enabling
infrastructure for moving people and physical goods from one point to another along varying stretches of
road, the Digital Highway has allowed for the proliferation of unlimited amounts of data—data that travels with
far fewer physical limits than what it takes for your car to get from point A to B.
Since highways were built, American consumerism has been fed like a wild beast that drove its size
to proportions previously unseen, but there are always barriers and limits to what can be created and
consumed. There are only so many resources in the world, so many items that can be dreamed up, so many
people who are able to obtain and use them, and only so much room to store these items while not in use or
once they have become waste.
When it comes to the Data Economy, there are no such limits.
The Internet and all the amazing, mundane, useful, and incredibly useless things we do with it has created
an invisible world where data can grow at rates virtually unconfined by the laws of physics. Sure, there’s only
so much storage for all that data. As storage options grow cheaper and sensors and other data collectors
create increasingly large amounts of data, there’s no stopping the snowball. The potential for data to grow
is presumably limitless, and that prospect is not always easy to grasp. The passage of time and history has
proven that humans fear of what we don’t know—what we can’t see, what we can’t wrap our minds around,
and what can’t be sensed.
The best news is that we don’t need to fear the data-driven Digital Highway.
The Big Data Economy: Our Future With Data

4
Data is a precious thing and
will last longer than the systems
themselves.”
Tim Berners-Lee
Inventor of the World Wide Web

Share this eBook:

The Big Data Economy: Our Future With Data

5
Power of Information
If you have ever had to drive through the massive Dallas-Fort Worth metropolitan area, you’ve probably been
on one of the five stacked bridge decks of the massive High Five Interchange. It’s a tangled mass of controlled
chaos. This major infrastructure achievement replaced a clover-leaf design left over from the Eisenhower period
and was finished in 2005 because there was simply too much demand from many streams of traffic for the
previous system to hold.
The current world of data is not unlike the High Five Interchange. It is a massive undertaking necessitated
by demand. Since humanity began to record the world around us and strive to make sense of it, we’ve used
words and numbers to collect our memory and track numbers pertinent to our lives. This road has been, for
the most part, a fairly linear one. Until Gutenberg introduced the first modern printing press in 1450, knowledge
had to be written or carved into stone by hand and preserved from the elements. The book allowed collected
knowledge both to be protected from the elements and disseminated with less error than the previous
expensive, manually intensive copying manuscripts.
In centuries since, literacy rates had to catch up to the slow and steady growth of recorded knowledge that
came with easier means of disseminating it.

The Big Data Economy: Our Future With Data

6
History does not see literacy rates catch up to affordable access to information until the early 20th Century—the same
century that witnessed a massive growth of available goods, services, and transformative new technologies like the
Internet. In a world that constantly demands more needs and wants, including disparate types of consumable information,
the Internet has become a multi-layered deck of services that are constantly being used to create more and more data.
While the highest point of the High Five interchange might be scary to approach, mounting this artificial hill is necessary
for a lot of traffic to travel efficiently where they need to go. As data’s size grows we will have to place more trust in the
infrastructures that we have built for ourselves to be efficient, well-engineered pieces of an oiled machine.
There’s no denying that the world has seen enormous change in the
last 100 years. The years since cars and planes were first invented
have been momentous and the growth of data that has come with
the introduction of a consumer-based Internet has caused what a
lot of people and businesses picture to be a daunting mountain of
insurmountable information.
The current world of data is forcing businesses, governments, and
other organizations to reassess how they are functioning and how they
can use the data they have collected so rapidly. As with most shifts in
thought and function, there ends up being more good than harm done,
even if the world spins in transition.
The essential rule to remember about data is this: don’t panic.

The Big Data Economy: Our Future With Data

7
Data are becoming the new raw material of business.”

Craig Mundie
Head of Research and Strategy at Microsoft

Share this eBook:

The Big Data Economy: Our Future With Data

8
The Challenge of Understanding Big Data
The challenge in our relationship with data comes into play because it’s an idea that lives in the realm of the imagination,
and we tend not to trust things that we cannot easily see and understand. With the exception of the computer screen you
use to access specific information that you have queried or created, the Internet and its infrastructure make up an invisible
landscape of data. The veins of the Internet’s body are fiber cables buried underground. They run along power lines that
most of us forget exist. These cables run from homes to large buildings full of servers that are like brains that store and
manage the flow of massive amounts of data every second. While that infrastructure is very much a thing of the physical
world, you cannot walk up to these buildings and see the actual data housed there.
It’s invisible. This is where the challenge of understanding massive data sets comes from.
Just how much data does there have to be to be considered “massive?”
Studies suggest that nearly all of the data that has been created in human
history has materialized in the last two years, and the speed at which it will
continue to grow is staggering. The amount of information swirling around
in those unseen wires is measured in units that dazzle the layperson.
While most regular computer users think of hundreds of gigabytes as
being a lot of storage space, the average supervisor of large data sets
encounters measurements like petabytes and zettabytes: 1,048,576 or
1,099,511,627,776 gigabytes, respectively. With big, scary sounding words
and numbers like petabyte and zettabyte, the knee-jerk way to talk about
and understand this type of data is to conflate it with hype and fear.
The Big Data Economy: Our Future With Data

9
Using Data to Create Value
With any new infrastructure or system come doubt, fear, confusion, excitement, anxiety, frustrations and awe. The thing to
accept is that this magnitude of data will continue to be mostly invisible and we will be required to press our ability to think
in the abstract. This presents a new opportunity to realize that while the magnitude is huge, most of the data that exists in
this invisible landscape is worthless on its own. Once you understand that something is both invisible and mostly worthless
without some addition, it’s easy to move past the shock and awe of it and come to grips with how to use the data to create
value.
The best way to move past the misunderstanding of this invisible force is to picture what it is made up of and where it’s
coming from. Since the invention of the Internet, a number of different data streams have developed that are augmented by
businesses’ data and the data coming from mobile devices.
If the Internet is an information superhighway, there are multiple lanes that represent different types of data.
•	
•	
•	
•	

Website data: the collection of information from and about websites that includes the information on the website, web
traffic data, rankings, where the website is linked in, and so on.
Social Media data: this data set encompasses all posts, messages, and emails, how they’re interconnected and
shared, the geolocational data around them, and how often they’re shared.
Mobile data: comes from your device’s interactions—usage times, data use, where the device has been, and any other
actions that you have performed on your device.
Machine data: generated by computers and other sensors.
The Big Data Economy: Our Future With Data

10
Despite the caveats, there seems to be no turning
back. Data is in the driver’s seat. It’s there, it’s
useful and it’s valuable, even hip.”

Steve Lohr
Technology Reporter at The New York Times

The Big Data Economy: Our Future With Data

11
Different Types of Data
There are also many other types of data that are not mentioned above, primarily because they are not nearly as big an
instigator in the recent boom of data’s invisible cloud or have not yet been digitized. A macro-level list of these types of data
includes, but certainly is not limited to, the following:
•	
•	
•	

•	

•	

Medical data: the information collected about your health during medical visits, whether they’re for preventative health
or treatment of a condition.
Governmental data: this information encompasses the various branches of governmental entities and the recorded
information about the places they are governing and their constituency.
Business data: businesses have collected data about purchases, inventory, and other transaction information in
varying forms for quite some time. This data can include sales lead information, pertinent customer data, inventory
information, and purchasing patterns. This is an incredibly varied set of data.
Scientific data: one of the largest creators of new data, physicists, geographers, ecologists, biologists, and chemists
are generating profound amounts of data as science moves towards the more ambitious experiments and investigations
that modern technology allow for.
Historical data: the annals of human history largely reside in books or other forms of record, a number of which are
either currently digitized or will become so as electronic archives grow in importance and funding.

The Big Data Economy: Our Future With Data

12
All of these varying types of data populate the invisible landscape of the data superhighway, whether they are currently
digitized or not. The amount of digitized data will only grow as these varying forms of data make their way into this ether.
As these varying types of data are considered, something becomes very evident about them: whereas most of human
existence was spent dealing with one type of data, numerical, we have moved into a brave new realm of data creation that
exists around so much more than strings of numbers.
•	

•	

•	
•	

Structured data: the image most of us recall when we think of data is a spreadsheet populated with numbers that
reflect certain information. This is one type of structured data. Structured data is data that is stored methodologically
and in an orderly fashion.
Unstructured data: one of the most relevant examples of unstructured data is Twitter. The stream of tweets really has
no rhyme or reason to it, other than the fact that it is a stream of words. Unstructured data is composed of photos, text,
documents, videos, and emails, to name a few types.
Numerical: this type of data is exactly what it sounds like—data in numbers. This type of data is what we have analyzed
for thousands of years with mathematics.
Non-numeric: yet another part of an oppositional pair with a name that does a lot of the explaining itself, non-numeric
data is any other data that is not a number—words, pictures, images, etc.

The Big Data Economy: Our Future With Data

13
Consider these lists in the context of your life or the
business that you do every day. The data in your day
might consist of the geolocational information your
cell phone gives out on the way to work, the tweets of
articles sent out that are relevant to your industry (or
not), the financial data created by the bills paid, and
could end with the information your Netflix account
collected on the shows you watched and rated.
This ballooning world of invisible data probably means
many things to many people, but how do we create
some kind of value from it all? The answer lies in
collaboration.

The Big Data Economy: Our Future With Data

14
Big Data is the Holy Grail: It promises
to unearth the mathematical laws that
govern society at large.”
Albert-László Barabási
Physicist and author

Share this eBook:

The Big Data Economy: Our Future With Data

15
The Complexity of Big Data
Would you ever pay for Facebook? How about Twitter? Google is virtually a free utility to most of us these days, and
whether we like it or not, the engine has become a cultural icon of the connected age of free web utilities. Because so
many of these data-creating forces in our lives are free, they have allowed for the monstrous growth of data discussed
earlier in this chapter, and all generally without any potential for valuation as a stand-alone resource. What happens,
though, when data types are crossed? What do you get when you mix business data and social data? What about
structured governmental data with unstructured tweets from suspicious persons? And consider geolocated photo data
mixed with scientific data collected about a recent earthquake?
The dimensionality of combined data is what creates the opportunity to find real value in disparate, massive stores of
information. Just as the value of the Interstate Highway System grew when goods were transported more frequently along its
thoroughfares, the combination of two seemingly unrelated types of data frequently results in insights that did not exist within
its own context. Have you ever noticed the way your trips to the supermarket tend to work? When you subtract the marketing
tactics used to get you to pass things you don’t need, more often than not shoppers will walk out of the grocery store with
a thing or two more that was needed that was not on their list. Combining more data allows for broader, richer results and
insights that might be surprising. Just as the Highway system and the Internet were born of military defense projects that
became incredibly more valuable with civilian use, so will data increase in value as it’s allocated for more uses.
Possibilities abound across industries because of the interdimensionality of disparate data pairings—possibilities that
are already driving economic incentive and will continue to do so more and more as organizations of all kinds begin
collecting their data and sharing it. The next chapter explains the history of where analytics comes from, why we’re in for
a new order of analysis, and why analyzing data efficiently will set us all on the fast-track to a robust data economy.
The Big Data Economy: Our Future With Data

16
Catching Up to Data - A Brief History.

The Big Data Economy: Our Future With Data

17
Hypothesis has been the choice for thinkers and researchers for the length of modern thought. Formalized processes
of logic and hypothesis emerged independently around the world as a means to explain ideas and opinions, from
the supposition of planetary motion that Copernicus posited to CERN’s search for the Higgs-Boson particle. The
approaches and arguments for and against logic and hypothesis as a framework to understand the world have evolved
as we have explored, and we’re beginning to approach a time when this type of thought must evolve to solve new
magnitudes of problems.
Since we’ve created data over a period that extends all the way back into our history as intelligent modern
beings, we’ve created a lot, but the realm of data we could analyze systematically has been limited. Compared
to the amount of data that is building up in the world, much of the data from human history is an incomplete picture of
our world has existed for some time, but we’ve had the tools to understand the information we have had access to within
context, so we’ve made do.
Now that we have more and varied kinds of data to understand, the mathematical engine that has been established and
built off of for thousands of years is not quite the sufficient logical framework to understand what we are collecting, and
hypotheses have become less of a strength and more of a weakness. Theoretical mathematics has to catch up to this new
framework of data, and it has to do so faster than the human mind can really catch up. While we are intelligent creatures, our
minds have been cast into the massive amounts of data and we’re scrambling to understand it with little conceptual context
in place to guide us.
In order to understand the Herculean task of catching up to the data around us, we must consider the history of where our
mathematical mechanics came from, the type of challenges that thinkers have faced, and that slow and steady was able to
win the race up until now.

The Big Data Economy: Our Future With Data

18
Rational Thinking Over Time
The history of mathematics is long, and since we began counting, measuring, and recording the world around us, we’ve
sought ways to develop systems that quantify and track everything from the passage of the stars through the sky to what
our hunting trips looked like on cave walls. The earliest evidence of any suspected human recording of time, number, or
complex expression of numbers happened through bone carvings.
Between 20,000 and 35,000 years ago, humans on separate continents chose to carve representations of what scientists’
best guesses are mathematical expressions. Due to our own current limitation of knowledge about their world and what
might have driven these people to create these markings, we can’t be sure what the ideas expressed were, but they
demonstrate the earliest of human curiosity regarding what the world could show, and how to record things for more
complex understanding later.
The great ancient civilizations display a vast amount of knowledge based off the observable world, and what they
developed through hypothesis and proof is an established foundation of mathematics as we know it today. Hellenistic
thinkers borrowed ideas from Egyptian and Babylonian tradition and knowledge, developed their own theories and
hypotheses, and proved the theories with deductive reasoning patterns that paid more vigorous attention to sound proof
than any other system seen before. Pythagoras formed a school of thought that actually gave mathematics its name and
embraced the idea of math for math’s sake--not to mention solidifying the Pythagorean theorem, everyone’s favorite high
school geometry proof. Two hundred years later the Platonic Academy reinforced the importance of Pythagoras’ work and
allowed for the development of the hypothesis as a means for investigating and proving presumptions about the physical
world through mathematical reasoning.
The Big Data Economy: Our Future With Data

19
It’s a revolution. We’re really just getting under way.
But the march of quantification, made possible by
enormous new sources of data, will sweep through
academia, business and government. There is no area
that is going to be untouched.”
Gary King
Director of Harvard’s Institute for Quantitative Social Science

The Big Data Economy: Our Future With Data

20
As time progressed and ideas spread this system grew more refined and
complex across civilizations, but the information available for analysis was still
mostly limited by the observable world and the speed by which the human mind
could postulate a hypothesis and prove it. Developments in Arabic mathematics
between the 8th and 15th centuries A.D. included algebra the cornerstone of
modern applied mathematics. Other traditions granted their knowledge as trade
and exploration expanded what we could observe around us.
As citizens of a fast-paced world where time consistently feels crunched into
smaller and smaller proportions, this staggeringly slow process of obtaining
more and more enlightenment seems virtually unimaginable. The truth is that we
have been grasping at whatever is around us to understand the world, and until
now this set of information was far more limited than it currently is.
There was, however, a historical turning point—one that most people around
the world learn about as a turning point that fueled our tumbling run towards
the age of modernity. The collection of mathematical concepts and other
observations based on hypothesis allowed eventually allowed human thought
to move beyond superstition into digging even deeper to prove facts, a
development that paved the way for one of the most productive periods of
thought that we have ever seen: the Renaissance.

The Big Data Economy: Our Future With Data

21
Many of the advances made up until the Renaissance were made in the name of particular biases meant to prove the
existence or importance of philosophical ideas that served as many people’s understanding of life, the universe, and their
sense of morality. Plato approached the world around him not simply to observe, learn, and present an unbiased image of
his most recent point of enlightenment.
Instead, those in the Platonic school of thought approached their hypotheses with a larger end goal in mind: what is the
Truth? History proves that other mathematicians and scientists approached questions about the observable universe with
the end-goal of using what was found to prove their own philosophical framework.
Before Copernicus, astronomy was a melting pot of ideas that placed the earth is at the absolute center of the known
universe and painted the oddly modern, science fiction-like picture that there was a so-called counter-earth balancing
the cosmos.
While Copernicus was a religious man, he made the bold move to present a hypothesis based around what he could actually
observe. Instead of forcing dogma that evolved over hundreds of years into the shaping of his theory, Copernicus developed
a theory based on mathematics and left the interpretation of his heliocentric idea open, regarding it neither as truth or fact.
Copernican heliocentrism challenged Ptolemy’s theory of the earth being at the center of the universe. Placing the sun near
the center of the universe versus the earth would maintain a mathematically elegant explanation for why a year was the
length that it was, all the while retaining the idea of an ordered cosmos that was essential to appeasing skeptics of his time.
Copernicus did his best to function within the system of observation and thought that he was working in. He’d seen some
light, but was not ready to completely compromise everything he knew and believed regardless of observable fact.

The Big Data Economy: Our Future With Data

22
Big data means much more than a change in
technology, it represents a structural transformation
in how we will manage our enterprises.”

Greg Satell
Technology Reporter at Forbes

The Big Data Economy: Our Future With Data

23
Galileo Galilei, the man regarded as the father of modern science, was perhaps a greater champion
of Copernican heliocentrism than the reluctant Copernicus himself was. Galileo was one of the
Renaissance’s foremost astronomers, the creator of the modern telescope, and one of the first in
history to separate scientific thought from religious or philosophical prescriptions or presumptions.
Because Galileo believed mathematics to be the basis for the laws of nature, an idea that laid the path
for Newton’s Laws, his observations were freer of preconceived notions than any other scientist and
mathematician before him. This revolutionary freedom allowed Galileo to prove Copernicus’ theory by
applying an understanding of ellipses to the orbital system that Copernicus hypothesized—and the
consequence was his pursuit by the Church under charges of heresy.
Instead of approaching information assuming that the outcome should support
a specific philosophy or dogma, Galileo observed what he could of the solar
system for what it was, with the tools he had or invented, and attempted to
understand the recordings of what he found within the least biased framework
he knew: applied mathematics.
Galileo’s collected data was by no means small, but in the age of the Renaissance,
the amount of data collected was still within the reach of human understanding.
This was an age of collecting, watching, and patiently noting information while
attempting to understand the meaning of larger questions about the world around
them through math, science, and other means. The tools developed from the
ancient civilizations that were rediscovered, reexamined, and refined during the
Renaissance were a fantastic mechanism for this set of thinkers to develop the
groundwork for the speed of advancement that we have seen since.
The Big Data Economy: Our Future With Data

24
Since the great thinkers of the Renaissance were brave enough to sacrifice the convenience of previous assumptions
about the existence of everything around us, science and technology have bloomed—sometimes slowly like the development of energy technologies that fuel our world, sometimes with the kind of brute speed that we’ve seen since computing
became a practical prospect. These Renaissance thinkers were like the first to step out of the cave and see what they
could of everything around them that was real, that was observable, that was reality freed from superstition.
They had their chance to shake the rest of humanity into realization that there are so many things to observe and understand as their own set of facts, and they did just that with a number of other great thinkers that pressed humanity into the
modern age.
There is, however, some trouble with what has developed since The Great Rebirth. The establishment of the modern
scientific method of developing a hypothesis, performing experiments to prove it, recording the results of the experiments,
and coming to a conclusion led to the ever-increasing growth of data available about the physical world.
This process allowed the world in the centuries after the Renaissance to move into the age of modernity—into the Industrial age of machines, the age that commonly enable people to do more, move more, and make more. The growing
complexity of the world required better ways to understand what was happening than the traditional methods of scientific
and mathematical hypotheses, a requirement that led to the development of many offshoots of applied mathematics.
Since the Renaissance, the world of mathematics has seen the development of many applications including statistical analysis, probability modeling, engineering mathematics, computational mathematics, and so on. Applied mathematics enabled
the growth of a number of theories and practices, including physics, chemistry, biology, engineering, and computing.

The Big Data Economy: Our Future With Data

25
Applied mathematics and the scientific method have allowed us
to build flying machines that take us into space, printers that build
three dimensional objects, have populated the earth with over 7
billion people, and have created hand-held computing devices
that possess the same order of computing power as the Saturn
V rocket.
We are able to solve incredibly complex equations, use
math to develop theories of quantum physics, and can
understand the inner workings of our ever-fluctuating
global economy.
The method of hypothesis has been a useful tool in helping us
understand the complexity of the world around us. Until now,
though, the realm of data available to us to understand things
have been somewhat limited to small, numerical, mostly structured sets of information.
Hypothesis and mathematics allowed us to create the most sophisticated inventions that have been conceived in all of our history. What
happens, though, when our frame of thought can’t keep up with the
data created around our creations?

The Big Data Economy: Our Future With Data

26
Just as it was important for Galileo to know that Platonic thought was not always the best answer to understanding the
world around us, and for modern scientists like Einstein to challenge his contemporaries’ theories, it is very important
for those currently up against the invisible force of data challenges to question whether or not the formation of their
thought process is the right one to tackle this magnitude of data.
While we might have thousands of years of mathematical thought to guide us to this point, as well as the scientific
method of hypothesis, massive amounts of data require a new framework of thought that does not include approaching
the data with any thought of what one might want to come from it.
By attempting to apply the principles of hypothesis to the order of data that we now have, we’re attempting to fit a
square peg through a round hole. We’re like the order of scientists and mathematicians that Copernicus and Galileo
were bold enough to look past, only the order of data that we’re dealing with leave us helpless. Why?
The previous generations were dealing with data sets that were manageable by human brains, and the urgency of the
data that was collected was not so important that time was of the essence. The greatest challenge facing the data
economy is that the demands of time and the urgency of understanding the data collected leaves no room to hypothesize, test, and come to conclusions about new thought frameworks and solutions in the manner that we’ve devised
them since ancient philosophers began seeking the enlightenment of systematic understanding of the world.
In the coming age of the Data Economy, we must force ourselves to consider where we have been, wash our hands of
what tools brought us here, and be bold enough to approach the challenge with a fresh, minimal perspective.

The Big Data Economy: Our Future With Data

27
The volume of Big Data demands a change in the human
relationship to data. The algorithms have to do the work,
not the humans. The role of the analysts will be to select
the best algorithms and approve the quality of results
based on speed, quality and economics.”
Radhika Subramanian
Chief Executive Officer at Emcien

The Big Data Economy: Our Future With Data

28
4 Ways the Future of Data is
Cleaner, Leaner, and Smarter
than its storied past.

The Big Data Economy: Our Future With Data

29
The Future is Clear
The key interruption in the way we think about the universe right now is the fact that our systems were built with the
assumption that we’d have to work with what little data and knowledge we had. Building a hypothesis is based around the
notion of considering what you know, what’s possible from what you know, measuring based on experiments, and finding if
you learned anything new based on what you already knew. Now that so much data is readily available from so many places,
we have to step back and consider that thinking about outcomes within the scope of this previously limited approach simply
does not work. The sheer scope of data available to us indicates that many of the assumptions we’ve made about the world
based on limited data might, in fact, be entirely wrong.
While we may not yet know everything about everything, data is out there, just waiting to be collected or generating, and
the capacity to store it is very much in place. It’s hard to conceive of thinking like Galileo when all he had was a rudimentary
telescope to see beyond the veil of the earth’s atmosphere when we have telescopes that are cataloguing billions of stars,
their billions of attributes, and the billions of moons and satellites within their systems. As we approach larger and larger data
sets, the tools we use to analyze them cannot, by necessity, be the tools of the past. They have no biases.
We cannot know what direction data will drive us in over the coming decades, but we can understand the types of thinking
that must and must not be retained or creating to move forward in understanding this invisible force that’s changing our lives.
Here’s a list of how and why the future of data is cleaner, leaner, and smarter than its long, storied past.

The Big Data Economy: Our Future With Data

30
1

We Have to be Bold Like Galileo
One of the biggest challenges that analytics project leaders across
industries are having right now is one of general innocence, but is
shrouded in the veil of being wrong. Because we’ve always been
taught to move into a research-based project with an end-goal in
mind by the thousands of years of tradition that inform our thought
process and education, we approach new information and projects
with biased objectives. Data of the magnitude that we’re currently
encountering begs the user to allow its insights to come
independently of any presumptions or framework.
There must be an end to the hypothesis when approaching data sets this
large. If we do not end this thought pattern, we could potentially make
disastrous decisions based on data that we assumed was unbiased,
but that was, in fact, dangerously biased due to human preconception.
Imagine the trajectory of history if no one like Galileo ever made the bold
choice to stand back and let the reality of what he was observing speak
for itself, rather than impose a perspective on it. While it’s unlikely that
we’d still believe the earth was at the center of the universe, it might have
taken us much longer to come to that conclusion.

The Big Data Economy: Our Future With Data

31
2

Learn to think at the scale of data
One of the greatest limitations on human thought around
our quickly-growing technology frenzy is it’s growing faster
than we can think of ways to keep up with it. Machines
are generating large amounts of data each moment of the
day, let alone the data generated by humans through their
use of the machines. The human brain simply can’t
conceptualize new solutions fast enough to keep pace
with these developments.
That’s why we must start doing our best human thinking to
give our able-minded computers the tools needed to create
their own solutions. After all, the computer is engineered
to function in ways that our mind, or the collective minds of
many people could never function alone. Why not assume
that computers can create equations of their own if they can
quickly sift through trivial human history and tell a studio
audience that Galileo was accused of heresy on an order of
magnitude faster than a human can?
The Big Data Economy: Our Future With Data

32
3

The Data Will Self Identify
With the development of machines that can piece together the best ways to understand
their own data, we’ll also come to the point where the important data will self identify. Once
we know what data is useless, and a lot of it will be, the machines will analyze the relevant
information and we can use the results to creatively solve problems in an order of magnitude
faster than in any other point in history.

The Big Data Economy: Our Future With Data

33
4

We Will See Interdisciplinary Data
Layering Creating Value
We’ll also be able to tell if more data that has already been
deemed valuable needs to be layered onto another set to create
value. As we begin to approach large data sets, we must not
allow ourselves to make presumptions when first analyzing the
data, and we must accept that data isn’t telling us anything
when it isn’t.
If we find that data doesn’t tell us something, we can toss it out
or decide to layer more data onto it, and move on to finding
whatever value the layering might offer us.

The Big Data Economy: Our Future With Data

34
Looking Forward to Life with Big Data
The magnitude of data that we’ve come across is almost so
dazzling that we don’t know what direction to move in. While we
might have been out of the cave and into the light for quite some
time, the light of reality just got much brighter. As we remember
the history of how thought carried us to where we are now, we
have to consider the fact that trail blazers are who moved us
forward into a world that’s closer to the clearest picture of reality.
As we move forward, we’ll look at the tools that we currently
have to handle the magnitude of data that exists, which are
best, which might function too close to the old order, and
where they will take us in the Data Economy.

Click here to see data analytics
tools in action!
Share this eBook:

The Big Data Economy: Our Future With Data

35

Contenu connexe

Tendances

Making sense-of-the-chaos
Making sense-of-the-chaosMaking sense-of-the-chaos
Making sense-of-the-chaosswaipnew
 
The rise of graph databases
The rise of graph databasesThe rise of graph databases
The rise of graph databasesAlexis Martinez
 
A Big Data Timeline
A Big Data TimelineA Big Data Timeline
A Big Data TimelineBig Cloud
 
The Evolving Realities of Digital Marketing:
The Evolving Realities of Digital Marketing: The Evolving Realities of Digital Marketing:
The Evolving Realities of Digital Marketing: RENGAN SRINIVASAN
 
The implications of Big Data for BTS and COS
The implications of Big Data for BTS and COSThe implications of Big Data for BTS and COS
The implications of Big Data for BTS and COSGeorge Kershoff
 
Open Data Comitee Meeting
Open Data Comitee MeetingOpen Data Comitee Meeting
Open Data Comitee Meetingvarunity
 
Steven adler ibm big data predictions
Steven adler   ibm big data predictionsSteven adler   ibm big data predictions
Steven adler ibm big data predictionsGlobalForum
 
Transparency in Financial Disclosure Bill Lutz
Transparency in Financial Disclosure Bill LutzTransparency in Financial Disclosure Bill Lutz
Transparency in Financial Disclosure Bill LutzClarity2010
 
Ppt shark global forum session 3 2012 v4
Ppt shark global forum session 3 2012 v4Ppt shark global forum session 3 2012 v4
Ppt shark global forum session 3 2012 v4GlobalForum
 
Research issues in the big data and its Challenges
Research issues in the big data and its ChallengesResearch issues in the big data and its Challenges
Research issues in the big data and its ChallengesKathirvel Ayyaswamy
 
5. global forum sg grumbach
5. global forum sg grumbach5. global forum sg grumbach
5. global forum sg grumbachGlobalForum
 
An era of game changing insight from Big Data
An era of game changing insight from Big DataAn era of game changing insight from Big Data
An era of game changing insight from Big DataIBM Government
 
Big data introduction by quontra solutions
Big data introduction by quontra solutionsBig data introduction by quontra solutions
Big data introduction by quontra solutionsQUONTRASOLUTIONS
 
Bjmc i, dcm,unit-ii, internet as a mass medium
Bjmc i, dcm,unit-ii, internet as a mass mediumBjmc i, dcm,unit-ii, internet as a mass medium
Bjmc i, dcm,unit-ii, internet as a mass mediumRai University
 
Government 2.0: architecting for collaboration
Government 2.0: architecting for collaborationGovernment 2.0: architecting for collaboration
Government 2.0: architecting for collaborationTara Hunt
 
Bridget Jung, Executive Creative Director, Jungle
Bridget Jung, Executive Creative Director, JungleBridget Jung, Executive Creative Director, Jungle
Bridget Jung, Executive Creative Director, JungleB&T Magazine
 
Data data everywhere and not a byte to eat...
Data data everywhere and not a byte to eat...Data data everywhere and not a byte to eat...
Data data everywhere and not a byte to eat...Tim Willoughby
 
Analytics revolution and democratization of data
Analytics revolution and democratization of dataAnalytics revolution and democratization of data
Analytics revolution and democratization of dataDerek Gibson
 

Tendances (20)

Making sense-of-the-chaos
Making sense-of-the-chaosMaking sense-of-the-chaos
Making sense-of-the-chaos
 
The rise of graph databases
The rise of graph databasesThe rise of graph databases
The rise of graph databases
 
A Big Data Timeline
A Big Data TimelineA Big Data Timeline
A Big Data Timeline
 
The Evolving Realities of Digital Marketing:
The Evolving Realities of Digital Marketing: The Evolving Realities of Digital Marketing:
The Evolving Realities of Digital Marketing:
 
The implications of Big Data for BTS and COS
The implications of Big Data for BTS and COSThe implications of Big Data for BTS and COS
The implications of Big Data for BTS and COS
 
Open Data Comitee Meeting
Open Data Comitee MeetingOpen Data Comitee Meeting
Open Data Comitee Meeting
 
Steven adler ibm big data predictions
Steven adler   ibm big data predictionsSteven adler   ibm big data predictions
Steven adler ibm big data predictions
 
Transparency in Financial Disclosure Bill Lutz
Transparency in Financial Disclosure Bill LutzTransparency in Financial Disclosure Bill Lutz
Transparency in Financial Disclosure Bill Lutz
 
Ppt shark global forum session 3 2012 v4
Ppt shark global forum session 3 2012 v4Ppt shark global forum session 3 2012 v4
Ppt shark global forum session 3 2012 v4
 
Research issues in the big data and its Challenges
Research issues in the big data and its ChallengesResearch issues in the big data and its Challenges
Research issues in the big data and its Challenges
 
5. global forum sg grumbach
5. global forum sg grumbach5. global forum sg grumbach
5. global forum sg grumbach
 
An era of game changing insight from Big Data
An era of game changing insight from Big DataAn era of game changing insight from Big Data
An era of game changing insight from Big Data
 
Big data introduction by quontra solutions
Big data introduction by quontra solutionsBig data introduction by quontra solutions
Big data introduction by quontra solutions
 
Bjmc i, dcm,unit-ii, internet as a mass medium
Bjmc i, dcm,unit-ii, internet as a mass mediumBjmc i, dcm,unit-ii, internet as a mass medium
Bjmc i, dcm,unit-ii, internet as a mass medium
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
Government 2.0: architecting for collaboration
Government 2.0: architecting for collaborationGovernment 2.0: architecting for collaboration
Government 2.0: architecting for collaboration
 
Bridget Jung, Executive Creative Director, Jungle
Bridget Jung, Executive Creative Director, JungleBridget Jung, Executive Creative Director, Jungle
Bridget Jung, Executive Creative Director, Jungle
 
Data data everywhere and not a byte to eat...
Data data everywhere and not a byte to eat...Data data everywhere and not a byte to eat...
Data data everywhere and not a byte to eat...
 
Analytics revolution and democratization of data
Analytics revolution and democratization of dataAnalytics revolution and democratization of data
Analytics revolution and democratization of data
 
5 Direccoes
5 Direccoes5 Direccoes
5 Direccoes
 

En vedette

The New SAP Simon Dale, Mastering SAP: Enabling digital transformation
The New SAP Simon Dale, Mastering SAP: Enabling digital transformationThe New SAP Simon Dale, Mastering SAP: Enabling digital transformation
The New SAP Simon Dale, Mastering SAP: Enabling digital transformationSAP Asia Pacific
 
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business OperationsSpark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business OperationsSAP Technology
 
The Data Economy: 2016 Horizonwatch Trend Brief
The Data Economy:  2016 Horizonwatch Trend BriefThe Data Economy:  2016 Horizonwatch Trend Brief
The Data Economy: 2016 Horizonwatch Trend BriefBill Chamberlin
 
Spotlight on Financial Services with Calypso and SAP ASE
Spotlight on Financial Services with Calypso and SAP ASESpotlight on Financial Services with Calypso and SAP ASE
Spotlight on Financial Services with Calypso and SAP ASESAP Technology
 
SAP Helps Reduce Silos Between Business and Spatial Data
SAP Helps Reduce Silos Between Business and Spatial DataSAP Helps Reduce Silos Between Business and Spatial Data
SAP Helps Reduce Silos Between Business and Spatial DataSAP Technology
 
Cellular V2X is Gaining Momentum
Cellular V2X is Gaining MomentumCellular V2X is Gaining Momentum
Cellular V2X is Gaining MomentumQualcomm Research
 
Big Data and the Economy; Application to Global Payments Processing Industry
Big Data and the Economy; Application to Global Payments Processing IndustryBig Data and the Economy; Application to Global Payments Processing Industry
Big Data and the Economy; Application to Global Payments Processing IndustryChristine Ries
 
The new Data Economy & The results of shifting social, cultural, and personal...
The new Data Economy & The results of shifting social, cultural, and personal...The new Data Economy & The results of shifting social, cultural, and personal...
The new Data Economy & The results of shifting social, cultural, and personal...Intel IT Center
 
Infographic: What is Artificial Intelligence?
Infographic: What is Artificial Intelligence?Infographic: What is Artificial Intelligence?
Infographic: What is Artificial Intelligence?Narrative Science
 
Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...
Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...
Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...IT Network marcus evans
 
BbWorld 2013 - Learning Analytics: A Journey to Implementation
BbWorld 2013 - Learning Analytics: A Journey to ImplementationBbWorld 2013 - Learning Analytics: A Journey to Implementation
BbWorld 2013 - Learning Analytics: A Journey to Implementationekunnen
 
QuickView #3 - Big Data
QuickView #3 - Big DataQuickView #3 - Big Data
QuickView #3 - Big DataSonovate
 
Bio bigdata
Bio bigdata Bio bigdata
Bio bigdata Mk Kim
 
Influence mapping Toolbox Presentation London 2015
Influence mapping Toolbox Presentation London 2015Influence mapping Toolbox Presentation London 2015
Influence mapping Toolbox Presentation London 2015Jun Julien Matsushita
 
Unleashing Twitter Data for Fun and Insight
Unleashing Twitter Data for Fun and InsightUnleashing Twitter Data for Fun and Insight
Unleashing Twitter Data for Fun and InsightMatthew Russell
 
Searching lexis nexis in power search mode
Searching lexis nexis in power search modeSearching lexis nexis in power search mode
Searching lexis nexis in power search modeJoyce Johnston
 
Business Models in the Data Economy: A Case Study from the Business Partner D...
Business Models in the Data Economy: A Case Study from the Business Partner D...Business Models in the Data Economy: A Case Study from the Business Partner D...
Business Models in the Data Economy: A Case Study from the Business Partner D...Boris Otto
 
Can Digital Data help predict the results of the US elections?
Can Digital Data help predict the results of the US elections? Can Digital Data help predict the results of the US elections?
Can Digital Data help predict the results of the US elections? Laurence Borel
 

En vedette (20)

The New SAP Simon Dale, Mastering SAP: Enabling digital transformation
The New SAP Simon Dale, Mastering SAP: Enabling digital transformationThe New SAP Simon Dale, Mastering SAP: Enabling digital transformation
The New SAP Simon Dale, Mastering SAP: Enabling digital transformation
 
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business OperationsSpark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
 
The Data Economy: 2016 Horizonwatch Trend Brief
The Data Economy:  2016 Horizonwatch Trend BriefThe Data Economy:  2016 Horizonwatch Trend Brief
The Data Economy: 2016 Horizonwatch Trend Brief
 
Spotlight on Financial Services with Calypso and SAP ASE
Spotlight on Financial Services with Calypso and SAP ASESpotlight on Financial Services with Calypso and SAP ASE
Spotlight on Financial Services with Calypso and SAP ASE
 
SAP Helps Reduce Silos Between Business and Spatial Data
SAP Helps Reduce Silos Between Business and Spatial DataSAP Helps Reduce Silos Between Business and Spatial Data
SAP Helps Reduce Silos Between Business and Spatial Data
 
Why SAP HANA?
Why SAP HANA?Why SAP HANA?
Why SAP HANA?
 
Cellular V2X is Gaining Momentum
Cellular V2X is Gaining MomentumCellular V2X is Gaining Momentum
Cellular V2X is Gaining Momentum
 
Big Data and the Economy; Application to Global Payments Processing Industry
Big Data and the Economy; Application to Global Payments Processing IndustryBig Data and the Economy; Application to Global Payments Processing Industry
Big Data and the Economy; Application to Global Payments Processing Industry
 
The new Data Economy & The results of shifting social, cultural, and personal...
The new Data Economy & The results of shifting social, cultural, and personal...The new Data Economy & The results of shifting social, cultural, and personal...
The new Data Economy & The results of shifting social, cultural, and personal...
 
Ftc data broker report
Ftc data broker reportFtc data broker report
Ftc data broker report
 
Infographic: What is Artificial Intelligence?
Infographic: What is Artificial Intelligence?Infographic: What is Artificial Intelligence?
Infographic: What is Artificial Intelligence?
 
Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...
Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...
Australian CIO Summit 2012: A Strategic Approach To BIG DATA Analytics. Separ...
 
BbWorld 2013 - Learning Analytics: A Journey to Implementation
BbWorld 2013 - Learning Analytics: A Journey to ImplementationBbWorld 2013 - Learning Analytics: A Journey to Implementation
BbWorld 2013 - Learning Analytics: A Journey to Implementation
 
QuickView #3 - Big Data
QuickView #3 - Big DataQuickView #3 - Big Data
QuickView #3 - Big Data
 
Bio bigdata
Bio bigdata Bio bigdata
Bio bigdata
 
Influence mapping Toolbox Presentation London 2015
Influence mapping Toolbox Presentation London 2015Influence mapping Toolbox Presentation London 2015
Influence mapping Toolbox Presentation London 2015
 
Unleashing Twitter Data for Fun and Insight
Unleashing Twitter Data for Fun and InsightUnleashing Twitter Data for Fun and Insight
Unleashing Twitter Data for Fun and Insight
 
Searching lexis nexis in power search mode
Searching lexis nexis in power search modeSearching lexis nexis in power search mode
Searching lexis nexis in power search mode
 
Business Models in the Data Economy: A Case Study from the Business Partner D...
Business Models in the Data Economy: A Case Study from the Business Partner D...Business Models in the Data Economy: A Case Study from the Business Partner D...
Business Models in the Data Economy: A Case Study from the Business Partner D...
 
Can Digital Data help predict the results of the US elections?
Can Digital Data help predict the results of the US elections? Can Digital Data help predict the results of the US elections?
Can Digital Data help predict the results of the US elections?
 

Similaire à The Big Data Economy

From AI to Z: How AI is changing the relationship between people and data
From AI to Z: How AI is changing the relationship between people and dataFrom AI to Z: How AI is changing the relationship between people and data
From AI to Z: How AI is changing the relationship between people and dataiGenius
 
Short and Long of Data Driven Innovation
Short and Long of Data Driven InnovationShort and Long of Data Driven Innovation
Short and Long of Data Driven InnovationDavid De Roure
 
Smart Data Module 1 introduction to big and smart data
Smart Data Module 1 introduction to big and smart dataSmart Data Module 1 introduction to big and smart data
Smart Data Module 1 introduction to big and smart datacaniceconsulting
 
TCS Innovation Forum - The Digital World in 2025 - 28 05 15
TCS Innovation Forum - The Digital World in 2025 - 28 05 15TCS Innovation Forum - The Digital World in 2025 - 28 05 15
TCS Innovation Forum - The Digital World in 2025 - 28 05 15Future Agenda
 
The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)
The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)
The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)Gerd Leonhard
 
The future of internet
The future of internetThe future of internet
The future of internetpsp710
 
Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution gngeorge
 
Open Innovation - Winter 2014 - Socrata, Inc.
Open Innovation - Winter 2014 - Socrata, Inc.Open Innovation - Winter 2014 - Socrata, Inc.
Open Innovation - Winter 2014 - Socrata, Inc.Socrata
 
Rasetti fondazioneisi 29_06_2015
Rasetti fondazioneisi 29_06_2015Rasetti fondazioneisi 29_06_2015
Rasetti fondazioneisi 29_06_2015CSI Piemonte
 
Open Data Newthinking
Open Data NewthinkingOpen Data Newthinking
Open Data NewthinkingEdial Dekker
 
Social Bar - Open Data
Social Bar - Open DataSocial Bar - Open Data
Social Bar - Open DataEdial Dekker
 
Big Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMUBig Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMUEdison Lim Jun Hao
 
ATTOBahn NETWORK(part1)The Internet of Things
ATTOBahn NETWORK(part1)The Internet of ThingsATTOBahn NETWORK(part1)The Internet of Things
ATTOBahn NETWORK(part1)The Internet of ThingsDarryl Gray
 
A Short History of Big Data
A Short History of Big DataA Short History of Big Data
A Short History of Big DataGadi Eichhorn
 
Future of data - Insights from Discussions Building on an Initial Perspective...
Future of data - Insights from Discussions Building on an Initial Perspective...Future of data - Insights from Discussions Building on an Initial Perspective...
Future of data - Insights from Discussions Building on an Initial Perspective...Future Agenda
 
Sensory transformation
Sensory transformationSensory transformation
Sensory transformationKarlos Svoboda
 

Similaire à The Big Data Economy (20)

Big Data Paper
Big Data PaperBig Data Paper
Big Data Paper
 
From AI to Z: How AI is changing the relationship between people and data
From AI to Z: How AI is changing the relationship between people and dataFrom AI to Z: How AI is changing the relationship between people and data
From AI to Z: How AI is changing the relationship between people and data
 
7682_RiskWatch_Winter2015_The Future of Cyber Risk
7682_RiskWatch_Winter2015_The Future of Cyber Risk7682_RiskWatch_Winter2015_The Future of Cyber Risk
7682_RiskWatch_Winter2015_The Future of Cyber Risk
 
Short and Long of Data Driven Innovation
Short and Long of Data Driven InnovationShort and Long of Data Driven Innovation
Short and Long of Data Driven Innovation
 
Smart Data Module 1 introduction to big and smart data
Smart Data Module 1 introduction to big and smart dataSmart Data Module 1 introduction to big and smart data
Smart Data Module 1 introduction to big and smart data
 
TCS Innovation Forum - The Digital World in 2025 - 28 05 15
TCS Innovation Forum - The Digital World in 2025 - 28 05 15TCS Innovation Forum - The Digital World in 2025 - 28 05 15
TCS Innovation Forum - The Digital World in 2025 - 28 05 15
 
The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)
The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)
The Future of the Internet: the key trends (Futurist Speaker Gerd Leonhard)
 
The future of internet
The future of internetThe future of internet
The future of internet
 
Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution
 
Internet Essays
Internet EssaysInternet Essays
Internet Essays
 
Open Innovation - Winter 2014 - Socrata, Inc.
Open Innovation - Winter 2014 - Socrata, Inc.Open Innovation - Winter 2014 - Socrata, Inc.
Open Innovation - Winter 2014 - Socrata, Inc.
 
Rasetti fondazioneisi 29_06_2015
Rasetti fondazioneisi 29_06_2015Rasetti fondazioneisi 29_06_2015
Rasetti fondazioneisi 29_06_2015
 
Open Data Newthinking
Open Data NewthinkingOpen Data Newthinking
Open Data Newthinking
 
Social Bar - Open Data
Social Bar - Open DataSocial Bar - Open Data
Social Bar - Open Data
 
Big Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMUBig Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMU
 
ATTOBahn NETWORK(part1)The Internet of Things
ATTOBahn NETWORK(part1)The Internet of ThingsATTOBahn NETWORK(part1)The Internet of Things
ATTOBahn NETWORK(part1)The Internet of Things
 
Big Data
Big DataBig Data
Big Data
 
A Short History of Big Data
A Short History of Big DataA Short History of Big Data
A Short History of Big Data
 
Future of data - Insights from Discussions Building on an Initial Perspective...
Future of data - Insights from Discussions Building on an Initial Perspective...Future of data - Insights from Discussions Building on an Initial Perspective...
Future of data - Insights from Discussions Building on an Initial Perspective...
 
Sensory transformation
Sensory transformationSensory transformation
Sensory transformation
 

Dernier

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 

Dernier (20)

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 

The Big Data Economy

  • 1. The Big Data Economy Why and how our future with data is cleaner, leaner, and smarter.
  • 2. Table of Contents 3... 6... 9... 10... 12... 16... Part I Introduction to the Big Data Economy Power of Information The Challenge of Understanding Big Data Using Data to Create Value Different Types of Data The Complexity of Big Data 17... 19... Part II Catching Up to Data - A Brief History Rational Thinking Over Time 30... 31... 32... 33... 34... Part III The Future is Clear We Have to be Bold Like Galileo Learn to Think at the Scale of Data The Data Will Self Identify We Will See Interdisciplinary Data Layering Creating Value The Big Data Economy: Our Future With Data 2
  • 3. Welcome to the Big Data Economy So much of our American identity is tied up in the iconic image: the ability to move freely, to experience a wider array of goods than when we were regionally locked by limited infrastructure, and the opportunity to pursue wildly different dreams than what were possible before these paths to prosperity were laid. For example, there’s a 76.6% chance that you or a loved one uses a car as your primary source of transportation to and from work. Considering that nearly 130 million people were employed in 2010, that’s a lot of cars buzzing around every day. Just a century ago this number was impossible. Cars were expensive, roads were spotty, and the morning commutes were much shorter than those seen in modern American cities today. The 20th century witnessed the shift from a rail-centric transportation system to one dominated by the automotive centric industry—one that opened up the path for people and goods to travel wider, more distant paths on extensive public roads projects. The Big Data Economy: Our Future With Data 3
  • 4. The advent of the Internet in the latter half of the 20th Century as an accessible, user-friendly apparatus created a new highway: the Digital Highway. Whereas the Interstate Highway System exists as an enabling infrastructure for moving people and physical goods from one point to another along varying stretches of road, the Digital Highway has allowed for the proliferation of unlimited amounts of data—data that travels with far fewer physical limits than what it takes for your car to get from point A to B. Since highways were built, American consumerism has been fed like a wild beast that drove its size to proportions previously unseen, but there are always barriers and limits to what can be created and consumed. There are only so many resources in the world, so many items that can be dreamed up, so many people who are able to obtain and use them, and only so much room to store these items while not in use or once they have become waste. When it comes to the Data Economy, there are no such limits. The Internet and all the amazing, mundane, useful, and incredibly useless things we do with it has created an invisible world where data can grow at rates virtually unconfined by the laws of physics. Sure, there’s only so much storage for all that data. As storage options grow cheaper and sensors and other data collectors create increasingly large amounts of data, there’s no stopping the snowball. The potential for data to grow is presumably limitless, and that prospect is not always easy to grasp. The passage of time and history has proven that humans fear of what we don’t know—what we can’t see, what we can’t wrap our minds around, and what can’t be sensed. The best news is that we don’t need to fear the data-driven Digital Highway. The Big Data Economy: Our Future With Data 4
  • 5. Data is a precious thing and will last longer than the systems themselves.” Tim Berners-Lee Inventor of the World Wide Web Share this eBook: The Big Data Economy: Our Future With Data 5
  • 6. Power of Information If you have ever had to drive through the massive Dallas-Fort Worth metropolitan area, you’ve probably been on one of the five stacked bridge decks of the massive High Five Interchange. It’s a tangled mass of controlled chaos. This major infrastructure achievement replaced a clover-leaf design left over from the Eisenhower period and was finished in 2005 because there was simply too much demand from many streams of traffic for the previous system to hold. The current world of data is not unlike the High Five Interchange. It is a massive undertaking necessitated by demand. Since humanity began to record the world around us and strive to make sense of it, we’ve used words and numbers to collect our memory and track numbers pertinent to our lives. This road has been, for the most part, a fairly linear one. Until Gutenberg introduced the first modern printing press in 1450, knowledge had to be written or carved into stone by hand and preserved from the elements. The book allowed collected knowledge both to be protected from the elements and disseminated with less error than the previous expensive, manually intensive copying manuscripts. In centuries since, literacy rates had to catch up to the slow and steady growth of recorded knowledge that came with easier means of disseminating it. The Big Data Economy: Our Future With Data 6
  • 7. History does not see literacy rates catch up to affordable access to information until the early 20th Century—the same century that witnessed a massive growth of available goods, services, and transformative new technologies like the Internet. In a world that constantly demands more needs and wants, including disparate types of consumable information, the Internet has become a multi-layered deck of services that are constantly being used to create more and more data. While the highest point of the High Five interchange might be scary to approach, mounting this artificial hill is necessary for a lot of traffic to travel efficiently where they need to go. As data’s size grows we will have to place more trust in the infrastructures that we have built for ourselves to be efficient, well-engineered pieces of an oiled machine. There’s no denying that the world has seen enormous change in the last 100 years. The years since cars and planes were first invented have been momentous and the growth of data that has come with the introduction of a consumer-based Internet has caused what a lot of people and businesses picture to be a daunting mountain of insurmountable information. The current world of data is forcing businesses, governments, and other organizations to reassess how they are functioning and how they can use the data they have collected so rapidly. As with most shifts in thought and function, there ends up being more good than harm done, even if the world spins in transition. The essential rule to remember about data is this: don’t panic. The Big Data Economy: Our Future With Data 7
  • 8. Data are becoming the new raw material of business.” Craig Mundie Head of Research and Strategy at Microsoft Share this eBook: The Big Data Economy: Our Future With Data 8
  • 9. The Challenge of Understanding Big Data The challenge in our relationship with data comes into play because it’s an idea that lives in the realm of the imagination, and we tend not to trust things that we cannot easily see and understand. With the exception of the computer screen you use to access specific information that you have queried or created, the Internet and its infrastructure make up an invisible landscape of data. The veins of the Internet’s body are fiber cables buried underground. They run along power lines that most of us forget exist. These cables run from homes to large buildings full of servers that are like brains that store and manage the flow of massive amounts of data every second. While that infrastructure is very much a thing of the physical world, you cannot walk up to these buildings and see the actual data housed there. It’s invisible. This is where the challenge of understanding massive data sets comes from. Just how much data does there have to be to be considered “massive?” Studies suggest that nearly all of the data that has been created in human history has materialized in the last two years, and the speed at which it will continue to grow is staggering. The amount of information swirling around in those unseen wires is measured in units that dazzle the layperson. While most regular computer users think of hundreds of gigabytes as being a lot of storage space, the average supervisor of large data sets encounters measurements like petabytes and zettabytes: 1,048,576 or 1,099,511,627,776 gigabytes, respectively. With big, scary sounding words and numbers like petabyte and zettabyte, the knee-jerk way to talk about and understand this type of data is to conflate it with hype and fear. The Big Data Economy: Our Future With Data 9
  • 10. Using Data to Create Value With any new infrastructure or system come doubt, fear, confusion, excitement, anxiety, frustrations and awe. The thing to accept is that this magnitude of data will continue to be mostly invisible and we will be required to press our ability to think in the abstract. This presents a new opportunity to realize that while the magnitude is huge, most of the data that exists in this invisible landscape is worthless on its own. Once you understand that something is both invisible and mostly worthless without some addition, it’s easy to move past the shock and awe of it and come to grips with how to use the data to create value. The best way to move past the misunderstanding of this invisible force is to picture what it is made up of and where it’s coming from. Since the invention of the Internet, a number of different data streams have developed that are augmented by businesses’ data and the data coming from mobile devices. If the Internet is an information superhighway, there are multiple lanes that represent different types of data. • • • • Website data: the collection of information from and about websites that includes the information on the website, web traffic data, rankings, where the website is linked in, and so on. Social Media data: this data set encompasses all posts, messages, and emails, how they’re interconnected and shared, the geolocational data around them, and how often they’re shared. Mobile data: comes from your device’s interactions—usage times, data use, where the device has been, and any other actions that you have performed on your device. Machine data: generated by computers and other sensors. The Big Data Economy: Our Future With Data 10
  • 11. Despite the caveats, there seems to be no turning back. Data is in the driver’s seat. It’s there, it’s useful and it’s valuable, even hip.” Steve Lohr Technology Reporter at The New York Times The Big Data Economy: Our Future With Data 11
  • 12. Different Types of Data There are also many other types of data that are not mentioned above, primarily because they are not nearly as big an instigator in the recent boom of data’s invisible cloud or have not yet been digitized. A macro-level list of these types of data includes, but certainly is not limited to, the following: • • • • • Medical data: the information collected about your health during medical visits, whether they’re for preventative health or treatment of a condition. Governmental data: this information encompasses the various branches of governmental entities and the recorded information about the places they are governing and their constituency. Business data: businesses have collected data about purchases, inventory, and other transaction information in varying forms for quite some time. This data can include sales lead information, pertinent customer data, inventory information, and purchasing patterns. This is an incredibly varied set of data. Scientific data: one of the largest creators of new data, physicists, geographers, ecologists, biologists, and chemists are generating profound amounts of data as science moves towards the more ambitious experiments and investigations that modern technology allow for. Historical data: the annals of human history largely reside in books or other forms of record, a number of which are either currently digitized or will become so as electronic archives grow in importance and funding. The Big Data Economy: Our Future With Data 12
  • 13. All of these varying types of data populate the invisible landscape of the data superhighway, whether they are currently digitized or not. The amount of digitized data will only grow as these varying forms of data make their way into this ether. As these varying types of data are considered, something becomes very evident about them: whereas most of human existence was spent dealing with one type of data, numerical, we have moved into a brave new realm of data creation that exists around so much more than strings of numbers. • • • • Structured data: the image most of us recall when we think of data is a spreadsheet populated with numbers that reflect certain information. This is one type of structured data. Structured data is data that is stored methodologically and in an orderly fashion. Unstructured data: one of the most relevant examples of unstructured data is Twitter. The stream of tweets really has no rhyme or reason to it, other than the fact that it is a stream of words. Unstructured data is composed of photos, text, documents, videos, and emails, to name a few types. Numerical: this type of data is exactly what it sounds like—data in numbers. This type of data is what we have analyzed for thousands of years with mathematics. Non-numeric: yet another part of an oppositional pair with a name that does a lot of the explaining itself, non-numeric data is any other data that is not a number—words, pictures, images, etc. The Big Data Economy: Our Future With Data 13
  • 14. Consider these lists in the context of your life or the business that you do every day. The data in your day might consist of the geolocational information your cell phone gives out on the way to work, the tweets of articles sent out that are relevant to your industry (or not), the financial data created by the bills paid, and could end with the information your Netflix account collected on the shows you watched and rated. This ballooning world of invisible data probably means many things to many people, but how do we create some kind of value from it all? The answer lies in collaboration. The Big Data Economy: Our Future With Data 14
  • 15. Big Data is the Holy Grail: It promises to unearth the mathematical laws that govern society at large.” Albert-László Barabási Physicist and author Share this eBook: The Big Data Economy: Our Future With Data 15
  • 16. The Complexity of Big Data Would you ever pay for Facebook? How about Twitter? Google is virtually a free utility to most of us these days, and whether we like it or not, the engine has become a cultural icon of the connected age of free web utilities. Because so many of these data-creating forces in our lives are free, they have allowed for the monstrous growth of data discussed earlier in this chapter, and all generally without any potential for valuation as a stand-alone resource. What happens, though, when data types are crossed? What do you get when you mix business data and social data? What about structured governmental data with unstructured tweets from suspicious persons? And consider geolocated photo data mixed with scientific data collected about a recent earthquake? The dimensionality of combined data is what creates the opportunity to find real value in disparate, massive stores of information. Just as the value of the Interstate Highway System grew when goods were transported more frequently along its thoroughfares, the combination of two seemingly unrelated types of data frequently results in insights that did not exist within its own context. Have you ever noticed the way your trips to the supermarket tend to work? When you subtract the marketing tactics used to get you to pass things you don’t need, more often than not shoppers will walk out of the grocery store with a thing or two more that was needed that was not on their list. Combining more data allows for broader, richer results and insights that might be surprising. Just as the Highway system and the Internet were born of military defense projects that became incredibly more valuable with civilian use, so will data increase in value as it’s allocated for more uses. Possibilities abound across industries because of the interdimensionality of disparate data pairings—possibilities that are already driving economic incentive and will continue to do so more and more as organizations of all kinds begin collecting their data and sharing it. The next chapter explains the history of where analytics comes from, why we’re in for a new order of analysis, and why analyzing data efficiently will set us all on the fast-track to a robust data economy. The Big Data Economy: Our Future With Data 16
  • 17. Catching Up to Data - A Brief History. The Big Data Economy: Our Future With Data 17
  • 18. Hypothesis has been the choice for thinkers and researchers for the length of modern thought. Formalized processes of logic and hypothesis emerged independently around the world as a means to explain ideas and opinions, from the supposition of planetary motion that Copernicus posited to CERN’s search for the Higgs-Boson particle. The approaches and arguments for and against logic and hypothesis as a framework to understand the world have evolved as we have explored, and we’re beginning to approach a time when this type of thought must evolve to solve new magnitudes of problems. Since we’ve created data over a period that extends all the way back into our history as intelligent modern beings, we’ve created a lot, but the realm of data we could analyze systematically has been limited. Compared to the amount of data that is building up in the world, much of the data from human history is an incomplete picture of our world has existed for some time, but we’ve had the tools to understand the information we have had access to within context, so we’ve made do. Now that we have more and varied kinds of data to understand, the mathematical engine that has been established and built off of for thousands of years is not quite the sufficient logical framework to understand what we are collecting, and hypotheses have become less of a strength and more of a weakness. Theoretical mathematics has to catch up to this new framework of data, and it has to do so faster than the human mind can really catch up. While we are intelligent creatures, our minds have been cast into the massive amounts of data and we’re scrambling to understand it with little conceptual context in place to guide us. In order to understand the Herculean task of catching up to the data around us, we must consider the history of where our mathematical mechanics came from, the type of challenges that thinkers have faced, and that slow and steady was able to win the race up until now. The Big Data Economy: Our Future With Data 18
  • 19. Rational Thinking Over Time The history of mathematics is long, and since we began counting, measuring, and recording the world around us, we’ve sought ways to develop systems that quantify and track everything from the passage of the stars through the sky to what our hunting trips looked like on cave walls. The earliest evidence of any suspected human recording of time, number, or complex expression of numbers happened through bone carvings. Between 20,000 and 35,000 years ago, humans on separate continents chose to carve representations of what scientists’ best guesses are mathematical expressions. Due to our own current limitation of knowledge about their world and what might have driven these people to create these markings, we can’t be sure what the ideas expressed were, but they demonstrate the earliest of human curiosity regarding what the world could show, and how to record things for more complex understanding later. The great ancient civilizations display a vast amount of knowledge based off the observable world, and what they developed through hypothesis and proof is an established foundation of mathematics as we know it today. Hellenistic thinkers borrowed ideas from Egyptian and Babylonian tradition and knowledge, developed their own theories and hypotheses, and proved the theories with deductive reasoning patterns that paid more vigorous attention to sound proof than any other system seen before. Pythagoras formed a school of thought that actually gave mathematics its name and embraced the idea of math for math’s sake--not to mention solidifying the Pythagorean theorem, everyone’s favorite high school geometry proof. Two hundred years later the Platonic Academy reinforced the importance of Pythagoras’ work and allowed for the development of the hypothesis as a means for investigating and proving presumptions about the physical world through mathematical reasoning. The Big Data Economy: Our Future With Data 19
  • 20. It’s a revolution. We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.” Gary King Director of Harvard’s Institute for Quantitative Social Science The Big Data Economy: Our Future With Data 20
  • 21. As time progressed and ideas spread this system grew more refined and complex across civilizations, but the information available for analysis was still mostly limited by the observable world and the speed by which the human mind could postulate a hypothesis and prove it. Developments in Arabic mathematics between the 8th and 15th centuries A.D. included algebra the cornerstone of modern applied mathematics. Other traditions granted their knowledge as trade and exploration expanded what we could observe around us. As citizens of a fast-paced world where time consistently feels crunched into smaller and smaller proportions, this staggeringly slow process of obtaining more and more enlightenment seems virtually unimaginable. The truth is that we have been grasping at whatever is around us to understand the world, and until now this set of information was far more limited than it currently is. There was, however, a historical turning point—one that most people around the world learn about as a turning point that fueled our tumbling run towards the age of modernity. The collection of mathematical concepts and other observations based on hypothesis allowed eventually allowed human thought to move beyond superstition into digging even deeper to prove facts, a development that paved the way for one of the most productive periods of thought that we have ever seen: the Renaissance. The Big Data Economy: Our Future With Data 21
  • 22. Many of the advances made up until the Renaissance were made in the name of particular biases meant to prove the existence or importance of philosophical ideas that served as many people’s understanding of life, the universe, and their sense of morality. Plato approached the world around him not simply to observe, learn, and present an unbiased image of his most recent point of enlightenment. Instead, those in the Platonic school of thought approached their hypotheses with a larger end goal in mind: what is the Truth? History proves that other mathematicians and scientists approached questions about the observable universe with the end-goal of using what was found to prove their own philosophical framework. Before Copernicus, astronomy was a melting pot of ideas that placed the earth is at the absolute center of the known universe and painted the oddly modern, science fiction-like picture that there was a so-called counter-earth balancing the cosmos. While Copernicus was a religious man, he made the bold move to present a hypothesis based around what he could actually observe. Instead of forcing dogma that evolved over hundreds of years into the shaping of his theory, Copernicus developed a theory based on mathematics and left the interpretation of his heliocentric idea open, regarding it neither as truth or fact. Copernican heliocentrism challenged Ptolemy’s theory of the earth being at the center of the universe. Placing the sun near the center of the universe versus the earth would maintain a mathematically elegant explanation for why a year was the length that it was, all the while retaining the idea of an ordered cosmos that was essential to appeasing skeptics of his time. Copernicus did his best to function within the system of observation and thought that he was working in. He’d seen some light, but was not ready to completely compromise everything he knew and believed regardless of observable fact. The Big Data Economy: Our Future With Data 22
  • 23. Big data means much more than a change in technology, it represents a structural transformation in how we will manage our enterprises.” Greg Satell Technology Reporter at Forbes The Big Data Economy: Our Future With Data 23
  • 24. Galileo Galilei, the man regarded as the father of modern science, was perhaps a greater champion of Copernican heliocentrism than the reluctant Copernicus himself was. Galileo was one of the Renaissance’s foremost astronomers, the creator of the modern telescope, and one of the first in history to separate scientific thought from religious or philosophical prescriptions or presumptions. Because Galileo believed mathematics to be the basis for the laws of nature, an idea that laid the path for Newton’s Laws, his observations were freer of preconceived notions than any other scientist and mathematician before him. This revolutionary freedom allowed Galileo to prove Copernicus’ theory by applying an understanding of ellipses to the orbital system that Copernicus hypothesized—and the consequence was his pursuit by the Church under charges of heresy. Instead of approaching information assuming that the outcome should support a specific philosophy or dogma, Galileo observed what he could of the solar system for what it was, with the tools he had or invented, and attempted to understand the recordings of what he found within the least biased framework he knew: applied mathematics. Galileo’s collected data was by no means small, but in the age of the Renaissance, the amount of data collected was still within the reach of human understanding. This was an age of collecting, watching, and patiently noting information while attempting to understand the meaning of larger questions about the world around them through math, science, and other means. The tools developed from the ancient civilizations that were rediscovered, reexamined, and refined during the Renaissance were a fantastic mechanism for this set of thinkers to develop the groundwork for the speed of advancement that we have seen since. The Big Data Economy: Our Future With Data 24
  • 25. Since the great thinkers of the Renaissance were brave enough to sacrifice the convenience of previous assumptions about the existence of everything around us, science and technology have bloomed—sometimes slowly like the development of energy technologies that fuel our world, sometimes with the kind of brute speed that we’ve seen since computing became a practical prospect. These Renaissance thinkers were like the first to step out of the cave and see what they could of everything around them that was real, that was observable, that was reality freed from superstition. They had their chance to shake the rest of humanity into realization that there are so many things to observe and understand as their own set of facts, and they did just that with a number of other great thinkers that pressed humanity into the modern age. There is, however, some trouble with what has developed since The Great Rebirth. The establishment of the modern scientific method of developing a hypothesis, performing experiments to prove it, recording the results of the experiments, and coming to a conclusion led to the ever-increasing growth of data available about the physical world. This process allowed the world in the centuries after the Renaissance to move into the age of modernity—into the Industrial age of machines, the age that commonly enable people to do more, move more, and make more. The growing complexity of the world required better ways to understand what was happening than the traditional methods of scientific and mathematical hypotheses, a requirement that led to the development of many offshoots of applied mathematics. Since the Renaissance, the world of mathematics has seen the development of many applications including statistical analysis, probability modeling, engineering mathematics, computational mathematics, and so on. Applied mathematics enabled the growth of a number of theories and practices, including physics, chemistry, biology, engineering, and computing. The Big Data Economy: Our Future With Data 25
  • 26. Applied mathematics and the scientific method have allowed us to build flying machines that take us into space, printers that build three dimensional objects, have populated the earth with over 7 billion people, and have created hand-held computing devices that possess the same order of computing power as the Saturn V rocket. We are able to solve incredibly complex equations, use math to develop theories of quantum physics, and can understand the inner workings of our ever-fluctuating global economy. The method of hypothesis has been a useful tool in helping us understand the complexity of the world around us. Until now, though, the realm of data available to us to understand things have been somewhat limited to small, numerical, mostly structured sets of information. Hypothesis and mathematics allowed us to create the most sophisticated inventions that have been conceived in all of our history. What happens, though, when our frame of thought can’t keep up with the data created around our creations? The Big Data Economy: Our Future With Data 26
  • 27. Just as it was important for Galileo to know that Platonic thought was not always the best answer to understanding the world around us, and for modern scientists like Einstein to challenge his contemporaries’ theories, it is very important for those currently up against the invisible force of data challenges to question whether or not the formation of their thought process is the right one to tackle this magnitude of data. While we might have thousands of years of mathematical thought to guide us to this point, as well as the scientific method of hypothesis, massive amounts of data require a new framework of thought that does not include approaching the data with any thought of what one might want to come from it. By attempting to apply the principles of hypothesis to the order of data that we now have, we’re attempting to fit a square peg through a round hole. We’re like the order of scientists and mathematicians that Copernicus and Galileo were bold enough to look past, only the order of data that we’re dealing with leave us helpless. Why? The previous generations were dealing with data sets that were manageable by human brains, and the urgency of the data that was collected was not so important that time was of the essence. The greatest challenge facing the data economy is that the demands of time and the urgency of understanding the data collected leaves no room to hypothesize, test, and come to conclusions about new thought frameworks and solutions in the manner that we’ve devised them since ancient philosophers began seeking the enlightenment of systematic understanding of the world. In the coming age of the Data Economy, we must force ourselves to consider where we have been, wash our hands of what tools brought us here, and be bold enough to approach the challenge with a fresh, minimal perspective. The Big Data Economy: Our Future With Data 27
  • 28. The volume of Big Data demands a change in the human relationship to data. The algorithms have to do the work, not the humans. The role of the analysts will be to select the best algorithms and approve the quality of results based on speed, quality and economics.” Radhika Subramanian Chief Executive Officer at Emcien The Big Data Economy: Our Future With Data 28
  • 29. 4 Ways the Future of Data is Cleaner, Leaner, and Smarter than its storied past. The Big Data Economy: Our Future With Data 29
  • 30. The Future is Clear The key interruption in the way we think about the universe right now is the fact that our systems were built with the assumption that we’d have to work with what little data and knowledge we had. Building a hypothesis is based around the notion of considering what you know, what’s possible from what you know, measuring based on experiments, and finding if you learned anything new based on what you already knew. Now that so much data is readily available from so many places, we have to step back and consider that thinking about outcomes within the scope of this previously limited approach simply does not work. The sheer scope of data available to us indicates that many of the assumptions we’ve made about the world based on limited data might, in fact, be entirely wrong. While we may not yet know everything about everything, data is out there, just waiting to be collected or generating, and the capacity to store it is very much in place. It’s hard to conceive of thinking like Galileo when all he had was a rudimentary telescope to see beyond the veil of the earth’s atmosphere when we have telescopes that are cataloguing billions of stars, their billions of attributes, and the billions of moons and satellites within their systems. As we approach larger and larger data sets, the tools we use to analyze them cannot, by necessity, be the tools of the past. They have no biases. We cannot know what direction data will drive us in over the coming decades, but we can understand the types of thinking that must and must not be retained or creating to move forward in understanding this invisible force that’s changing our lives. Here’s a list of how and why the future of data is cleaner, leaner, and smarter than its long, storied past. The Big Data Economy: Our Future With Data 30
  • 31. 1 We Have to be Bold Like Galileo One of the biggest challenges that analytics project leaders across industries are having right now is one of general innocence, but is shrouded in the veil of being wrong. Because we’ve always been taught to move into a research-based project with an end-goal in mind by the thousands of years of tradition that inform our thought process and education, we approach new information and projects with biased objectives. Data of the magnitude that we’re currently encountering begs the user to allow its insights to come independently of any presumptions or framework. There must be an end to the hypothesis when approaching data sets this large. If we do not end this thought pattern, we could potentially make disastrous decisions based on data that we assumed was unbiased, but that was, in fact, dangerously biased due to human preconception. Imagine the trajectory of history if no one like Galileo ever made the bold choice to stand back and let the reality of what he was observing speak for itself, rather than impose a perspective on it. While it’s unlikely that we’d still believe the earth was at the center of the universe, it might have taken us much longer to come to that conclusion. The Big Data Economy: Our Future With Data 31
  • 32. 2 Learn to think at the scale of data One of the greatest limitations on human thought around our quickly-growing technology frenzy is it’s growing faster than we can think of ways to keep up with it. Machines are generating large amounts of data each moment of the day, let alone the data generated by humans through their use of the machines. The human brain simply can’t conceptualize new solutions fast enough to keep pace with these developments. That’s why we must start doing our best human thinking to give our able-minded computers the tools needed to create their own solutions. After all, the computer is engineered to function in ways that our mind, or the collective minds of many people could never function alone. Why not assume that computers can create equations of their own if they can quickly sift through trivial human history and tell a studio audience that Galileo was accused of heresy on an order of magnitude faster than a human can? The Big Data Economy: Our Future With Data 32
  • 33. 3 The Data Will Self Identify With the development of machines that can piece together the best ways to understand their own data, we’ll also come to the point where the important data will self identify. Once we know what data is useless, and a lot of it will be, the machines will analyze the relevant information and we can use the results to creatively solve problems in an order of magnitude faster than in any other point in history. The Big Data Economy: Our Future With Data 33
  • 34. 4 We Will See Interdisciplinary Data Layering Creating Value We’ll also be able to tell if more data that has already been deemed valuable needs to be layered onto another set to create value. As we begin to approach large data sets, we must not allow ourselves to make presumptions when first analyzing the data, and we must accept that data isn’t telling us anything when it isn’t. If we find that data doesn’t tell us something, we can toss it out or decide to layer more data onto it, and move on to finding whatever value the layering might offer us. The Big Data Economy: Our Future With Data 34
  • 35. Looking Forward to Life with Big Data The magnitude of data that we’ve come across is almost so dazzling that we don’t know what direction to move in. While we might have been out of the cave and into the light for quite some time, the light of reality just got much brighter. As we remember the history of how thought carried us to where we are now, we have to consider the fact that trail blazers are who moved us forward into a world that’s closer to the clearest picture of reality. As we move forward, we’ll look at the tools that we currently have to handle the magnitude of data that exists, which are best, which might function too close to the old order, and where they will take us in the Data Economy. Click here to see data analytics tools in action! Share this eBook: The Big Data Economy: Our Future With Data 35