Aucune remarque pour cette diapositive
Much of that data is in traditional databases and data warehouses, and those kinds of data – product orders, new customer records, etc. – grow at a more linear rate. What’s driving the exponential growth is often less structured data. This is in the form of log files, images, video, sensor or device output, and public data. Much of this data never makes it into a relational database, and the technologies used to process this kind of data go by the names “Data Intensive Scalable Computing”, “Large Scale Data Analytics”, or the more trendy term “Big Data”.In the past, much of this kind of data was simply archived, or even thrown out after a relatively short time. But now, it’s economically feasible to not only store, but process and gain insight from this kind of data.Increasing Data VolumesAccording to Gartner, the current annual growth of WW information volume is 59% and continues to rise. This data explosion is being driven by the full range of traditional and non-traditional sources like sensors, devices, bots and crawlers. According to an IDC report, the volume of digital records is forecasted to hit 1.2M Zetabytes (1021 bytes) this year – and predicted to grow 44x over the next decade. Increasing Data ComplexityHistorically, the data has been largely structured in type; however the real growth is coming from non-structured data. The success of search engine providers and e-retailers who unlocked the value of click-stream data has debunked the myth that 80% of unstructured data has no value. The requirement to analyze and mine unstructured and structured data together is increasingly on the agenda for many enterprises today. Increasing Analysis ComplexityIncreasing analysis complexity comes hand in hand with Increased Data Complexity. For example, image processing for facial recognition, search engine classification of videos and use of click-stream data for behavioral analytics. Models for transactional data are mature and well understood and have driven the value behind the last two decades of Data Warehousing and BI. The models governing complex data and behavioral interactions are in their infancy. Increasing Demand for New InsightsDespite the growth in useful information, we also know that the number of users in an organization who have access to Business Intelligence tools and capabilities is less than 20%. This fact combined with the real time nature of data is given rise to demand for real-time and predictive analytics by an increasingly larger user population. Changing EconomicsCloud computing and commodity hardware have radically reduced the acquisition cost of computational and storage capacity. The decreasing cost of distributed compute, memory and storage is fundamentally changing the economics of data processing. The rise of the Data Warehouse appliance has more than halved the cost per terabyte of EDW systems over the past 3yrs. Cloud Data Warehouse systems hold the promise of a 10x TCO advantage over traditional on-premises systems. Emerging TechnologiesEasy to scale commodity hardware is being complimented by new distributed parallel processing frameworks and tools, which combined are providing a rich and inexpensive platform for tackling massive data processing tasks. MapReduce style programming models are enabling new types of analytics that were not practical or possible with SQL. The maturity and commercialization of several open source software products has paved the way for their inclusion in product evaluations for larger scale software projects. The cloud model puts another layer of abstraction between the user and the infrastructure and application platform layer further reducing barriers to adoption of technologies like Hadoop.