In the June 2, 2014 issue, Database Trends and Applications magazine introduces the second annual “DBTA 100” list of companies that matter most in data. Data management challenges and opportunities have intensified over the past year. Big data continues to impact organizations as they seek to extract value from the large quantities of data flowing in from traditional and newer sources social media sites, blogs, sensors, meters, and transactional systems, to name just a few. Separately, although related, massive data growth has continued to put pressure on IT resources and budgets.
The confluence of trends affecting data growth and management was underscored by a recent study of 322 data managers and professionals conducted by Unisphere Research, a division of Information Today, Inc. The “Achieving Enterprise Data Performance: 2013 IOUG Database Growth Survey,” sponsored by Oracle, found that for many organizations data growth is in double digit ranges. In the survey, 41% of respondents reported a growth rate of more than 25% a year, and 17% of respondents reported a whopping growth rate of more than 50%.
There is the promise of big data—with its associated volume, variety, and velocity—but also just gross data growth —with data being duplicated for use by analytics platforms and tools, and for business protection backup and recovery, among other reasons. According to the survey, data is duplicated three or more times, providing a multiplying effect to data growth. Further exacerbating the situation, many companies retain data well beyond the 7-year legal requirement to meet compliance requirements, or in case of litigation.
While all this data is often described as a treasure trove, the reality is that there are challenges associated with efficiently and cost-effectively capturing, mining, storing, and securing it, while achieving necessary performance levels, and adhering to regulatory and internal company mandates. Increases in data variety, concerns about database performance, and the need to control data management costs were some of the main concerns expressed by survey respondents as a result of this data growth.
The sign post for many companies on the road to becoming a data-driven enterprise is “Under Construction.” Understanding the potential value as well as the hurdles presented by the proliferation of structured and unstructured information, IT companies are seeking to provide effective management, integration, and analysis techniques that will support the rapid delivery of information when it is needed, and in the way users want to consume it. Innovative open source and proprietary approaches are coming from well established vendors and newer companies. NewSQL, and NoSQL (Not Only SQL) databases—including Key-Value, Graph, Column, and Document—as well as Hardtop distributions and related projects that provide enterprise features—are gaining ground. MultiValue technology(sometimes called the fifth NoSQL database) products as well are expanding with new integration sand capabilities. And, addressing the trend toward BYOD (bring your own device), organizations are seeking to provide greater security and options for multiplatform development. In addition, cloud approaches are increasingly providing the agility and efficiency demanded by the enterprise. Read on...