Big Data Solution "Secret" Sauce

Better, faster, cheaper — and now simplified, automated, visualized, interactive and self-servicing.
Such is the mantra of successful big data solutions — or so says research published by Pentaho and summarized below by Caron Carlson of FierceCIO.
It makes me wonder, however: Was there ever any doubt?
Emphasis in red added by me.
Brian Wood, VP Marketing

Survey: Big data solutions need to be simpler, more interactive and more automated

New types–and growing volumes–of data are presenting significant information management problems, and to add to the difficulties, the highly coveted data scientists out there want direct access to big data sources. The good news is that there are companies finding value in big data applications once they’ve gotten past these hurdles–this according to a newly released survey by Pentaho Corp.
Nearly half of the IT pros and business analysts (45 percent, to be exact) surveyed by Pentaho said resource availability was a top challenge in deploying big data.
Skills and the complexity of big data were cited as top challenges by 43 percent and 38 percent of the participants, respectively.
The study concludes that big data solutions need to be simpler, more automated, more interactive, and they also need to include self-service functionality.
Nearly half of the survey participants said they use standard relational databases in addition to NoSQL or Hadoop. The combination of legacy databases and new big data sources has prompted a need for reconciling existing storage infrastructure with new data stores.
The most popular big data application, according to the study, is customer transaction analysis, and application log analysis came in second.

Pentaho Survey Finds Technology, Data Complexity Hinder Big Data Adoption

With 50 percent of respondents in production or planning big data projects, survey identifies emerging applications and use cases in response to business needs
Orlando, FL — February 21, 2013 —  Delivering the future of business analytics, Pentaho Corporation today announced results of a survey that found nearly 50 percent of respondents are either in production with a big data project or planning to deploy one in the next 18 months.  The poll also revealed three key trends with big data deployments, according to Pentaho:

  • Big data talent crisis is exacerbated by overly complex Hadoop technologies combined with the expectations of data analysts for access to and self-service for big data sources.
  • Data diversity prevails and new types and volumes of data present considerable data management problems.
  • Revenue and customer big data applications demonstrate immense value to companies on the front lines with big data who are solving real problems and uncovering new business opportunities.

Detailed findings from the survey include:
Big Data Talent Crisis –The primary consumers of big data are either business analysts (55%) or data analysts (42%) with some serving both roles, which demands an understanding of analytics and a deeper technical acumen. Only 20 percent of respondents identify themselves as data scientists.
The survey also pinpointed the top three challenges in deploying big data as: resource availability (45%), skills (43%), and complexity of big data (38%). Clearly, respondents are struggling with programming-intensive data analytics technologies and complex data sources. Continued improvement in the simplification, automation, visualization, interactivity and self-service functionality of big data solutions are needed.
Data Diversity Rules – With multiple data sources the norm, over 50% of survey respondents cited both managing data and manipulating data as big bottlenecks.  Furthermore, more than 60% of respondents report that they are managing three or more data sources (32% with five or more).  Almost half (47%) say their organizations are using standard relational databases as well as Hadoop or NoSQL.

With the growing prevalence of data environments that include legacy relational databases and big data sources, organization must be able to quickly and cost-effectively reconcile existing database and data warehousing infrastructure with their big data stores,” said Richard Daley, founder and chief strategy officer at Pentaho. “An effective, holistic big data analytics platform such as Pentaho’s includes not only analytics, but also data integration, data quality and data manipulation capabilities.”

First Big Data Winners: Revenue & Customer Applications – Emerging big data applications and use cases can reveal highly vital insights. According to the survey, customer transaction analysis (43%) is the most popular application in use today, followed by application log analysis (37%) and e-commerce transactions analysis (20%).
Among the early market adopters, half are visualizing their big data via reports/dashboards and interactive analysis. Finally, regardless of application type, the majority of the respondents said they are deploying on-premise, with only one-third using the cloud.

Conducted by Pentaho, the survey polled approximately 270 respondents on big data-related questions ranging from deployment stage, infrastructure and software type, to data sources, target users, problem areas and early application uses. More than half of the survey participants were developers, ETL developers, or BI developers; while around one third were business analysts.
Pentaho Business Analytics for big data enables companies to realize business value from large volumes of diverse data by dramatically reducing the time and complexity required to design, develop and deploy a big data analytics solution.  Pentaho’s solution covers the entire big data lifecycle, from data extraction and preparation to data discovery and predictive analytics.

About Pentaho Corporation

Pentaho is delivering the future of business analytics. Pentaho’s open source heritage drives our continued innovation in a modern, integrated, embeddable platform built for the future of analytics, including diverse and big data requirements. Powerful business analytics are made easy with Pentaho’s cost-effective suite for data access, visualization, integration, analysis and mining. For a free evaluation, download Pentaho Business Analytics at

Would love your thoughts, please comment.x