Integration of Big Data & Business Intelligence

With the emergence & evolution of new technologies, current canvas of data collection and analysis is revolutionizing rapidly. The enormous data collected, data sources, storage and analysis of data all are major concern areas for any organisation. Big Data is changing the way business decisions are made as it exceeds the capacity and capability of conventional storage, reporting and analytics systems.

What is required is a fresh and new attitude towards problem solving. Big Data technologies will shrink the gap between the information available with the company and the data available for specific users when they need. But not just the technology, we also require skills and processes to utilise the technology. Here comes Business Intelligence (BI) into the account.

Business Intelligence represents tools and techniques used to transform data into meaningful information for business analysis. The BI skills will complement the new sources of data and help in planning the integration of data and data sources. Once the systems are integrated for Big Data and BI, the better rate of ROI can be achieved. Both are complementary: Big Data provides deep, exploratory outlook on the data, whereas BI systems help in structured result oriented information.

The analytical tools with data discovery features will help business users to explore data and blend data insights into the decision making process and reduce the time and complexity involved. The software developers are focussing on developing algorithms that automate the daily routine activities. For example, software that segregates the different e-mails user receives on basis of urgency of work required will reduce the employee workload and simplify the processes.

The data of the customer buying behaviour, stock levels, security and fraud control, incentive and referral programs, customer satisfaction levels all require combining Big Data and BI techniques. The companies should tap both BI and Big Data technologies for greater and better understanding and analysis of the incidents and business patterns.
Let’s consider a scenario wherein two different customers buy the same product say a Laptop.

Case 1: The customer watches an advertisement of a laptop manufacturing company, visits the website to know more about the product, purchases it from a local store and gets in touch with service personnel for software installations.

Case 2: The customer urgently needs a new laptop, purchases it from an ecommerce site at a discounted price, views the company advertisement on new laptop, gets the collateral software installed a week after from website and five months later visits the service personnel for heating issues.

These two scenarios put across examples of different consumer behaviour, which can be analysed and put to use for various marketing initiative. The first customer follows the precise course of buying behaviour whereas the other customer is spontaneous and constantly breaks the standard rules. When data is segmented and further fragmented it helps monitor consumer behaviour and derive patterns.

Moreover, Big Data & BI services are being offered as combined service by IT companies. The BI services include: pooling and manipulation of unstructured data, big data architecture implementation, data integration from all sources of information, and advanced analysis (data visualisation, predictive analysis, graph analysis, geographical information system). Big Data does not rule out the need for BI basics like data governance, data management, data modelling and much more. In other words, it makes such tasks more challenging.

Today it’s not enough to collect the accurate and timely data, professionals want powerful and user friendly BI tools to understand context and trends. One needs to focus on evolving BI approach in line with big data analysis to gain a competitive advantage. As can be seen from above Business Intelligence and Big Data are both about enabling enterprises to make more data driven business decisions than previously.

Some Big Data tools to integrate it with BI:

Pentaho Business Analytics: It is well known suite which includes data integrator and powerful visualization reporting tool. Pentaho provides graphical drag-and-drop components for Hadoop ecosystem projects such as Sqoop and Oozie, drastically reducing the amount of time needed to use these powerful bulk-data load and workflow utilities.

Jaspersoft: Easy to use, drag and drop report designer lets you build interactive crosstabs, tables and chart based reports using a standard browser. With Hadoop integration you can easily fulfil periodic or ad-hoc reporting needs.

Tableau: It provide direct connections to most popular Hadoop distribution like Cloudera, Hortonworks, IBM InfoSphere BigInsights, MapR, Pivotal.

Deepesh Tiwari is a System Analyst and Big Data & BI Expert @ PSI
Image Courtesy: www.datacar.comWith the emergence & evolution of new technologies, current canvas of data collection and analysis is revolutionizing rapidly. The enormous data collected, data sources, storage and analysis of data all are major concern areas for any organisation. Big Data is changing the way business decisions are made as it exceeds the capacity and capability of conventional storage, reporting and analytics systems.
What is required is a fresh and new attitude towards problem solving. Big Data technologies will shrink the gap between the information available with the company and the data available for specific users when they need. But not just the technology, we also require skills and processes to utilise the technology. Here comes Business Intelligence (BI) into the account.
Business Intelligence represents tools and techniques used to transform data into meaningful information for business analysis. The BI skills will complement the new sources of data and help in planning the integration of data and data sources. Once the systems are integrated for Big Data and BI, the better rate of ROI can be achieved. Both are complementary: Big Data provides deep, exploratory outlook on the data, whereas BI systems help in structured result oriented information.
The analytical tools with data discovery features will help business users to explore data and blend data insights into the decision making process and reduce the time and complexity involved. The software developers are focussing on developing algorithms that automate the daily routine activities. For example, software that segregates the different e-mails user receives on basis of urgency of work required will reduce the employee workload and simplify the processes.
The data of the customer buying behaviour, stock levels, security and fraud control, incentive and referral programs, customer satisfaction levels all require combining Big Data and BI techniques. The companies should tap both BI and Big Data technologies for greater and better understanding and analysis of the incidents and business patterns.
Let’s consider a scenario wherein two different customers buy the same product say a Laptop.
Case 1: The customer watches an advertisement of a laptop manufacturing company, visits the website to know more about the product, purchases it from a local store and gets in touch with service personnel for software installations.
Case 2: The customer urgently needs a new laptop, purchases it from an ecommerce site at a discounted price, views the company advertisement on new laptop, gets the collateral software installed a week after from website and five months later visits the service personnel for heating issues.
These two scenarios put across examples of different consumer behaviour, which can be analysed and put to use for various marketing initiative. The first customer follows the precise course of buying behaviour whereas the other customer is spontaneous and constantly breaks the standard rules. When data is segmented and further fragmented it helps monitor consumer behaviour and derive patterns.
Moreover, Big Data & BI services are being offered as combined service by IT companies. The BI services include: pooling and manipulation of unstructured data, big data architecture implementation, data integration from all sources of information, and advanced analysis (data visualisation, predictive analysis, graph analysis, geographical information system). Big Data does not rule out the need for BI basics like data governance, data management, data modelling and much more. In other words, it makes such tasks more challenging.
Today it’s not enough to collect the accurate and timely data, professionals want powerful and user friendly BI tools to understand context and trends. One needs to focus on evolving BI approach in line with big data analysis to gain a competitive advantage. As can be seen from above Business Intelligence and Big Data are both about enabling enterprises to make more data driven business decisions than previously.
Some Big Data tools to integrate it with BI:

Pentaho Business Analytics: It is well known suite which includes data integrator and powerful visualization reporting tool. Pentaho provides graphical drag-and-drop components for Hadoop ecosystem projects such as Sqoop and Oozie, drastically reducing the amount of time needed to use these powerful bulk-data load and workflow utilities.

Jaspersoft: Easy to use, drag and drop report designer lets you build interactive crosstabs, tables and chart based reports using a standard browser. With Hadoop integration you can easily fulfil periodic or ad-hoc reporting needs.