Big Data

Definition

Basic kinds

• Descriptive analytics: It provides information on historical data, helping to understand past patterns and trends.
• Predictive analytics: It enables forecasting of future events based on analysis of historical data and machine learning algorithms.
• Prescriptive analytics: It identifies optimal solutions based on predictions and recommendations, suggesting the best actions.
• Diagnostic analytics: It helps to understand the causes of certain events by analysing correlations and relationships between different variables.

Main roles

• Big Data has applications in many areas, including:
• Banking – e.g. fraud alerting, enterprise credit risk reporting, social analytics for commerce.
• Communications, media and entertainment – e.g. collecting, analysing and using consumer insights, leveraging mobile and social media content, creating media content consumption patterns.
• Healthcare sector – e.g. improving service delivery and customer service, reducing healthcare costs.
• Education – e.g. measuring the effectiveness of progression and development.
• Manufacturing industry and natural resource management – e.g. reducing costs, increasing efficiency, increasing sales, increasing speed of innovation, more effective research and development.
• Insurance – e.g. tailoring products to customer needs, analysing and predicting customer behaviour.
• Retail and wholesale – e.g. control of customer loyalty, inventory, insight into local demographic data collected by retail and wholesale shops.
• Transport – e.g. traffic control, route planning, intelligent transport systems, traffic management.
• Energy and utilities sector – e.g. utility consumption analysis, better asset and employee management.

Basic elements

• NoSQL databases: Systems for storing and managing unstructured data, such as MongoDB and Cassandra.
• Analytics platforms: Data analytics tools, such as Apache Hadoop, Spark, and Kafka, that process large volumes of data in real time.
• Machine learning algorithms: Analytical models that use data to make predictions and recommendations based on patterns.
• Data visualisation tools: Applications, such as Tableau and Power BI, that facilitate the presentation of data analysis results in graphical form.
• Cloud infrastructure: Environments, such as AWS, Google Cloud and Microsoft Azure, for scalable data storage and processing.

Mechanism of action

• Data collection: The data comes from a variety of sources, such as IoT devices, social media, online transactions, server logs, industrial sensors, and ERP systems. Data collection covers both structured data (e.g. databases) and unstructured data (e.g. texts, images).
• Data storage: The collected data is stored in large databases, often in the cloud, to ensure scalability and availability. NoSQL databases are popular for managing large volumes of unstructured data.
• Data processing: The data is processed to prepare it for analysis. Tools such as Hadoop and Spark enable distributed data processing, which reduces the time needed to analyse large data sets.
• Data analysis: The processed data is analysed using advanced algorithms, such as machine learning algorithms, to help discover patterns, trends, and relationships. The analysis can be done in real time or in batch, depending on the needs.
• Data visualisation: The results of the analysis are presented in the form of visualisations, such as charts, dashboards, and heat maps, which facilitate data interpretation and decision-making.
• Decision-making: Based on the results of the analysis, organisations can make better reasoned decisions about production, marketing, risk management, and other areas of the business.
• Optimisation and updating: The results of the analysis can be used to continuously optimise processes, products, and services. Machine learning algorithms are regularly updated to adapt to changing patterns and conditions.