Data Analytics and Business Intelligence

Data Analytics and Business Intelligence

One of the most powerful capabilities digital technology offers organisations is the ability to turn raw data into actionable insight. This lecture traces the journey from raw data to business intelligence, covering the concepts, tools, and practices that enable data-driven decision making. In an increasingly competitive environment, organisations that can effectively collect, manage, and analyse data gain significant strategic advantages.

From Data to Information to Knowledge

There is an important conceptual hierarchy to understand before exploring data tools:

• Data — raw, unprocessed facts and figures without context (e.g. the number 150, or the string "Melbourne"). Data alone tells us nothing.
• Information — data organised with context, structure, and meaning that makes it useful (e.g. 150 units of Product X sold in Melbourne in April 2024). Transformation of data into information requires organisation, structuring, and the addition of context.
• Knowledge — the understanding gained from analysing information to reveal patterns, trends, and relationships that can guide decisions (e.g. Melbourne sales of Product X peak in autumn — increase stock by 20% from March onwards).

Effective information systems manage this hierarchy: they collect data reliably, transform it into meaningful information, and support the analysis that generates actionable knowledge.

Data Quality

The quality of insights from any analytical system depends entirely on the quality of the underlying data. Poor data quality leads to poor decisions — a phenomenon sometimes summarised as garbage in, garbage out (GIGO). Key dimensions of data quality include:

• Accuracy — does the data correctly represent the real-world entity or event it describes?
• Completeness — are all required data elements present, or are values missing?
• Validity — do entered values make logical sense within defined rules and constraints? (e.g. a birth date of 32/13/2024 is invalid)
• Consistency — is the same data represented the same way across all systems? (e.g. the same customer not recorded differently in different databases)
• Timeliness — is data current and up-to-date with respect to business requirements? Stale data can be as misleading as inaccurate data
• Uniqueness — does the same entity (customer, product, supplier) appear only once in the data, without duplicates?

Databases and Relational Databases

A database is an organised collection of data, managed by a Database Management System (DBMS) that controls access, ensures data integrity, and handles concurrent users. Relational databases organise data in tables with defined relationships between them, using Structured Query Language (SQL) to retrieve and manipulate data.

The key benefits of relational databases over separate departmental data stores or flat files include:

• Data redundancy is minimised — the same data is not stored in multiple places
• Data inconsistency is reduced — a single update propagates consistently across related records
• Data can be shared across departments and applications through a common, well-managed repository
• Access can be controlled — different users can be given appropriate levels of access to different parts of the data
• Data integrity can be enforced — rules (constraints) prevent invalid data from being entered

Data Silos and Their Problems

A data silo occurs when data is trapped in a single system or department and is not accessible to the rest of the organisation. Data silos cause:

• Data inconsistency — the same information (e.g. a customer's contact details) may differ between systems
• Duplication — the same data is maintained in multiple places, creating overhead and error risk
• Poor decision-making — managers making decisions without access to complete, integrated information
• Inefficiency — manual processes required to combine data from different sources for reporting

ERP systems and enterprise data warehouses are key tools for breaking down data silos.

Data Warehouses and Data Marts

A data warehouse is a large, centralised repository of integrated historical data from multiple operational systems, designed specifically for analytical reporting and decision support — not for day-to-day transaction processing. Data warehouses:

• Consolidate data from multiple source systems (ERP, CRM, financial systems, etc.) into a single integrated repository
• Store historical data over long time periods, enabling trend analysis
• Optimise data structures for fast querying and reporting rather than transaction speed
• Support business intelligence tools and dashboards

A data mart is a smaller, more focused subset of a data warehouse serving a particular business area or department (e.g. a sales data mart, a finance data mart). Data marts are easier and faster to implement than full data warehouses.

Business Intelligence (BI)

Business Intelligence (BI) refers to the technologies, processes, and practices used to collect, integrate, analyse, and present business information to support better decision-making. BI tools include:

• Reporting tools — generate standard and ad-hoc reports from data warehouses
• Dashboards — visual displays of key performance indicators (KPIs) updated in real time or near-real time
• Online Analytical Processing (OLAP) — multidimensional data analysis tools that allow users to explore data across multiple dimensions (time, geography, product, etc.)

BI primarily focuses on historical data — describing what has happened.

Business Analytics

Business Analytics (BA) goes further than BI. It uses statistical analysis, data mining, and predictive modelling to forecast future outcomes and optimise decisions. Key types include:

• Descriptive analytics — what happened? (BI territory)
• Diagnostic analytics — why did it happen? (root cause analysis)
• Predictive analytics — what is likely to happen? (statistical modelling and forecasting)
• Prescriptive analytics — what should we do? (optimisation and decision support)

Together, BI and BA give organisations the capacity to move from reactive to proactive management — anticipating challenges and opportunities rather than simply responding to them.