A business intelligence architecture is the framework for the various technologies an organization deploys to run business intelligence and analytics applications. It includes the IT systems and software tools that are used to collect, integrate, store and analyze BI data and then present information on business operations and trends to corporate executives and other business users.
The underlying BI architecture is a key element in the implementation of a successful business intelligence program that uses data analysis and reporting to help an organization track business performance, optimize business processes, identify new revenue opportunities, improve strategic planning and make more informed decisions overall.
Importance of a BI architecture
A business intelligence architecture articulates the technology standards and data management and analytics practices that support an organization's BI efforts, as well as the specific platforms and tools that will be deployed. It serves as a technology blueprint for collecting, organizing and managing BI data and then making the data available for analysis, data visualization and reporting. A strong BI architecture also incorporates policies to govern the use of the technology components.
Putting such a framework in place enables a BI team to work in a coordinated and disciplined way to build an enterprise BI program that meets its organization's data analytics needs. The BI architecture also helps BI and data managers create an efficient process for handling and managing the data that's pulled into the environment.
Enterprises benefit from an effective BI architecture by using the insights generated by business intelligence tools to make data-driven decisions that help increase revenue and profits. To ensure their needs are met, C-suite executives, business managers and other business users who rely on data analysis to formulate strategies and guide their decision-making should have a stake in creating the architecture.
Business intelligence architecture components and diagram
A BI architecture can be deployed in an on-premises data center or the cloud. In either case, it contains a set of core components that collectively support the different stages of the BI process, from data collection, integration, storage and analysis to data visualization, information delivery and the use of BI data in business decision-making.
An example of what's included in a BI architecture
As shown in the accompanying business intelligence architecture diagram, the core components include the following items.
Source systems. These are all of the systems that capture and hold the transactional and operational data identified as essential for the enterprise BI program -- for example, ERP, CRM, finance, manufacturing and supply chain management systems. They can also include secondary sources, such as market data and customer databases from outside information providers. As a result, both internal and external data sources are often incorporated into a BI architecture.
Important criteria in the data source selection process include data relevancy, data currency, data quality and the level of detail in the available data sets. In addition, a combination of structured, semistructured and unstructured data types may be required to meet the data analysis and decision-making needs of executives and other business users.
Data integration and cleansing tools. To effectively analyze the data collected for a BI program, an organization must integrate and consolidate different data sets to create unified views of them. The most widely used data integration technology for BI applications is extract, transform and load (ETL) software, which pulls data from source systems in batch processes. A variant of ETL is extract, load and transform (ELT), in which data is extracted and loaded as is and transformed later for specific BI uses. Other methods include real-time data integration, such as change data capture and streaming integration to support real-time analytics applications, and data virtualization, which combines data from different source systems virtually.
A BI architecture typically also includes data profiling and data cleansing tools that are used to identify and fix data quality issues. They help BI and data management teams provide clean and consistent data that's suitable for BI uses.
Analytics data stores: This encompasses the various repositories where BI data is stored and managed. The primary one is a data warehouse, which usually stores structured data in a relational, columnar or multidimensional database and makes it available for querying and analysis. An enterprise data warehouse can also be tied to smaller data marts set up for individual departments and business units with data that's specific to their BI needs.
In addition, BI architectures often include an operational data store that's an interim repository for data before it goes into a data warehouse; an ODS can also be used to run analytical queries against recent transaction data. Depending on the size of a BI environment, a data warehouse, data marts and an ODS can be deployed on a single database server or separate systems.
A data lake running on a Hadoop cluster or other big data platform can also be incorporated into a BI architecture as a repository for raw data of various types. The data can be analyzed in the data lake itself or filtered and loaded into a data warehouse for analysis. A well-planned architecture should specify which of the different data stores is best suited for particular BI uses.
BI and data visualization tools. The tools used to analyze data and present information to business users include a suite of technologies that can be built into a BI architecture -- for example, ad hoc query, data mining and online analytical processing, or OLAP, software. In addition, the growing adoption of self-service BI tools enables business analysts and managers to run queries themselves instead of relying on the members of a BI team to do that for them.
BI software also includes data visualization tools that can be used to create graphical representations of data, in the form of charts, graphs and other types of visualizations designed to illustrate trends, patterns and outlier elements in data sets.
Dashboards, portals and reports. These information delivery tools give business users visibility into the results of BI and analytics applications, with built-in data visualizations and, often, self-service capabilities to do additional data analysis. For example, BI dashboards and online portals can both be designed to provide real-time data access with configurable views and the ability to drill down into data. Reports tend to present data in a more static format.
Other components that increasingly are part of a business architecture include data preparation software used to structure and organize data for analysis and a metadata repository, a business glossary and a data catalog, which can all help users find relevant data and understand its lineage and meaning.
The key BI team roles and responsibilities?
It's no secret that organizations are trying to do more with their data -- by using it to drive more informed business decisions, gain competitive advantages and boost the bottom line. That's why there is strong demand for the business intelligence professionals who make up a BI team.
There are a variety of BI team roles and responsibilities, ranging from program managers or directors to BI architects, developers and analysts. They work collaboratively with an organization's IT and data management teams and business users to plan, implement and run business intelligence applications that are designed to deliver actionable information to executives and operational workers.
The BI team typically is responsible overall for doing data discovery, data validation and source system identification and connectivity; working with end users to define business requirements and needs; writing BI policies and procedures for management or steering committee approval; overseeing project implementations; and ensuring that the promised benefits of BI projects are achieved.
Building a BI team can be a big investment, but it's a key step for organizations looking to take advantage of "the abundance of information" they create and collect, said Justin Zhen, co-founder of Thinknum Alternative Data. In recent years, data-savvy businesses "have started learning about their users, customers and employees in a way that was not previously possible," added Zhen, whose company provides data sets from public sources for BI and analytics uses.
An effective BI team pulls useful business insights from data sets -- for example, a better understanding of customer needs and buying habits to help a company increase sales and create new revenue streams, said Carmen Lapham, technical recruiter and client manager at recruiting firm The Sourcery.
What is a BI team, and what does it do?
Initially, the business intelligence team often was part of the IT department. While that's still the case in many organizations, it's increasingly a separate unit that reports to the chief data officer instead of the CIO. An enterprise BI team is involved in data analytics initiatives across the organization. However, depending on a company's size and organizational culture, it may have multiple BI teams in departments and business units, typically supported by a centralized group.
The key functions of a BI team start with designing and deploying a BI architecture that incorporates source systems, data repositories, and a combination of BI, data visualization and reporting tools. Once that's in place, the team's ongoing functions include data collection, preparation, retrieval and analysis, plus communication of analytics results to business users.
"Someone has to get the underlying raw data," Zhen said. "Then, you need someone to clean and normalize the data. Then, you need an analyst who distills the data into information for a less technical audience. Lastly, you need someone to manage the whole process and present findings to management."
The shift toward self-service BI has changed the overall role of the BI team, though. Self-service BI tools enable business analysts, managers and other end users to analyze data and create data visualizations, dashboards and reports themselves. As a result, BI teams in organizations with self-service environments may do less hands-on analytics work and focus more on creating the BI infrastructure, training and supporting self-service users, and promoting BI best practices.
Key steps in the BI process
Individual BI team roles and their responsibilities
A business intelligence team includes a mix of project and program roles that cover both specific BI initiatives and the overall management of an organization's BI activities. In large organizations, each role may be handled by a separate person. In small and midsize ones with lower budgets and fewer resources, BI team members may have to fill multiple roles. In addition, different organizations may use different titles for positions that have similar responsibilities or some overlapping duties.
As a result, BI teams don't always include all of the positions listed below. But, based on interviews with recruiters and postings on the Robert Half, Indeed and Dice job sites, these are the top BI roles and an overview of their primary responsibilities.
BI manager
Also sometimes known as the director or vice president of BI, this person has overall responsibility for an organization's business intelligence program and the BI team. That includes overseeing the development of a BI strategy and supporting BI architecture; structuring, staffing and managing the BI team; and ensuring that BI projects are completed as planned.
Lead the process of deploying and using BI resources to deliver information to business users.
Partner with business stakeholders to understand their data needs, and act as an evangelist for the BI team.
Identify opportunities to maximize the adoption and use of BI tools.
Manage the BI project pipeline, and help to prioritize the work of project teams.
Direct efforts to measure the success of BI projects and document their ROI.
BI architect
This position leads the process of building a BI architecture, which typically pulls data from internal and external sources into a data warehousing environment and makes it available for analysis with BI tools. The BI architect typically is also responsible for designing individual BI systems and applications.
Oversee the overall architecture and design of BI and data warehouse systems.
Design and develop BI systems, applications and reporting platforms.
Act as the project lead on multiteam BI architecture and development projects.
Work with developers to create metadata and data models to support BI applications.
Create and maintain documentation on the BI architecture and systems.
BI systems administrator
This role monitors and manages BI systems, handling a variety of system maintenance, testing, performance tuning and security tasks.
Monitor system performance and resource utilization, and optimize systems as needed.
Troubleshoot system issues, and coordinate responses with IT and business managers.
Configure and manage user access controls, and apply security patches and updates.
Track and report on performance to ensure that service-level agreements are met.
Automate BI systems management processes via scripting and programming.
BI project manager
In some organizations, the manager of the BI program leads individual projects. Others have a separate BI project manager role that's responsible for managing project teams, budgets and schedules and ensuring that applications and tools are deployed successfully.
Work with business executives to define the scope of BI projects and criteria for success on them.
Develop project plans, and work to make sure deployment milestones are met.
Manage the work of other BI team members assigned to projects.
Communicate regularly with involved business leaders on the status and progress of projects.
Take the lead in resolving roadblocks, conflicts and other challenges on BI projects.
BI developer
A BI developer creates data visualizations and builds reports and dashboards as part of BI applications. Sometimes referred to as a BI engineer, this role also is often involved in BI architecture design and analytical data model development.
Work closely with business users to identify data analysis and reporting opportunities.
Develop and deploy business intelligence applications for BI and analytics projects.
Design dashboards and reports that deliver relevant business information to users.
Monitor recurring BI processes, and ensure the accuracy and timeliness of outputs.
Recommend and help implement improvements to BI, data modeling and data management processes.
BI analyst
Like a BI developer, the duties of a business intelligence analyst include performing data analysis and designing visualizations, dashboards and reports for business users. BI analyst is often a more junior role that requires a lower level of technical skills than a BI developer job does.
Work with business users to identify KPIs to track and relevant data sets to analyze.
Gather data requirements for BI projects, and translate them into functional specifications.
Use BI tools to run queries and generate visualizations, dashboards and reports.
Support users of self-service BI tools on data analysis and reporting.
Analyze data warehouse design, and recommend improvements to benefit BI efforts.
BI specialist and BI consultant are two other positions commonly found on business intelligence teams, often with similar duties and responsibilities to those of a BI analyst or, in some cases, a BI developer.
Related analytics and data management roles
In addition to the BI team roles themselves, other analytics professionals and data management workers are often involved in the BI process. Here are some of those positions and their key responsibilities.
Data engineer
A data engineer's job duties primarily involve providing and preparing data for use by the BI team and other analysts in an organization.
Work with BI and analytics teams to assess and support their data infrastructure needs.
Build data pipelines to make data accessible to BI analysts and data scientists.
Help deploy data warehouse and data lake systems for data extraction, integration and storage.
Cleanse and consolidate data sets to ensure that data quality levels are high.
Do data preparation work to filter and model data for individual analytics applications.
Data architect
A data architect designs database systems and other data repositories, as well as data models. The job can overlap with the BI architect role for analytics applications, but data architects also are involved in designing operational systems.
Architect data management systems and the way data is processed and stored in them.
Build an inventory of available data to help users find relevant information.
Create data models that reflect intended business uses for analytics and operations.
Ensure that data is easily accessible and scalable for use within an organization.
Evaluate data management technologies, and recommend purchases.
Data scientist
A data scientist is typically involved in advanced analytics applications, such as machine learning, predictive analytics and big data analytics. But data scientists sometimes also work in conjunction with BI teams on analytics projects.
Help business managers identify opportunities to use data to drive decisions.
Develop data models, and apply machine learning algorithms to analyze large data sets.
Analyze data to help plan product development, marketing campaigns and promotions.
Use predictive modeling to enable targeted marketing and an improved customer experience.
Gather and prepare data for analysis, sometimes in collaboration with data engineers.
Data analyst
The jobs of data analysts and data scientists overlap, but data analyst typically is a lower-level position that has more limited responsibilities and doesn't require all the advanced skills of a data scientist.
Work with business teams to understand their data analysis needs.
Develop ideas on how data should be analyzed and used.
Collect and analyze data for business uses.
Identify patterns and trends in data sets.
Report data analysis results to appropriate business executives and managers.
Business analyst
A business analyst generally works in a business unit or department and handles a variety of data-driven tasks to help improve IT systems and business processes.
Serve as the liaison between the technical and nontechnical teams in an organization.
Gather and communicate the requirements for data management and analytics projects.
Create a detailed analysis of data opportunities and challenges for the business.
Develop data quality metrics, and ensure that reporting needs are met.
Evaluate business processes, and recommend changes to improve them.
Key skills and traits of a top BI team
What skills and personal traits make for a successful BI team member?
"The ability to 'translate' between the IT organization and the business is easily one of the most valuable proficiencies and the hardest to find," said Patrick Leonard, a global analytics advisory and enablement leader at a large consumer packaged goods company, which he asked not be named. That includes being able to "understand the business, ask questions [and] dive into the data" to do analysis, while also passing user requests "back to the 'more technical' group to execute development," Leonard continued.
BI professionals must be effective communicators and problem solvers, said Allen Cunningham, principal at recruiting firm SoundSearch LLC. Other important traits for BI pros include intellectual curiosity, emotional maturity and the ability to flourish in a fast-paced and dynamic environment, he added.
"You have to be able to learn, present and wear many different hats at all times," Cunningham said. "It's a fluid environment. You want someone who excels within a constantly changing setting."
Above all, the members of a BI team need to be able to accurately interpret data and convey its potential decision-making uses to business executives and workers.
"Not all BI professionals need to learn how to code, but they need to at least understand basic data frameworks," Thinknum's Zhen said. "Communication is the next key factor. When analyzing complex insights, the skill in relaying specific information to someone else is critical."
Source: Whatls.com