Tue, Sep 30, 2008

 What Can BI do for Me?

Howard Dresner coined the term "business intelligence" (BI) back in 1989 as “concepts and methods to improve business decision making by using fact-based support systems”, whilst an analyst for Gartner Inc. (and spent the next 17 years acting as a godfather like figure influencing the direction of the technology and the market before leaving Gartner).

BI will therefore soon be 20 years of age, I myself have been involved since 1995, 13 years. Yet despite the apparent age of BI, it still remains somewhat of an unknown to the majority of people involved in both Business and Technology. The intention of this article is to try and dispel some of the common misunderstanding about BI and to describe its purpose, benefits and risks in common, non-techno babble terms.

BI is not reporting.

In general, reporting tools simply deliver information based upon data within a source system, providing very minimal analytical capabilities. They are primarily designed to execute simple to moderate queries against operational systems and present the resulting information in an attractive manner, with minimal impact upon that operational system. After all, the operational systems you are most interested in getting reports from, are the one involved in your revenue stream. As such, you don't want a reporting tool to be taking processing power away from your revenue stream.

A classic example of this is an ERP system, which is specifically targeted towards running the bread and butter systems of an enterprise and whose performance directly affects the bottom line. As such, ERP systems are tuned for operational performance, characterized by a high volume of simple inserts, updates and queries.

BI however, is characterized by complex analytical queries intended to shed light onto or even answer complex, multi faceted questions about your business. Executing such queries directly within an ERP system risks significant negative performance impact upon the operational aspects of the ERP system, potentially disrupting your revenue generating systems. ERP is not built for the mixed workload of both operational and analytical queries.

Much of this is evidenced by the recent consolidation in the BI market, where the largest independent BI vendors were each bought up by large "everything-but-the-kitchen-sink" vendors. Taking SAP as an example, arguably the world's largest ERP vendor: SAP realized the importance of BI several years ago and built their own BI toolset in the form of SAP NetWeaver BI (a.k.a. SAP BW).

Let it suffice to say, ERP is not BI and as such the large ERP vendors are expanding their product lines to offer a more comprehensive BI solution.

BI is not Excel

Another misconception about BI is that it can be achieved with minimal effort, on a broad scale and on an individual basis, through the use of Excel. I am not going to deny that excel is a powerful analytical tool or that excel does not have its place in a BI solution. Excel indeed does have its place within a BI solution, but it does not constitute a BI tool in and of itself. Excel is missing several key functional items and must rely upon other components within a BI infrastructure for:

o Integration with non Excel tools and other data management capabilities

o Anything beyond the simple functionality of pivot tables

o Unstructured data analysis

o Visual production/operational reports

o Ability to schedule reports to be run and distributed within the organization

o Easy creation and refinement of complex analytical queries

BI is not a "Nice-to-have"

In the early years, BI was indeed a "nice to have" but in the intervening years it has grown to become a fundamental imperative: If you don't have it, then you are losing ground to your competitors. Several recent surveys underscore the point:

o Nearly 70% of retailers have already implemented BI and another 26% plan to (Aberdeen Group, 2008)

o BI is the top technology priority of CIO's and has been for three consecutive years (Gartner, 2008)

o 9 out of 10 senior executives at Fortune 1000 companies place business intelligence at the top of their list in preparing them for their biggest challenge ahead (Accenture, 2005)

o 69% of mid market (less than $1B revenue) firms have or are deploying BI (WiseAnalytics, 2008)

(Only 4% of retailers surveyed do not have nor are planning to implement BI… I wonder what the company failure rate in the retail industry is?)

I will not claim that BI will help you make better decisions than your competitors, but it will help you make better decisions than you are currently making and it will help you do it faster and more efficiently.

BI is not a Data Warehouse or a Tool

Contrary to popular belief, just because you have something called a data warehouse, does not mean that you have BI. Indeed, there is still misunderstanding of what a data warehouse is. It does not take much searching through various on-line tech forums to encounter discussions about how to improve the BI performance only to realize that the discussion is focused on historical data rather than on analytics.

Even if you do have a data warehouse that is used for BI, in terms of either Bill Inmon (Corporate Information Factory) or Ralph Kimball (Enterprise Service Bus), it is only a single link in the value chain of BI.

Whilst there are several platforms on the market, both commercial and open source, that offer a comprehensive suite of BI components (ETL, reporting, OLAP, workflow, data mining and advanced analytics etc., and even pre-packaged in an appliance with the underlying data warehouse), but just because a single platform can give you all the components, does not mean it is a simple job of install, configure and 'hey presto!", you have BI. No, sorry, we are not talking e-mail servers.

BI cannot be described in the terms of a single tool; it is comprised of a collection of people, processes, technologies and culture. For example, one of the biggest and unfortunately most difficult tenets of BI is the "single version of the truth" mantra - a problem that grows more difficult as you add more people and more departments to the scope. As an example, just what is a "customer"? Do all departments (marketing, sales, shipping, customer support, finance, legal) within your organization use a single definition of customer? And that is before you even get to the problem of agreeing the correct name, address and contact information from the multiple sources within these departments).

If the BI tool or tools you implement are not backed up with the right people, the right processes and the right culture, your BI program will fail.

BI is not an IT-thing

Perhaps most importantly, BI is not or at least should not, be an IT or a technology thing: If you drive your BI program from the perspective of it being a technology or in terms of the data it provides, you not only face significant risks of poor user adoption, but even more so of complete failure and a costly one at that.

Any BI program should be driven from the perspective of augmenting and benefitting the business processes within your company and as such, needs to be a partnership. Tipping the balance too much in either direction is likely to lead to failure and dissatisfaction.

BI only reaches its full potential when it aligns and augments the underlying business processes and delivers relevant results, through the appropriate use of technology. If you miss the mark you will end up with, at best, good looking historical reports that nobody uses due to their lack of relevancy. At worst, an expensive mish mash of a system that is costly to maintain and does not provide consistent results.

Well, truth be told, in the worst case you end up with nothing except a big waste of money and a reduced headcount.

What is Business Intelligence?

Forrester defines BI as:

A set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making.

A good description, but one that I would tweak to transforming raw data into actionable information: There is no business value to information that you cannot act upon. Whilst this sentence defines BI, it does not describe BI, nor how it can benefit a business and it rarely garners more than a basic "oh, okay" response.

But what can business intelligence do for me?

BI helps in:

o Understanding how processes and transactions are performing in conjunction with goals, objectives, strategic direction, competitors and regulatory compliance.

o Uncover hidden bottlenecks in your processes that you previously did not know about.

o Aligning individuals and departments with organizational goals and strategy

o Drive operational performance, increase accountability via transparency in results

o Identify the factors across targeted segments that increased lift in marketing campaigns

o Forecast business outcomes based on past, present and potential scenarios

o And many others too numerous to list.

Basically, any DECISION within your enterprise that depends upon metrics and analysis from one or more streams of data, internal or external, whether history is relevant or irrelevant, BI can help.

Just as long as you do it right, BI provides you the approach to capturing and storing data that encapsulates (or impacts) your business and analyze it to extract relevant and actionable information.

o Approach

· The building blocks of BI are formed from many different things, they not only include specific tools and technologies, but also other less tangible items such as people process and culture. It benefits cultural changes away from gut feel and towards fact based management. Indeed, many modern management practices that advocate the use of metrics of various forms have their own vertical niche within the overall business intelligence market. Both SAP (Business Objects) and Oracle (Hyperion and Seibel Analytics) have packaged applications based upon their BI tools for Corporate Performance Management, Financial Management, Budgeting and Planning etc.

· Bi is very much a synergistic approach, as each building block adds more value to the whole than it can offer in isolation. Indeed, a successful BI program could almost be described as a 'reinforcing loop', where the more relevant and actionable information a BI system provides, the more business process the business wants to be included. I say "almost", since the realities of business, technology, finances, resources, time etc. creates a "limits to success" scenario.

o Capturing

· Much of the data that impacts your business is created within your operational system, but not all of it. There are many data streams external to the boundaries of your company that can add significant value to the data you have. Examples are:

- Demographic data, frequently purchased by marketing departments can also provide valuable information to your Sales and Customer Support departments.

- Weather and other natural events can be a cause for significant business disruption for you or your clients. Do you really want to send the next delivery out of your warehouse knowing that a major hurricane or earthquake has significantly impacted the client it is intended for?

- 3rd-party on-line forums can be a significant source of consumer sentiment and market opinions about your product and services. Howard forums (http://www.howardforums.com/) for example is the largest non-wireless carrier operator consumer forum in the USA and Canada. Seeing significantly higher volumes of forum traffic than the carrier forums themselves.

· However, you must ensure that the data you capture is of sufficient quality and accuracy in order to make quality decisions - business decisions made on the basis of incomplete or incorrect information can seriously affect the health of your company.

o Storing

· As you are capturing the multitude of data streams, you obviously need appropriate storage beyond the present location of the data: your operational systems do not store that much historical data, neither do many historical data feeds. But then again, you can't just slap it into any old database or file system.

· Obviously, this is where the various data warehousing methodologies such as the "corporate information factory" and the "enterprise service bus" come into play, alongside the multitude of technological implementation options.

· Oh, and just in case there are not enough storage questions already, there is a commonly publicized theme that the volume of data your business captures, continues to grow at an unprecedented rate.

- In 1995 the largest system in the world contained a terabyte of data. Ten years later, the biggest we found was over 100 times larger (Winter Corp, 2005)

o Analyzing

· When you have the right data, you can start asking the questions you were previously unable to do so:

- Is selling more and maximizing revenue a good idea?

- Is your best customer the one who brings in the most money?

- Which customers are costing us the most money and thus eroding the profit we make from other customers?

- Is there a particular product mix for a particular customer segment that increases costs or maximizes profit?

- What changes to our product mix should we make in order to make our services more profitable?

- What is the impact of poor customer service representatives on future sales?

o Relevant and Actionable

· As previously stated: Any information that is not relevant and actionable to your business processes is of limited to no business value. In fact, such information would be of negative value to your business process due to the expense of producing it.