Insight from Agora Consultants

Being deceived by randomness

An important part of getting value out of a Business Intelligence system is interpreting results correctly. We can often be fooled into attributing cause to random results. Consider the following graph of sales over time. It looks like something caused Sales to drop for three months. Management may scurry around trying to find out what happened during those months. Inevitably some event that happened around the same time will be picked as the cause and blame will be assigned. Also note that the opposite happens too. Sales could be way up for three months and there would be a line of people willing to take credit. Any set of time series data is bound to have a string of events that looks like order – typically called a ‘streak’. Examples are everywhere from home run streaks, string of loses of a hockey team and investment performance results of a mutual fund manager. Certainly skill plays into the results but elements outside our control introduces randomness into the results. Stock markets are affected by many elements such as world events and thus are chaotic and very unpredictable. This introduces a significant random element to a mutual fund manager’s performance. There are other examples outside of time series data. Implied order from random events can be used to a marketers advantage. Consider an unscrupulous investment newsletter trying to obtain subscribers through direct mail. They pick four stocks and predict whether the price will be up or down at the end of the next 6 months. There are a total of sixteen possible outcomes in this scenario. The investment newsletter prepares 16 different mailings, each with predictions of the 16 different outcomes from the scenario. One of them will have correct predictions for all four stocks at the end of the six months. If 10,000 people are sent the mailings,  that means 625 people will receive a mailing that correctly predicts the stock direction of all four stocks. This would a a seemingly miraculous feat that motivates those 625 people to sign up for the newsletter. People seek to find reason in randomness. We are tricked because random data will have a sample set that looks like order. When collecting and interpreting data be careful about trying to assign conclusions to data that otherwise may be random. Specific considerations include: Use result metrics as a signal to investigate further cause but be sure to consider “random event” and one of the potential causes Use metrics that lead to results in addition to the results themselves. For instance, number of sales calls leads to revenue results If you have enough data, use statistical techniques to demonstrate that an event is statistically significant For further discussion on randomness see the book “The Drunkard’s Walk: How Randomness Rules Our Lives”.

Objective: Deliver Projects on Time

The last objective I will look at as part of the series supporting Metrics for Successful Project Delivery is “Deliver Projects on Time”. Along with budget it is the most watched metric for projects. The key point to recognize is different people in an organization have different ways to contributing to this objective. Although all roles are supporting the same objective, each role requires different metrics to support their individual job contribution to the objective. As an example, the Senior Management Team need projects to be delivered in order to support their business strategy. The aggregate view of how well projects are delivering to timelines provides a view of the progress of the implementation of their strategy. A team member has direct impact on the timeliness of project delivery by delivering their individual tasks on time. If a team member starts falling behind on tasks its a good indication that the overall project will be late. The following table provides a summary of the roles and metrics used to contribute to the objective of delivering projects on time. Role Context Sample Metric Sample Target Senior Management Team The SMT needs visibility of the timing of projects supporting their strategy. Number of projects forecast to be late. 0 Business Project Sponsor Needs to update the project schedule to communicate to the rest of the business. Forecasted end date - baseline end date 0 PMO Director Being aware of how the organization is performing permits the Director to focus resources to problems. Number of projects forecasted to be late. 0 Project Manager Responsible for the project state. Forecasted end date – Baseline end date 0 Team Member The project schedule is dependent on its composite tasks. A team member has impact on the tasks assigned to them and thus the overall schedule. Number of late tasks. Remaining work on started tasks. 0 I’ve selected three sample objectives and metrics for the past blog posts. These examples show how to align your organization around the same objectives while maximizing their individual contribution by having metrics targeted to their role.

Objective: Maintain high project data quality

In my previous post, Metrics for Successful Project Delivery, I mentioned that I am kicking off a series of posts that provide objectives and supporting metrics for  managing the delivery of projects. The objective “maintain high project data quality” is a key objective to implement first. Without it, you cannot depend on any of the other metrics since people may not trust the data. Data quality is affected by a number of factors. Time sheets are updated by team members, schedules are maintained by project managers, risks and issues may be updated by the whole team. This information is utilized to understand the health of a project. It needs to be entered accurately and in a timely fashion in order to have high data quality. This information is created and maintained by people. Simply by monitoring the quality of the data you can encourage (or discourage) behaviours such as entering time sheets on time. The table below shows examples of metrics that drive project quality and the meaning of them based on people`s roles. Role Context Metrics PMO Director Stakeholders will only trust reports about the projects if they trust the data. Number of project plans overdue to be published Number of projects with hours scheduled in the past Number of project with overdue timesheets Project Manager The project manager produces much of the data that drives other metrics Number of hours overdue for publishing the project plan Number of unscheduled hours Number of resources late on time submission Team Member Teams members are “end points” of measurement for tasks within projects. Number of days late on providing schedule updates. The final step in implementing the objective is to create a key performance indicator from the metric. This involves taking the actual value and comparing it to the target value. The table people shows an example for one of the metrics. Number of hours overdue for publishing the project plan (target) Indicator 0 Green < 4 Yellow > 4 Red Implementing the objective “maintain high project data quality” is the foundation for ensuring all other objectives for delivering projects have strong supporting metrics.

Metrics for Successful Project Delivery

With the start of the New Year I’ll be kicking off a series of related posts. Through January, the Business Intelligence posts will be focused on going deep with the key metrics required for delivering projects. As a start, you can read the web article on Performance Management for the PMO or you can read the full whitepaper. The difference between these posts and the content in the article is that for each metric discussed, I will provide some guidance on setting the targets for your organization. The strategy map below provides the model which I will follow. I’ll take some of the key objectives from the strategy map and outline the metrics different roles in an organization need to support the objectives.   Why bothering measuring? By having metrics in place your organization will have greater transparency and understanding of how well resources (financial and people) are being utilized in your organization. Through this understanding better decisions can be made leading to increased company value.

Social commerce: the next evolution of eCommerce

Online presence has been a requirement for corporate marketing for many years. Most companies have moved from brochureware to eCommerce. The next evolution of Web sites has begun and it is the move to social commerce. With social commerce, consumers have an unprecedented forum for communicating with each other and corporate organizations about their purchases and opinions about corporate organizations. Agora’s Craig McQueen has published an article about social commerce in Marketing Canada magazine.  Below is a link to see a complete copy of Craig’s article.

Using Microsoft SQL Server 2008® as a Business Intelligence Platform

Microsoft recently released SQL Server 2008.  There are many updates and changes to this very popular database platform.  Our Business Intelligence lead, Craig McQueen, has written a very informative article about using the new features of SQL Server 2008 to build a robust BI platform for any organization. You can access the detailed article here: