Insight from Agora Consultants

Project Management Office (PMO) Effectiveness

I just returned from the Microsoft Project Conference in Anaheim that ran from February 2 through February 5, 2014. At the conference I attended a very interesting session that included Gartner analyst Donna Fitzgerald.  She gave a very provocative presentation about how the PMO needs to change in order to better provide value to their organizations.  Her rallying cry was “from Tactical to Strategic” stating that simply delivering projects on time and on budget was not sufficient now.  It was more important to ensure that all projects delivered value in terms of contributing to the strategic objectives of the organization. I couldn’t agree more.   And it reminded me of a great article written by one of our staff over three years ago that addresses very much the same topic.  In order to provide value to an organization, the PMO has to fully understand this difference and execute on it every day.  In our work delivering project management technology solutions, this distinction is top of mind in all implementations. It’s a joy to work with progressive organizations who embrace this concept. 

Does forecast status make sense?

  Within the project management industry we often see people include “forecast status” as an element of project status reports along with current status and previous status. The expectation is Project Managers will include a status of what they think the status will be for the next reporting period. For instance, the Project Manager would forecast if finances will be red, yellow or green. Status is defined as “state or condition of affairs”. It is a measurement of where things are right now. Previous status makes sense. Previous status is the condition of affairs measured at a previous point in time. Forecast status does not make sense. It is not possible to measure something in the future. The intent of forecast status is “what do you think the condition of affairs will be at the next reporting period”. In order to make this forecast, a project manager will (should) base it on something measured today. By reporting that measurement as the indicator of predicted status, it is a truer representation of the information. Instead of trying to measure into the future, measure something today that is a predictor of future status. Some people refer to this as a leading indicator. For instance, if it is financial status, the number of pending change requests may be an indicator that the finance status may change. Moving towards metrics that are based on current information rather than an opinion of the future eventually results in better decision making since the information is more accurate.

Get control of this valuable resource in your organization.

Management of resources is a high priority on people’s list. Organizations have a lot of focus on financial resources. Financial resources are usually easy to report on because they are captured in detail in financial systems. However, time is an equally if not more important resource for organizations. Time measurement is a leading indicator of financial results. If you see that people’s time is being spent in non-productive areas you will know that poor financial results will follow. There are many resources for personal time management including blogs such as CEO Blog - Time Leadership. But how do you manage time for an organization? According to Gord Schmidt, who has implemented time tracking for many Agora customers, key things to keep in mind for end user adoption are: Remember the end-user experience – keep the number of time buckets manageable and relatively small Cover the spectrum – if you are collecting data to cover all business hours of the day, make sure there are categories that cover non-project work. Divide those categories into work (example: training) and non-work (example: sick time) If project schedules will be used to create tasks on the timesheets, train project managers on how to build schedules that make it easier for people to enter timesheets including: Distinctly and succinctly named tasks Estimated work and duration of tasks needs to be not too long and not too short Tasks should be scheduled (or rescheduled to) when they will realistically be done Senior executives need to own the implementation which means they should: Actively talk about the benefits Follow up to ensure compliance Define the process for resources entering time and managers approving time and ensure to handle the common exceptions (people away, fixing mistakes, etc.) Hands-on training for the users Finally, understand what the key objectives are for your people. Report on time by mapping them to those objectives. As well, ensure a feedback mechanism is in place for the employee. Can they see what they spend their time on last week vs. their peers? Can you see a trend over time? Can they see actual time allocations vs where they are expected to spend their time? By putting the same effort into time management as organizations do for financial management people’s behaviours will be focused on what’s important and corrective actions can be taken sooner.

How to define metrics that stick

 I hear and I forget. I see and I remember. I do and I understand. Confucius (680-740) I am often asked by customers about what industry metrics should be used for their dashboards and scorecards. The thought being that implementing best in class metrics will make them a best in class organization. However, defining and implementing metrics doesn’t necessarily mean the organization will follow along. People have to buy into the objectives and metrics. The best way to do this is for them to define them themselves. By participating in defining their strategic objectives and metrics they become the owners of them. They will understand the intent of them much better than something imposed on them. As well, the people that know your organization best are those people within the organization. They know what works and what doesn’t. In terms of finding the objectives that will make an impact and metrics to track them, your people know best. Industry metrics are a great for sources of ideas. However, when it comes to your own organization, if you want your metrics to stick you need metrics that are driven from your people.

Bernoulli and Choosing Metrics

  “One should not appraise human action on the basis of its results.” -  Jakob Bernoulli (1654-1705) This statement has significant implications on devising metrics for Performance Management. In fact, at first glance it could be interpreted as why should we measure at all. Let’s look at the background on Bernoulli’s comment and see how we can use his concepts to derive better metrics. Jakob Bernoulli was one of the founding fathers of probability theory. In particular, he developed the concept now called Bernoulli trials, whose result can be either of two possible outcomes, “success” or “failure”. Situations like the following are considered Bernoulli trials: Did the project complete on time? Did the sales team hit their sales targets? Did my child get over 90% on his exam? Did the CEO qualify for a bonus? Did the Toronto Maple Leafs win the game on Saturday? Each of these situations has a probability of success (and a probability of failure which is 1 - the probability of success). For each of these situations, no matter how hard the person (or group) tries and no matter what their skill there is still a chance of failure. Conversely, no matter how much the lack of effort or skills of a person (or group) there is still a probability of success. We have seen many times in real life a CEO getting an underserved bonus, a star pupil blowing a test and a weaker team winning the championship (Leaf fans…don’t get excited, the Stanley cup is still far away). To understand how to minimize chance we need to go to Bernoulli again. Bernoulli’s Law of large numbers states that over time, the average will trend towards the expected value. Looked at another way, with a greater number of observations (e.g. number of games, number of tests), the average will be more representative of the underlying person or group’s skills. Unfortunately, people tend to behave in the opposite fashion. People often assume a small sample is representative of the underlying situation when in fact it is too small to be reliable. This has been observed so many times it has sarcastically been called the Law of small numbers. It has also been called Hasty Generalization. This behaviour reinforces the importance of creating metrics that remove this human bias. Considering what Bernoulli has taught us, let’s look at characteristics of good metrics. Closely aligned with the desired action. Increased sales is what you desire from a sales rep but it is not directly tied to the actions or behaviours you want from the sales rep. If you know that number of sales calls in a week is a behaviour that leads to increased sales it should be considered as the metric. Number of sales calls in a week is a better metric because it is closely tied to the desired action. High number of observations. Repeated observations (measurements) increases the representation of the underlying situation. Looking at many projects delivered by a project manager rather than just one is more representative of his ability to delivery on time. Variety of independent metrics. In addition to minimizing chance by averaging across observations, chance can be reduced by measuring a number of independent metrics. The independent aspect is important otherwise correlation between the metrics will distort the true picture. Measuring a student through multiple methods such as class participation, oral communication and a written test measures comprehension better than a single multiple choice exam. The desire by people to look at results based on small observations is strong. It requires courage to change the way that results are looked at. Hopefully drawing on a mathematical foundation for metric design will remove some human bias. I wonder what Bernoulli would have thought if he knew his ideas from over 300 years ago are being applied to devising metrics to running organizations today.